A late goal from striker Pieros Sotiriou gave Cyprus a 2-1 win over Gibraltar at Estadio Algarve.
On 37 minutes, with the game goalless, Giroud picked up the ball just inside the box.
The victory moved Sweden to the top of Group A, level with second-place France on 13 points.
Sweden did get the match back on level terms though before halftime when Jimmy Durmaz tied it at 1-1 two minutes before the break.
Stabbed Millwall fan recounts fight with London Bridge attackers
And during the London Bridge terror attacks over the weekend, he invoked the strength of the Lions and became a hero. Recalling the events of Saturday night, he said "They had these long knives and started shouting about Allah".
However the France captain turned villain with just seconds remaining as he attempted to bring the ball out of defence before a terrible miskick found Toivonen, who expertly dispatched into an empty goal to claim a big win. With so much talent at their disposal, most people just assumed France would qualify easily for the tournament.
He was around five yards inside his own half.
France dominated the closing stages but they could not find the breakthrough they needed and were made to pay after Lloris made an terrible error.
It appeared as though any hope of a victor had evaporated when a late France corner was cleared, but having gone all the way back to Lloris the Tottenham Hotspur keeper then came out of his area and only succeeded in sending his clearance straight to Toivonen.
The France 'keeper will watch replays in horror after he dribbled out of his box and made a terrible cut back intercepted by Toivonen, who lofted a snap lob from just inside his half and watched Lloris lose his desperate run to stop the ball rolling into an empty goal.
In today's match of UEFA 2018 World Cup Qualification Sweden and France faces against each other at Friends Arena, Stockholm, Sweden.