Lionel Messi had scored on 15 minutes, running onto a long pass from Ever Banega and shooting past Francis Uzoho.
But four minutes into the second half Javier Mascherano was adjudged to have brought down Leon Balogun and Victor Moses scored the penalty to leave Argentina on the brink until Rojo's intervention.
Even then Argentina were relying on Iceland not winning elsewhere but they lost 2-1 to Group D winners Croatia.
Argentina finish as runners-up to play France on Saturday in Kazan, a day before Croatia take on Denmark. Nigeria end third, on three points, ahead of Iceland, on one.
Argentina had started brightly with Messi leading the charge before he was chopped down by John Obi Mikel as he ran at the Nigeria defence.
Both Nicolas Otamendi and Mascherano flew into tackles, the latter to cover his own mistake when he gave the ball straight to Kelechi Iheanacho,
Sampaoli’s players were fired up but they were anxious too. Messi settled the nerves on the quarter hour when he put Argentina ahead.
Banega played a long ball over the top and Messi took it in his stride with two touches and a right-footed shot past Uzoho.
Argentina’s tails were up now. When Angel Di Maria raced away he was clipped by Balogun on the edge of the area.
Argentina players wanted a red with Di Maria through on goal but the presence of covering Nigeria defender Kenneth Omeruo ensured the card was yellow.
Messi took the free-kick and sent it past Uzoho but the ball came back off the post and was cleared to safety.
Argentina were a different side to the one that folded against Croatia and Franco Armani, replacing Willy Caballero in goal, charged confidently out of his area to clear as Nigeria threatened on the counter.
Rojo had also been called-up and he had brought greater urgency to the Argentine defence although one high kick in the penalty area left Nigeria complaining to referee Cakir Cuneyt.
He had caught Ahmed Musa full in the face but the shouts for a penalty were ignored.
Nigeria did get their penalty, five minutes into the second half when Mascherano was penalized for grappling Balogun to the ground.
He complained his opponent leaned back into him but his arms were around Balogun and Moses rolled in the penalty before a couple of somersaults to celebrate.
Nigeria wanted another spot-kick with 15 minutes left but Cuneyt ruled that Rojo had headed the ball on to his own arm meaning it was not a penalty.
Argentina were still a goal away from going through when Gonzalo Higuain blasted over from the edge of the six yard box.
It looked as if their last chance had gone but from Mercado’s cross Rojo finished to send them through and Nigeria out.
But four minutes into the second half Javier Mascherano was adjudged to have brought down Leon Balogun and Victor Moses scored the penalty to leave Argentina on the brink until Rojo's intervention.
Even then Argentina were relying on Iceland not winning elsewhere but they lost 2-1 to Group D winners Croatia.
Argentina finish as runners-up to play France on Saturday in Kazan, a day before Croatia take on Denmark. Nigeria end third, on three points, ahead of Iceland, on one.
Argentina had started brightly with Messi leading the charge before he was chopped down by John Obi Mikel as he ran at the Nigeria defence.
Both Nicolas Otamendi and Mascherano flew into tackles, the latter to cover his own mistake when he gave the ball straight to Kelechi Iheanacho,
Sampaoli’s players were fired up but they were anxious too. Messi settled the nerves on the quarter hour when he put Argentina ahead.
Banega played a long ball over the top and Messi took it in his stride with two touches and a right-footed shot past Uzoho.
Argentina’s tails were up now. When Angel Di Maria raced away he was clipped by Balogun on the edge of the area.
Argentina players wanted a red with Di Maria through on goal but the presence of covering Nigeria defender Kenneth Omeruo ensured the card was yellow.
Messi took the free-kick and sent it past Uzoho but the ball came back off the post and was cleared to safety.
Argentina were a different side to the one that folded against Croatia and Franco Armani, replacing Willy Caballero in goal, charged confidently out of his area to clear as Nigeria threatened on the counter.
Rojo had also been called-up and he had brought greater urgency to the Argentine defence although one high kick in the penalty area left Nigeria complaining to referee Cakir Cuneyt.
He had caught Ahmed Musa full in the face but the shouts for a penalty were ignored.
Nigeria did get their penalty, five minutes into the second half when Mascherano was penalized for grappling Balogun to the ground.
He complained his opponent leaned back into him but his arms were around Balogun and Moses rolled in the penalty before a couple of somersaults to celebrate.
Nigeria wanted another spot-kick with 15 minutes left but Cuneyt ruled that Rojo had headed the ball on to his own arm meaning it was not a penalty.
Argentina were still a goal away from going through when Gonzalo Higuain blasted over from the edge of the six yard box.
It looked as if their last chance had gone but from Mercado’s cross Rojo finished to send them through and Nigeria out.