Simon Grayson made one enforced change to his starting line-up for Leeds United's second ever league trip to London Road, Mika Vayrynen coming in to make his full debut in place of the injured Robert Snodgrass.
And the game got off to the perfect start for Leeds with Andy Keogh scoring inside four minutes. Adam Clayon played the ball into the box and Keogh showed tremendous skill to fire home to give United an early lead.
It was a great start by United and Keogh threatened again moments later before finding his route to goal blocked by some good defending.
The opening 10 minutes saw a return to the pressing and closing down that served United so well at Doncaster in the last away game with Peterborough being denied space and time.
The home side serve notice of their threat, though, on 13 minutes when Mark Little whipped in a low cross from the right which George Boyd narrowly failed to get a touch on. And Lee Tomlin should have done better in front of goal when Emile Sinclair escaped down the left and pulled the ball back.
United's response was for Ross McCormack to send an angled shot wide while Clayton also sent the ball across the face goal.
But Peterborough were level in the 23rd minute when Gabriel Zakuani rose well to head home a looping corner kick at the far post.
Leeds almost replied immediately, courtesy of Keogh, but Paul Jones spread himself well to deny the Irishman. Paul Connolly also delivered a ball forward for Vayrynen, but the Finn was unable to get on the end of the delivery Jones was able to collect.
The game took a twist in the 37th minute when Tomlin went straight through the back of Keogh in a challenge midway in the Peterborough half and the referee pulled out a straight red card. Keogh needed treatment following what looked like a reckless challenge.
As the game edged towards half-time, the home defence made a hash of a Connolly ball into the box and United won a corner. Claytron produced a good delivery, but Jones rose well to collect with Keogh waiting to pounce.
And deep into first half stoppage time, Jones was again called upon in a one-on-one situation and got a vital hand to a McCormack effort as the Scot looked to slip the ball into the far corner.
United also started the second half on the front foot and some constructive build-up play ended with Jonny Howson sending a powerful strike over the top.
Within four minutes of the re-start Grayson also made his first change of the afternoon, Lloyd Sam replacing Vayrynen who had taken a knock during the first half. Darren O'Dea also launched an effort wide from 40 yards out.
Clayton made no mistake on 54 minutes when he received the ball on the edge of the area, though. The youngster controlled the ball, picked his spot, and hammered the ball beyond the reach of Jones
The game had steadily been building in terms of challenges and the final half-hour was threatening to become a real feisty affair as free-kicks became common place.
United were stlll playing some nice stuff and McCormack was denied by some good defending after being picked out at the far post by Connolly.
As the game ticked into the final 15 minutes, United were still in control, and you couldn't help but feel Grayson's men were one clinical move away from adding a third to put the game beyond doubt.
Ramon Nunez was brought on for McCormack and there was a short stoppage while Clayton received treatment as the clock started to count down.
But disaster struck for Leeds two minutes from time. A high ball was pumped into the box and while no one dealt with the ball, Little stole in on the melee of Leeds defenders to stab the ball home.
The hosts hadn't really had a look in during the second half and it was another bitter blow for Leeds, who had surrendered a lead against Coventry in midweek.
And there was more pressure to follow deep in stoppage time when Rachubka had to be alert save a free-kick from distance as the hosts looked to snatch the game.
But it was Leeds who snatched the game when Darren O';Dea smashed a strike against the underside of the bar in the fifth minute of stoppage time to snatch the points. Luciano Becchio was on the line to make sure, although video replays confirmed that the Irishman's initial strike had crossed the line.
It was a dramatic winner for Grasyson's men and, on the balance of play, was deserved.