Toronto scored twice off Rivera in the ninth before Blue Jays outfielder Travis Snider doubled home the winning run in the 10th, beating the Yankees 6-5 on Tuesday night.
"That's all the inspiration we need as players is to see what we did in the ninth inning against Mariano," Snider said. "To be able to give ourselves a chance, to go out there and just have that shot, I'm very thankful for it."
Rivera suffered his first blown save in eight chances before Toronto won it against Ivan Nova (1-2).
"We got it to where we wanted to and we just didn't hold onto it," Yankees manager Joe Girardi said.
Edwin Encarnacion opened the 10th with a single. After Jayson Nix and Yunel Escobar flied out, Snider drove a double to right-center, scoring Encarnacion with the winning run. Snider, who went 1 for 6, tossed his helmet away in celebration as the winning run scored.
Jon Rauch (1-1) pitched one inning for the win as Toronto snapped a three-game losing streak and won for the first time this season when trailing after eight innings.
Jose Bautista hit a solo home run for the Blue Jays, his fourth.
Toronto trailed 5-3 to begin the ninth against Rivera, who hadn't allowed a run in 19 consecutive April appearances.
"Going in there I had it in mind that everything was going to be fine," Rivera said. "Today wasn't a good day."
Escobar led off with a double to center, took third on a grounder and scored on a wild pitch that was also ball four to Bautista.
"I held the ball too long," Rivera said. "I wanted to go away and it was way too away and ended up being a wild pitch. I was upset with that one."
Adam Lind followed with a single, putting runners on the corners for John McDonald, who drove in the tying run with a squeeze bunt up the first base line. Teixeira charged the ball and flipped to catcher Russell Martin, but too late to get the sliding Bautista.
"It's just a great play," Teixeira said. "That's why you run it because when it's run well, it's almost impossible to defend. I thought there might have been a chance to get him so I gave it a quick flip but it was a good play."
The rally snapped Rivera's streak of 17 1/3 scoreless innings in April, a run that stretched back to April 30, 2009.
"That's the thing about Mo, he's not perfect but he's close to it," Teixeira said.
Bautista got the Blue Jays started with a solo homer off Burnett in the first, his fourth, but the Yankees answered with a two-run second.
Toronto tied it in the third, taking advantage of a throwing error by A.J. Burnett. Bautista drew a one-out walk, took second on a wild pitch, then escaped from a rundown and advanced to third after Jeter couldn't handle Burnett's throw on Lind's comebacker to the mound. Aaron Hill followed with a sacrifice fly.
"I just got rid of it too early," Burnett said of his errant throw. "I could have run him all the way back to second, looking back on it."
After Granderson singled to open the sixth, Teixeira followed with a drive to center.
Toronto clawed one back in the bottom half. Hill singled and stole second but had to leave with tightness in his right hamstring, with John McDonald coming on to run. Jose Molina flied out and Corey Patterson walked before Encarnacion hit an RBI double.
Burnett left after walking Nix to load the bases, but Robertson fanned Escobar and Snider to escape the jam. A frustrated Snider broke his bat over his thigh, then slammed the pieces into the dirt after going down swinging to end the inning.
Yankees third baseman Alex Rodriguez was held out of the starting lineup for the second straight game because of a sore left side. Rodriguez said he expects to return Wednesday.
New York also leads the majors in multihomer games (10). ... Toronto activated RHP Frank Francisco (right shoulder) off the 15-day DL and optioned RHP Jesse Litsch to Triple-A Las Vegas. ... Yankees RHP Carlos Silva threw 40 pitches during his second bullpen session at New York's spring training complex in Tampa and could face hitters early next week. Silva, released by the Chicago Cubs in spring training, signed a minor league deal with New York on April 9. ... Yankees C Francisco Cervelli, coming back from a broken left foot, caught Silva. Cervelli is playing in extended spring training games.