Matt Joyce had an RBI single, with a second run scoring when center fielder Peter Bourjos was charged with an error for misplaying the hit. After Wacha walked Yunel Escobar and Ryan Hanigan to load the bases, Jennings drove in two with a single.
"That's a great feeling to come through and really kick-start the party," Joyce said.
Wacha allowed four runs, four hits and four walks in five innings.
"He just never really found his fastball, for whatever reason," Cardinals manager Mike Matheny said. "His timing was off and it was a fight for him."
Wacha had given up three runs or less in 13 straight starts to begin the season, which was the longest season-opening stretch by a Cardinals pitcher since a 14-game run by Lee Meadows in 1917.
Bourjos was robbed of an extra-base hit with the bases loaded and two outs in the fifth when right fielder Kevin Kiermaier made a full-extension diving catch on a liner.
"Kiermaier flying through the air, that's the play of the game," Maddon said. "Kiermaier's play was totally uplifting."
Bourjos called it unbelievable.
"Initially when it left the bat I didn't see anybody over there," Bourjos said. "But he covered a lot of ground."
The Cardinals are optimistic that ace Adam Wainwright's right elbow discomfort is not a serious issue. He returned to St. Louis for tests . ... Former St. Louis manager and Tampa native Tony La Russa threw the ceremonial first pitch. He will be inducted into baseball's Hall of Fame on July 27. ... Tampa Bay RHP Jeremy Hellickson (elbow) will make his second rehab start Thursday for Triple-A Durham. He could rejoin the rotation at the end of the month. ... Maddon was given the neon back entrance sign from Derby Lane track that he drove by daily en route to the ballpark. The track was a favorite spot of Rays senior adviser Don Zimmer, who died last week at 83. "My head is spinning right now," said Maddon, when he first saw the sign outside the clubhouse. ... The Cardinals signed high school pitcher Ronnie Williams, their 2014 second-round draft pick.
15,930 (46.7% full) - % is based on regular season capacity
Home Plate - Hunter Wendelstedt, First Base - Jerry Layne, Second Base - Mike DiMuro, Third Base - Mike Estabrook
The Rays scored four runs in the 4th inning, snapping a club record 31-inning scoreless streak.
They hadn't scored since the 8th inning Saturday against the Mariners.
Michael Wacha has allowed a .167 opponent average this season in at-bats that end in an offspeed pitch, well below the league average of .222 and better than his .221 rate last season. The Tampa Bay Rays have hit .193 in such at-bats this year, the third-worst rate in baseball.