Jones hit a pair of first-pitch solo homers and Jonathan Schoop launched a tiebreaking, two-run shot that sent the Orioles over the Rays 4-3 on Wednesday night.
"Felt good to get the ball out of the infield," said Jones, who entered with just one homer this season. "You've got to keep swinging, got to stay after it."
Schoop's drive in the seventh inning off Brandon Gomes (2-2) put the Orioles ahead 4-2.
"It's really nice, especially to come through there and to get a `W," Schoop said.
Jones hit leadoff homers in both the second and fourth off Cesar Ramos. It was his fourth multihomer game.
"I just put some good swings on some pitches that I should have driven and I drove them," Jones said. "That's the rule of thumb. If it's right down the middle you might as well hit it."
Jones didn't get the chance to face Ramos a third time when Tampa Bay manager Joe Maddon brought in Gomes, who got a fly from the Orioles slugger with two outs in the sixth.
"You didn't want to keep pressing your luck right there," Maddon said. "Jones had some good at-bats. I loved Gomes on that part of the batting order."
David DeJesus homered for the Rays, who have dropped the first two games of a three-game series and are 0-4 overall against the Orioles this season.
One day after working out of a bases-loaded, one-out jam in a 5-3 win, Hunter again ran into trouble in the ninth.
"We're winning ballgames and that's the bottom line," Hunter said. "You would like it to go a little smoother. I'd liked to have a clean frame, make it nice and easy for everybody, but that's not the way it's going right now."
DeJesus cut the Rays' deficit to 2-1 on a solo homer off Bud Norris in the fifth. DeJesus has three homers in 15 at-bats against the Orioles right-hander.
Baltimore catcher Matt Wieters was out of the starting lineup after making a trip earlier in the day to Pensacola to have his sore right elbow examined by Dr. James Andrews. While it was determined that Wieters would avoid the disabled list, he will likely be limited to designated hitter duty for a period of time.
Tampa Bay tied it at 2 on a pinch-hit RBI single by Sean Rodriguez during the sixth.
Both Norris and Ramos gave up two runs and three hits in 5 2/3 innings.
Baltimore purchased the contract from Triple-A Norfolk of catcher Caleb Joseph, who went 0 for 3 in his major league debut. ... Baltimore RF Nick Markakis extended his hitting streak to 14 games with a first-inning single. ... Orioles 1B Chris Davis (strained left oblique) ran sprints, threw and hit in an indoor cage. ... Baltimore OF Francisco Pequero (right wrist) is set to start a minor league rehab assignment Friday with Norfolk. ... Orioles RHP Ubaldo Jimenez (1-4) and Rays LHP David Price (3-2) are the scheduled starters for the series finale Thursday night. ... Price has 55 strikeouts and five walks in 48 2/3 innings this season. ... The Rays are 6-1 in one-run games.