The victory breaks a five-game skid for the Astros, who had been shut out in their past two games.
Carter broke a 22-inning scoring drought for Houston with his solo homer in the second off Roberto Hernandez (4-10) that tied it 1-1. He connected on a full-count pitch from Jake McGee on his second home run to make it 4-1 in the seventh.
Carter leads the Astros with 17 homers and his 44 RBIs are tied for the team lead. He is the first Astro to reach 17 homers by July 3 since Lance Berkman in 2009. Just more than halfway through the season, Carter is one homer shy of the 18 Justin Maxwell hit all of last season to lead the club.
The Astros are enamored with Carter's power and impressed with his increased discipline at the plate recently.
"He has a chance to hit 30 home runs and drive in 100," Houston manager Bo Porter said. "When you see that kind of progression and you see the adjustments being made, you really say to yourself moving forward that this guy has a chance to be a middle-of-the-order bat who can be very productive."
He has reached base safely in 15 of his last 16 games and his hitting .315 with four homers and 10 RBIs in that span.
Carter was just glad he could help the team get back on track after a tough couple of games.
"It's nice to score some runs and get a win after the last two games where we didn't score and they outscored us by 20 runs," he said. "I'm just happy we came out with a win right there."
Norris (6-7) allowed six hits and one run with five strikeouts to break a three-game losing streak and pick up his first win since June 1. Jose Veras allowed a hit in a scoreless ninth for his 17th save.
"Bud was absolutely outstanding," Porter said. "I thought he threw probably the best game of the season this year. His slider just had unbelievable depth ... he's our No. 1 guy and he pitched like a No. 1 guy again tonight."
Hernandez allowed four hits and three runs in six-plus innings for his fourth consecutive loss.
Carter's homer in the second pinged off the facade on the wall above the Crawford Boxes in left field, giving the Astros their first earned run in 31 innings.
It also broke a team record-tying 24-inning scoreless streak for Rays pitchers. It was the third time in team history they'd reached the mark.
Jose Altuve singled with two outs in the third, but was caught stealing second to end the inning. Hernandez starting rolling after that and retired the next nine straight.
He ran into trouble in the seventh when Brett Wallace doubled with no outs before he was chased by a single from Jason Castro. He was replaced by McGee, and Carter sent his 98 mph fastball to the train tracks atop the wall in left field. It was Carter's second multi-homer game of the season and of his career.
"Carter hurt us," Tampa Bay manager Joe Maddon said. "He didn't miss two pitches. That one from Jake (McGee) was properly struck. That is back wall anywhere. That was incredible. You have to give them credit for the way they bounced back today, and their pitcher set up the whole thing."
Norris hasn't had much run support lately, with the Astros scoring just three runs combined in his last three losses. A relieved-looking Norris greeted Carter with a hug and a huge smile in the dugout after his homer put Houston on top.
The feeling Norris had when as he watched Carter's homer was simple.
"Pure joy," he said. "CC's (Carter) done it a few times for me, so we've just got to keep it going."
The Rays had trouble stringing together hits against Norris.
Their best chance to add to their run total came in the fifth. Jose Molina's comebacker sailed just inches above Norris's head for a single to start the inning. Norris then struck out Yunel Escobar and Molina was caught stealing second on the play. Norris walked consecutive batters, but escaped the jam by retiring Zobrist for the third out.
Escobar singled with one out in the seventh before Jennings grounded into a double play to end the inning.
The series wraps up Thursday when Tampa Bay's Chris Archer opposes Jordan Lyles. ... Houston DH Carlos Pena was back in the lineup after taking time to travel to the Dominican Republic following the drowning death of his nephew. He returned to the team on Tuesday after missing two games, but did not play. Pena said it was a difficult time for his family, but the support the team showed him made it a bit easier. "We have a lot of signs around here that we have family first and that's something that I saw firsthand, how they reacted to this situation," Pena said. "They did everything they could to make sure my family was all right and taken care of, which I appreciate."... Evan Longoria was Tampa Bay's designated hitter for the second straight game after missing the previous three games with plantar fasciitis in his right foot. Manager Joe Maddon said he doesn't know when the All-Star third baseman will feel good enough to play defense, but that's OK with him. "We'll just have to wait to get him on the field, but we have him as a DH we'll take it," Maddon said.