Allen pinch hit in the ninth inning on Thursday, and his two-run homer lifted the Rays over the Los Angeles Angels 4-3.
"Success like this isn't going to come all the time," Allen said. "Baseball is a lot of luck. I was just there hitting and stayed within myself."
Allen, acquired April 19 off waivers from Oakland, homered off Jordan Walden (0-1) after B.J. Upton's one-out single. Allen drew a bases-loaded walk off the bench in Wednesday night's 3-2 win over Los Angeles in his first Tampa Bay plate appearance.
"To his credit, he has not played in a while," Tampa Bay manager Joe Maddon said. "This guy has not had many swings over the last couple weeks, and that speaks a lot to his makeup that we've heard about."
Brandon Gomes (1-0) worked a 1-2-3 ninth, helping the Rays win their fifth straight.
"Those guys over there are going to make you earn every inch," Angels manager Mike Scioscia said of the Rays.
Albert Pujols went 1 for 4, ending his hitless streak at a career-worst 21 at-bats when he rolled a grounder up the middle to beat a three-man defensive shift. Pujols was thrown out attempting to stretch the single into a double.
With 445 home runs, Pujols has not gone deep in 19 games and 76 at-bats this season. He's homerless in 105 regular-season at-bats since Sept. 22, tying his longest drought: April 23-May 23 last year. The three-time NL MVP is hitting .224 with a .280 on-base percentage after leaving St. Louis to sign a $240 million, 10-year contract with the Angels.
Allen hit .200 with six homers and 18 RBIs in 52 games last season with Oakland and Arizona. It was his first homer since Aug. 27 off Boston's Jon Lester.
"He came up with a big swing on a guy throwing 97 (mph)," the Angels' Torii Hunter said.
"When it rains, it pours," Hunter said. "Right now it's pouring down."
Williams allowed two runs and five hits. Trumbo hit his third homer this season in the fifth, and added an RBI double one inning later.
The Angels had just one runner -- on Vernon Wells' second-inning walk -- through four against Moore. Trumbo opened the fifth with a homer that tied it at 1.
After Trumbo's two-out, run-scoring double, he scored on Wells' infield single for a 3-1 lead in the sixth.
Tampa Bay closed to 3-2 when Joyce tripled in a run in the sixth.
Desmond Jennings had a leadoff single in the first and scored for a 1-0 lead when Zobrist doubled. Jennings has a career-best eight-game hitting streak.
"Hard-fought game, one bad pitch," Walden said.
Allen got the home-run ball back and will give it to his mother. ... This was just Walden's second save opportunity this season. ... Williams' only other start against Tampa Bay came on June 8, 2004, with San Francisco. He picked up the win after allowing one run over seven innings. ... Los Angeles RHP Jered Weaver (3-0), who is 12-0 with a 1.84 ERA in 14 starts before May 1 since 2010, will go against Cleveland RHP Justin Masterson (0-2) on Friday night. ... Rays RHP James Shields and Texas LHP Matt Harrison, both 3-0 this season, are Friday night's scheduled starters.