Bianchi drove in two runs with a 10th-inning single up the middle as the Brewers beat the St. Louis Cardinals 6-4 on Saturday night.
Bianchi made his second start since coming off the disabled list on May 2 after he was sidelined by bursitis in his left hip.
"It feels great," he said. "Biggest thing was we got a win."
The Brewers snapped a four-game losing streak and improved to 3-13 in May. It also gave John Axford (1-3) his first win since Sept. 1, 2012.
Jim Henderson worked a perfect inning for his eighth save in eight chances.
Getting the start over the struggling Rickie Weeks, Bianchi singled just past diving shortstop Pete Kozma to give him his first two RBIs of the season. He had only five at-bats before Saturday and one hit in seven games.
The Brewers loaded the bases with one out in the 10th against Joe Kelly (0-2). Logan Schafer grounded into a fielder's choice to a pulled-in Kozma, who threw home to force out Ryan Braun and set up Bianchi's game-winner.
"You get a guy at third base with no out, all of a sudden the next two guys make outs and you're like, `OK," Milwaukee manager Ron Roenicke said. "And we get a big hit."
Kelly allowed runs for the first time in five appearances. He allowed two hits and a walk in one inning for the Cardinals, who fell to 0-3 in extra-inning games.
"I just gave up some hits and then I got the ground balls I needed to. I thought I was going to get out of it, and made another good pitch," Kelly said. "It found a way to stick up the middle."
Axford gave up one hit and two walks in 1 1/3 innings to earn the win. Axford struck out two, including pinch-hitter Daniel Descalso with the bases loaded to end a ninth-inning threat.
He also threw as hard as he had in some time, clocking one pitch at 97 mph.
"I've been feeling good lately," he said. "I looked in the mirror and got a little deeper inside myself the last few outings. I think I dug in and realized I need to do a little bit more."
The Cardinals used four consecutive singles with two outs in the sixth inning to tie it 4-4. Matt Carpenter's hit chased Milwaukee starter Marco Estrada and drove in Kozma who started the rally. Jon Jay's single brought in Ty Wigginton.
Braun gave Milwaukee a 4-2 lead in the fifth with his second single of the game.
The Brewers broke through in the third with a two-out rally. Segura's fourth triple of the season drove in Bianchi and Norichika Aoki to make it 2-2. Braun gave Milwaukee a 3-2 lead with his first single.
St. Louis scored twice in the second but left the bases loaded. Kozma drove in Allen Craig, who walked to start the inning. Carpenter's single gave the Cardinals a 2-0 lead.
Estrada gave up four runs, seven hits and four walks over 5 2/3 innings. He struck out two.
Lance Lynn lasted just five innings. He gave up four earned runs on eight hits and two walks as his ERA rose to 3.27 from 2.88.
He struck out three, his fewest since his first start of the season when he was lifted after four innings at Arizona on April 3.
"I would have liked to pitch better today to help us win the game," Lynn said. "But everybody else picked up the slack."
Kozma has a five-game hitting streak (7-of-19, .368). . Jay has 14 RBI in the past 14 games. . Aoki has multiple hits in five of six games (13-of-26, .500). . Brewers LHP Chris Naverson threw curveballs on Friday for the first time since going on the disabled list April 7 because of a strained left middle finger. Roenicke said it was the final test before he is cleared to throw in a rehab game. ... Milwaukee LHP Tom Gorzelanny is on pace to return from the disabled list at the end of the month. He is set to throw a bullpen session Sunday and a simulated game Tuesday. He has been sidelined since May 8 because of tendinitis in his left shoulder. ... Former Cardinals OF So Taguchi provided color commentary for Japanese broadcaster NHK from its Tokyo studio.