It was the Bulls second consecutive win without an offensive touchdown. USF (2-4) is 2-0 in the American Athletic Conference after beating Cincinnati last week.
Kloss' 44-yard field goal with just over 4 minutes left capped a more than 7 minute fourth-quarter drive, and was the game winner. He has converted on nine consecutive attempts, including a 47-yarder in the first half.
Lyle McCombs had a career-high 164 yards rushing and a touchdown for Connecticut, which has lost five straight games to open a season for the first time since 1977. The loss spoiled the debuts of freshman quarterback Tim Boyle, who went 15 of 43 for 149 yards, and interim coach T.J. Weist, who took over the UConn program two weeks ago when Paul Pasqualoni was fired.
The Bulls were playing without injured quarterback Steve Bench and starting running back Marcus Shaw, who had been averaging more than 110 yards a game rushing before suffering a leg injury a week ago. They won despite being outgained 356-228.
The Bulls winning drive went 65 yards in 13 plays. USF just missed out on a touchdown when Steven Bravo-Brown got his hands on a 26-yard pass, but dropped the ball in the end zone.
UConn, which was held scoreless in the second half, got the ball back twice more. They drove to midfield with 6 seconds left, but a Hail Mary on the game's final play fell incomplete in the end zone.
Boyle, who turned 19 last week, got off to a slow first start for the Huskies. He hit his initial throw, a 10-yard completion to wide receiver Geremy Davis, then missed his next seven, including two that would have been touchdowns, but went off the arms of his receivers.
USF's defense then got its third touchdown in three weeks. Ryne Giddins knocked the ball away from Boyle, and Lynch picked up the loose ball and rumbled 44-yards to make it 7-3 late in the first quarter.
It took the Huskies just four plays to retake the lead as McCombs weaved his way for a 52-yard touchdown run, his longest run of the season and longest since 2011.
The junior's previous rushing high this season had been 76 yards. He had 135 in the first half of this one.
Kloss, who hit four field goals last week, capped a six-play second quarter drive to score 10-10. The junior has five field goals of 47 yards or more on the year, a new school record.
Connecticut had a chance to take the lead in the third quarter, but Julius Forte blocked a 47-yard field goal attempt, the second week in a row that South Florida has blocked a kick.
Connecticut's running game came in averaging less than 46 yards a game, but had 207 on Saturday.
But neither team could move the ball, especially in the second half. They combined for 16 punts during the game.
The win gives the Bulls their first 2-0 conference start since 2003, when the program was in Conference USA.
Nike Men's UConn Huskies Blue Football Icon Legend T-ShirtShop
If you're not a preseason Heisman favorite, you need to grab the nation's attention with big plays in big games. Here are the games where 10 hopefuls can make their mark on the Heisman race.
With so much uncertainty at offensive tackle, the Boilermakers will be counting on newcomer Jalen Neal to make an immediate impact.
David Sills, who once committed to Lane Kiffin at USC in seventh grade, is leaving West Virginia to purse his dreams of playing quarterback.
Alleged 1998 rape victim Brenda Tracy said she left Lincoln, Nebraska with an advocate for her cause in football coach Mike Riley.
There are plenty of questions surrounding Oregon State's offensive trio, but with some consistency this group should rank higher by season's end.
Ole Miss will get a chance to prove itself in the early going when it faces Dalvin Cook and a loaded Florida State team in Week 1.