Maurice Barrow chipped in 10 points and three steals for Fairfield, which had 12 steals. Needham finished with seven assists, more than Milwaukee (4) managed as a team.
The Stags (6-5) took a 23-21 lead into the break after a back-and-forth first half that featured six lead changes. Milwaukee (3-7) missed 8 of 10 shots and committed three turnovers over the first nine minutes of the second half as Fairfield opened up a 43-26 advantage and began pull away.
Kyle Kelm paced the Panthers with 13 points and 14 boards.
Justin Jenkins, who had nine points, went 0 for 8 in his first seven games this year, but has made eight of his last 12 field-goal attempts over the last three contests.
Mike Krzyzewski says he has good reason to be excited for next season thanks to a combination of Duke's incoming freshmen and a core of experienced upperclassmen.
Georgia guard Juwan Parker has been awarded a medical redshirt, leaving him with two years of eligibility, after missing the 2015-16 season with a partially torn Achilles tendon.
Forward Trevon Bluiett has withdrawn from consideration from the NBA draft and will return to Xavier for his junior season.