Oliver finished with 19 points and 10 rebounds for the Flyers (16-12, 6-8 Atlantic 10 Conference), who have won two in a row.
Dayton led the majority of the first half and never trailed after halftime, but had a tough time holding off the Spiders. The Flyers had a seven-point advantage midway through the second half, then had their lead trimmed to 60-59 when Richmond's Cedrick Lindsay made a free throw with 6:11 left.
But the Spiders (17-12, 7-7) were unable to tie or pull ahead. Dayton's Josh Benson and Dyshawn Pierre each followed with a free throw, then Dillard converted a three-point play to push the Flyers lead to six and hold on.
Lindsay had 22 points and seven assists for Richmond.
Andrew White III will return to the Nebraska men's basketball team next season after removing his name from the NBA draft.
Mike Brey is looking at former Notre Dame players to fill two assistant jobs that have opened up with the departures of Martin Ingelsby and Anthony Solomon.
Purdue's 2016-17 prospects got a jolt Wednesday when forward Caleb Swanigan announced he will return for his sophomore season.