Drummond hit four free throws for Toledo (8-0) in the game's final minute, including two with 25 seconds remaining to extend the Rockets' lead to 80-77.
Drummond committed a foul with 1 second remaining, sending Robert Morris to the free throw line trailing by three. Karvel Anderson missed the front-end of the one-and-one and Toledo held on for the win.
Robert Morris shot 73.9 percent (17-for-23) from the field in the first half but led just 41-37 at halftime. The Colonials shot 63.3 percent for the game. Lucky Jones scored 25 for Robert Morris and Anderson added 23.
Maryland point guard Melo Trimble, considered a lock to leave the program before struggling in the second half last season, has withdrawn from the NBA draft and will return for his junior season.
Louisville's Chinanu Onuaku, who recently underwent a medical procedure to correct a heart rhythm issue detected during the combine, will keep his name in the NBA draft pool.
Andrew White III will return to the Nebraska men's basketball team next season after removing his name from the NBA draft.