Bayless was 6 for 6 on 3s -- five in the first half. He made 8 of 9 shots overall. Ed Davis had 14 points and seven rebounds for Memphis (4-2).
Paul Millsap scored nine of the Hawks' 14 first-quarter points. He finished with 19 points and eight rebounds before leaving the game with a sore left Achilles as a precaution late in the third quarter.
The Hawks (1-4) went up 1-0 14 seconds into the game and never led again.
Bayless' shot at the buzzer gave Memphis a 53-40 lead at halftime.
Toronto fans have nothing to cheer for in Game 5.
Take a behind-the-scenes look at the Thunder's dominating Game 4 performance against the Warriors.
LeBron James gets a steal and goes to the other end for the soaring two-handed slam dunk.