Coach Brad Stevens is ready for a long process, but will take a few wins like Friday night along the way.
"We talked before about making winning plays in winning time, and we did. That was neat," Stevens said afterward. "Overall, I'm very pleased with the result but we still have a lot to work on. We're not that much different than if we wouldn't have won the game."
Trailing by three, the Magic had a chance to tie the game in the final minute, but Arron Afflalo's jumper came from just a half-step inside the 3-point line.
The win is the Celtics' second straight and extends their win streak over Orlando to nine games. They will try to make it three straight Saturday at defending champion Miami.
Avery Bradley added 14 points and eight rebounds for the Celtics.
Boston survived a 36 percent shooting night from the field and 2-for-14 performance from beyond the arc.
It was the fourth straight opponent that the Magic held under 40 percent. But after taking an early lead they let the Celtics back in the game by committing a season-high 20 turnovers that led to 25 points on the other end.
Afflalo led the Magic, who had their three-game win streak ended, with 23 points. Nik Vucevic finished with 14 points and 13 rebounds but was mostly bottled up by Boston's defense.
Magic coach Jacque Vaughn said the turnovers also contributed to Vucevic's off night.
"That didn't help us initiate our offense and then secondly, they did a good job of trying to deny him the basketball," he said. "You've got to give them credit for their defense."
Boston slipped out at a seven-point lead to begin the fourth quarter, but the Magic fought back to tie it at 75.
The cushion grew again when Bradley stripped Victor Oladipo at midcourt and sprinted ahead for a dunk and eventual three-point play to put the Celtics back up 81-77 with under 6 minutes to play.
Afflalo knocked down a 3-pointer and was fouled by Bradley with 1:38 left. He promptly completed the four-point play to pull the Magic within 86-84.
The Magic weren't done. Afflalo saved a shaky possession with a driving layup and three-point play to cut the Celtics' lead back to 88-87 with 26.5 remaining.
Boston tried to run down the clock and Bass was eventually fouled and was able to connect on both of free-throws.
Following a timeout, Afflalo got loose and banked in a jumper from the wing, but a review confirmed it he was inside the 3-point line.
Bradley was fouled and connected on 1 of his 2 attempts. Oladipo then missed a desperation 3-pointer off the backboard at the buzzer.
"We stuck together when we were missing shots, a lot of easy shots," Celtics guard Jordan Crawford said. But everybody stuck together as a team."
Afflalo said the loss stings because they felt it was a throwback to lessons they thought they learned a year ago.
"This team has been through a lot last year in understanding how valuable wins are," he said. "Outside of that opener (10-point loss to Indiana), we've lost two games now that we felt we should have won. So we are learning in a process, but I think we came to play.
"I think the focus just needs to be sharper for a team that's trying to be a team that just won three in a row."
With the college basketball season opening on Friday night, Stevens said that he has stayed in tune with his former Butler team. "Yeah, I pay attention to it," he said. "You're always invested more in the people you know and the people that you're really tight with and friends with. I'll be interested to see if my family watches our game (Saturday) night or the Butler game."... Magic F Tobias Harris missed his sixth straight game with a sprained left ankle. ... Pro golfer Bubba Watson sat courtside at Friday's game.
Follow Kyle Hightower on Twitter at http://www.twitter.com/khightower .