A 3-pointer by Williams put the Pirates (11-20) ahead to stay, 57-55 with 2:22 remaining, the game's 16th lead change.
Pellum sank 8 of 10 free throws in the final 2 minutes to keep Hampton in front, and he finished with 16 free throws made in 20 attempts.
Aric Brooks scored 18 points and Kevin Thompson had 12 points and 17 rebounds to lead the Bears (9-20), who shot 12.5 percent (4 of 32) in the first half but trailed just 26-24 at halftime after making 14 of 20 first-half free throws to the Pirates' 6 of 14.
Christopher Tolson had 12 points and 10 rebounds for Hampton, which defeated Morgan State in last year's final. The Pirates will play top-seeded Savannah State on Wednesday.