Frazier also pulled down 10 rebounds in a successful return from an Achilles tendon injury that sidelined him for all but four games last season.
Wagner trimmed a 12-point halftime deficit to 33-30 when Penn State missed its first eight field-goal attempts of the second half.
The Nittany Lions responded with a 21-10 run capped by Frazier's 3-pointer to lead 54-41 with 8:21 to go.
The Lions didn't allow Wagner to within nine points after that.
Both teams shot poorly from the field -- Wagner at 29.9 percent and Penn State at 35.8. But Penn State converted 6 of 16 3-pointers as opposed to just 2 of 23 for Wagner.
Mario Moody's 11 rebounds enabled Wagner to outrebound Penn State 44-43, but the Seahawks missed a number of layups and other shots from the paint in their 20-for-67 shooting performance.
Nine players scored for Penn State as Miami, Ohio transfer Allen Roberts chipped in with seven rebounds and four points and forward Ross Travis added eight points.
It was a relatively clean game as Wagner committed just five turnovers to Penn State's six.
Wagner's Moody, who had eight points, was called for a technical foul in the first half for pointing at Penn State's student section after scoring a basket.
Ortiz, the Northeast Conference's defensive player of the year the past two seasons, scored 13 second-half points and Rivers had 10 but they were the Seahawks' only consistent scorers.
DouDou Gueye, a 6-foot-10 center from Senegal, has signed with Ball State as a graduate transfer from South Carolina State.
We have changes at the top of the draft and a shakeup in the lottery, with teams starting to make their final decisions. Get the latest in Chad Ford's Mock Draft 9.0.
Duke freshman point guard Derryck Thornton Jr. is transferring to USC, his father announced on Friday to Scout.com.