Goins scored 23 points on 9-of-14 shooting with 11 rebounds, while Moore had 22 points on 9-of-15 shooting for the Dukes (4-6).
The Spartans (2-6), playing their first game in over two weeks, were led by Trevis Simpson's 24 points.
A.J. Davis's layup sparked a 13-1 James Madison run about 6 minutes in for a 21-9 lead.
UNC Greensboro cut James Madison's 49-29 halftime lead down to 64-61 with 7:05 left, but Goins hit two baskets and the Spartans never got within five points again.
James Madison is 5-0 all-time against UNC Greensboro as the teams met for the first time since 1999.
The game was a homecoming for UNC Greensboro coach Wes Miller, who played his freshman year at James Madison in 2002-03 before transferring to North Carolina.
Mississippi State freshman guard Malik Newman will withdraw from the NBA draft, but sources told ESPN he might explore other opportunities besides returning to Mississippi State.
Seton Hall sophomore Isaiah Whitehead will stay in the NBA draft and not return for his junior season.
Washington State's trio of Josh Hawkinson, Conor Clifford and Ike Iroegbu have withdrawn from the NBA draft and will play their senior season with the Cougars.