After Jason Pominville had several whacks at the puck from in close, Parise stuffed it past Rangers backup goalie Cam Talbot 1:03 into the third.
"Yeah, it's going to be important from here on out to learn how to protect leads and win games the hard way," Parise said. "Tonight, it was a hard game, the way they played. They were in our face all the time through the neutral zone."
Two nights after Kuemper failed to hold a three-goal lead against Edmonton, he bounced back for his 12th victory.
Derek Stepan scored for the Rangers, who buzzed around the net in the final seconds but couldn't get one past Kuemper. New York lost its second straight and for only the seventh time on the road in its past 24 games.
"After they scored their second goal in the third period, they sort of backed off a little bit on their forecheck, and we were able to get some chances there but weren't able to finish," Rangers coach Alain Vigneault said.
Kuemper's highlights included a save from close range on Derick Brassard with eight seconds left and a key stop on Rick Nash when the Rangers were on the power play late in the third.
After the game, Kuemper was asked whether he felt any added pressure after Tuesday night's collapse.
"No, you know what, maybe a little bit before the game. But once you get out there, it's just another game and you just want to do what you've done all year," he said. "I think as a whole we did a good job of that."
Niederreiter scored from the slot in the first after confusion between Nash and John Moore led to a turnover deep in the New York zone.
Stepan tied it on the power play at 3:19 of the second with a wrist shot that Kuemper never saw. Stepan had hit the post on a shot 15 seconds earlier.
It was the second straight game in which New York was held to a single goal.
"I don't think it was so much them; I think we're more worried about ourselves," Nash said. "We have to establish more offense, more shots. It's just not going our way right now. We've got to work through this."
Talbot -- starting the first game of a back-to-back in place of Henrik Lundqvist -- made 24 saves.
Both teams are trying to hold on to playoff spots as the season winds down.
The Wild are in the top wild-card slot in the Western Conference, while the Rangers are in a battle to finish among the top three teams in the Metropolitan Division in the East.
"What I liked was the two goals that we scored were goals that we can score in the playoffs," said Wild coach Mike Yeo, whose team has played in six straight one-goal games, going 3-1-2. "Those are the kind of goals that we have to score going forward. That was one of our keys to the game, we need a playoff-offense mentality. I still think we can be a little bit more aggressive to the net."
While calling the game from between the benches in the first period, Wild television analyst Mike Greenlay took an accidental stick to the face from New York's Mats Zuccarello. Greenlay got stitches during the intermission and called the rest of the game with a small bandage under his left eye. "Upper-body injury," Yeo joked after the game. ... Stepan has 10 points in his past eight games. New York lost for only the second time when Stepan scores. ... Parise's goal was his 23rd, one behind Pominville for the team lead.
Copyright by STATS LLC and The Associated Press
ST PAUL, MN - MARCH 13: Brian Boyle #22 of the New York Rangers shoots the puck against Jared Spurgeon #46 and...