Price, who has started each of Montreal's seven games, has five career shutouts. It was his first in 90 games since Nov. 11, 2008, when he stopped 28 shots in a 4-0 win over Ottawa.
"It was awesome -- it's just secondary, though," Price said. "We all wanted to come in and get the win, that's the biggest thing."
Canadiens left wing Michael Cammalleri, who assisted on Kostitsyn's second goal, said "that's probably all of our fault" when he was told of the shutout drought and Price's comment.
"It's like a guy saying that when he scores in the first period, "Ah, it's just a goal,' but it feels like the rest of the night you're playing with the house's money a little bit," Cammalleri said. "So that's a good feeling for him, for sure, and hopefully it gets him going here. He's been good."
Kostitsyn opened the scoring in the first and added his second goal of the game late in the second to put Montreal up by three. Benoit Pouliot also scored in the second for the Canadiens, 4-1-1 since losing their season opener in Toronto.
"Our team play was awesome tonight," Price said. "That's one of the best team games I think I've played behind in a while. They shut 'em down. It was a really good team effort out there."
Brian Elliott stopped 31 shots for Ottawa (2-5-1), which won 4-2 in Buffalo one night earlier.
"We looked a little bit tired," Senators coach Cory Clouston said. "We didn't have the jump that we had last night and they came hard on us and our defense couldn't handle some of the pressure at times."
After reaching 1,000 career points with a hat trick Friday, Daniel Alfredsson was disappointed with his team's latest game.
"It feels terrible to have an effort like this at home coming off a big win," the Ottawa captain said.
Tomas Plekanec played his 400th game for Montreal. He had an assist on Kostitsyn's first goal of the game 10:54 in as the Canadiens held an early 7-0 advantage in shots.
"I liked our start," Cammalleri said. "Our energy was good and our execution was good."
Price made a fine glove save on Chris Kelly's backhander moments after the Senators had a potential tying goal disallowed at 13:27. Referee Kevin Pollock pointed to the net but a video review clearly showed that Ottawa's Nick Foligno kicked the puck into the net with his right skate.
After sticking out his left pad to deny Milan Michalek's goalmouth shot just before the midway mark of the second, Price protected the Canadiens' one-goal lead once again at 11:38 when he stuck out his left glove to stop Chris Campoli's slap shot from the slot.
The Canadiens, 0 for 4 with the man advantage, made it 2-0 on Pouliot's first of the season 13:42 into the second. Pouliot put a backhander past Elliott 4 seconds after Matt Carkner's interference penalty expired.
Montreal is 1 for 24 on the power play this season.
Kostitsyn increased the lead to three with his second of the game and fourth of the season with 1:23 left.
Price has recorded three of his five shutouts against Ottawa. ... C Dustin Boyd and D Ryan O'Byrne were healthy scratches for the Canadiens. ... Alfredsson has 101 points, including 29 goals and 72 assists, in 77 games against Montreal. ... Senators C Jason Spezza missed his second straight game because of a groin injury.