After Alexander Semin scored midway through the third period to give Carolina a 3-2 lead, the Hurricanes struggled in the final five minutes to allow the Columbus comeback.
"You can't make mistakes late in the game that are going to cost you," Carolina coach Kirk Muller said.
"We made two mistakes in the third. It was a game of errors and we made two and they made us pay for both of them."
Johansen took advantage of a Carolina turnover and scored on an odd-man rush with a pass from Nick Foligno at 15:58 of the third to make it 3-3.
Then at 17:26, Carolina executed a poor line change that allowed Skille to find open ice on the right wing that led to the game-winning goal.
Skille said the comeback was a team effort.
"We wanted to finish strong and we're the type of team that will never give up," Skille said. "Everyone wants to win and that was the key to that third period."
Mike McKenna, playing his fourth game with the Blue Jackets, finished with 30 saves and earned his first win of the season.
"It was a little different than the other games I've played this year," McKenna said. "What a fortuitous turn of events. That was about the best five minutes I've been a part of. Those two guys went in and put two perfect shots."
Jeff Skinner, who has 10 goals in his past 11 games and leads Carolina with 14, scored twice on the power play. Both came when the Hurricanes were down a goal to tie the game.
His first came with 1:11 left in the first period to make the score 1-1 on a 5-on-3. And he tied the game at 2-2 with 3:35 left in the second period.
Carolina has struggled since winning three straight earlier in the month. The Hurricanes have lost six of their past seven, though three were in overtime.
C Boone Jenner has points (goal, assist) in consecutive games and Anisimov has points (one goal, two assists) in three straight for Columbus. ... C Eric Staal has 21 points (six goals, 15 assists) in his past 19 games. ... The Hurricanes scored multiple power-play goals in a game for the first time this season.