The win snapped a two-game Senators' losing streak.
"It was a great effort by everybody and everyone was chipping in and doing their job and they got rewarded for it," Anderson said after collecting his league-tying sixth victory in his ninth game of the season.
"Overall I think this could have been a 4-1 game but it was unfortunate that Buffalo got some lucky bounces and they ended up in the back of the net."
Tyler Ennis, Jordan Leopold and Jason Pominville scored for Buffalo. Jhonas Enroth made the start in place of Ryan Miller and is still searching for his first win in more than a calendar year. He finished with 20 saves for the Sabres, who have dropped three straight.
"We need better goaltending. There's no secret to that," Sabres coach Lindy Ruff said following his team's third straight loss and seventh in their past eight games.
"I think (Enroth) will admit that he has to have a couple of those. Now you have to look to bounce back. I tried to let him bounce back in the game and I thought the team answered the bell.
"Right now we've got to worry about winning a game, whatever that takes."
Buffalo's Thomas Vanek leads the NHL in scoring with 19 points but he was held off the score sheet for the first time in nine games.
Pominville cut Ottawa's lead to 4-3 at 13:55 of a third period that had four total shots up to that points. The Sabres pressed furiously in the dying moments but were unable to put a fourth puck past Anderson.
Buffalo fell behind 4-1 at the 8:12 mark of the second period after Neil's one-timer beat Enroth, but quickly got that goal back with Leopold redirecting a puck past Anderson from the side of the goal.
The goal was the first that Anderson has given up after the first period all season.
"We battled back and gave ourselves a chance. We've just got to bear down and play a complete game," Pominville said. "Our first period wasn't good enough."
The Senators jumped out to a 3-1 after the first period. Karlsson opened the scoring at 8:17, just 27 seconds into the game's first power play when his point shot eluded Enroth.
The lead was very short lived though, as Ennis collected a rebound and put a backhand off the far post and in behind Anderson just 21 seconds later.
But Phillips and Alfredsson scored a little more than three minutes apart later in the period to put the home side up 3-1. Phillips took a point shot that dribbled across the goal line after Enroth got a piece of it.
Alfredsson then took a snapshot from the slot off a feed from Mika Zibanejad that beat Enroth cleanly.
"Once it got to 4-3 I thought we did a really good job of keeping them on the outside and doing the right things defensively and holding on to the puck offensively. That was nice to see," Alfredsson said.
Ottawa finished the night 1-for-2 on the power play, while Buffalo was scoreless on four chances.
Christian Ehrhoff, Matt Ellis and Andrej Sekera were the scratches for the Sabres while Kaspars Daugavins and Sergei Gonchar sat out for the Senators. ... Four players in Tuesday's game were all selected in the first round of the 2008 NHL draft in Ottawa. Cody Hodgson was taken 10th overall, Tyler Myers (12th), Erik Karlsson (15th) and Tyler Ennis (26th).