Millsap had 19 points, 13 rebounds and 10 assists to lead the Hawks, who have bounced back after losing 14 of 15 to strengthen their hold on the final playoff spot in the East.
The Hawks overcame an early 13-point deficit in a game that went back and forth in the final quarter. Millsap missed a chance to win it at the end of regulation, but Teague knocked down four free throws in the closing seconds to clinch it for Atlanta.
DeMar DeRozan led the Raptors with 29 points, but he had two crucial turnovers in the last minute of regulation, getting called for an offensive foul and slipping on another drive, losing the ball out of bounds.
The Hawks went ahead for good when Millsap found DeMarre Carroll all alone under the basket for the layup, which put Atlanta up 111-109 with 2:37 remaining and gave Millsap his 10th assist.
Toronto went more than 3 1-2 minutes without scoring in the overtime, allowing the Hawks to hang on.
Kyle Korver added 16 points for Atlanta.
DeRozan had a double-double, pulling down 10 rebounds, while Kyle Lowry chipped in with 20 points.
The Hawks got off to a miserable start, shooting just 6-of-24 in the opening period and falling behind 22-9.
But the Raptors failed to take advantage of the poor shooting, and the Hawks bench got the home team back in the game.
Shelvin Mack knocked down a buzzer-beating 3-pointer at the end of the first quarter, cutting the Raptors lead to 26-17. Then, Atlanta started the second with a quick 8-0 run and, just like that, it was a ballgame again. Cartier Martin swished a pair of 3s, sandwiched around Mike Scott’s driving layup.
The game was tied 49-all at halftime. Neither team could break away in the third, the margin never more than four points, and there were 10 lead changes in the final period.
NOTES: Toronto C Jonas Valanciunas left the game with a lower back sprain and didn’t return. ... The Hawks lost rookie G Dennis Schroder, who sprained his left ankle while hustling back on defense in the fourth quarter. ... Martin went crashing over the Hawks bench chasing a loose back in the second quarter. The crowd gasped, but Martin popped right back to his feet. ... Carroll wasn’t so fortunate. He took an elbow near his left eye and had to be taken back to the locker room to get four stiches. He finished the game with a bandage over his wound. ... Atlanta C Pero Antic and Lowry picked up double-technicals in the third quarter for jawing at each other after a play that left Antic sprawled on the court. The big man tried to draw a charging foul on Terrence Ross, and both players went down in front of the Hawks bench. As the teams headed to a timeout, Antic and Lowry picked up Ts.