The Bucs opened the scoring with a Sean Tucker touchdown before the Falcons led 14-13 at half-time after Pitts, who came into Thursday’s game with just one touchdown all season, claimed his first two scores.
The home team went into a 28-14 lead early in the fourth quarter after Baker Mayfield made touchdown passes to Devin Culp and Chris Godwin Jr.
Veteran Kirk Cousins was making his fourth straight start after a season-ending injury to Atlanta’s starting quarterback Michael Penix Jr and the 37-year-old led the Falcons to the 32nd fourth-quarter comeback of his career.
Running back Bijan Robinson got into the end zone before Pitts claimed his hat-trick and Atlanta regained possession with less than two minutes left.
And despite being backed up to third and 28 after recovering his own fumble and a 10-yard penalty, Cousins made key passes to Pitts and receiver David Sills V to get the Falcons in range for Gonzalez to kick a 43-yard game winner.
Cousins finished with 373 passing yards and three touchdown passes to Pitts, who claimed 166 yards from 11 receptions.
Atlanta are now 5-9 and their upset of Tampa Bay did Carolina a favour as the Panthers (7-6) now top the NFC South and face the Bucs in two of their last four games, with only the division winner guaranteed a play-off spot.