Third-tier Ipswich were beaten 10-0 on their previous visit to the Blues in 2020 so this marked a much-improved display.
England forward Lauren James also returned for Chelsea as the five-time winners beat Charlton 4-0.
James, who has not played since pulling out of a Champions League game against Twente in the warm-up on 17 October, played 45 minutes, before being taken off at half-time.
A header from Mayra Ramirez, a Guro Reiten penalty, a clever Aggie Beever-Jones finish and a Niamh Charles strike meant Chelsea extended their unbeaten start to 17 games under Sonia Bompastor.
There was concern for the French boss in stoppage time, though, when Dutch midfielder Wieke Kaptein was carried off on a stretcher after landing awkwardly.
Fulham, who play in the fifth tier and were the lowest-ranked side in the fourth round, were comfortably beaten 5-0 by Championship side London City Lionesses.
Chantelle Boye-Hlorkah netted a first-half hat-trick, before Kosovare Asllani scored a penalty and Issy Goodwin stroked home the fifth goal.
Two third-tier sides – Wolves and Rugby Borough – are also through and will be the lowest-ranked sides in the next stage.
Seven ties – including Arsenal v Bristol City, Everton v Tottenham and West Ham v Liverpool – were postponed across the weekend due to the inclement weather across the country.
