Rangers were sitting eighth in the Scottish Premiership, 11 points behind leaders Heart of Midlothian, and had failed to qualify for the Champions League when former Southampton boss Martin was axed.
They have narrowed the gap to nine domestically under new team boss Danny Rohl and are up to fourth but have now lost all four Europa League games, the latest two under the German, amid criticism of their summer transfer business.
“Responsible change is measured and steady,” Cavanagh said. “Our first priority has been to improve the sporting performance and that led to our decision to part ways with Russell and bring in Danny.
“Danny has now started the process of improving the on-pitch performance and that has given us the chance to focus on other parts of the club.”
Stewart took over as chairman in December after 18 years at Old Trafford, rising to become United’s chief executive, having spent five years previously with Uefa’s Champions League commercial agency.
He was also appointed to the Scottish Professional Football League board five months ago.
Thelwell joined Rangers after three years as Everton’s director of football, having previously been head of sport for Major League Soccer club New York Red Bulls.
“Patrick and Kev are both skilled executives and have delivered a lot for the club in their time with it,” Cavenagh said. “I am not here and will not throw dirt on either of them.
“I think very highly of them both on personal and professional levels. It’s just the reality that the club needs different things today than it did six months ago.
“We have already started the hiring process and hope to have appointments soon, but we will prioritise quality and fit over speed.
“Change like this is always painful, especially when it involves people we care about.
“But, at the same time, we’re very excited about this opportunity to bring in fresh leadership to this great club.”
Cavanagh added that Rohl “will continue to have all the support that he needs”.