The 46-year-old was sacked six months into his 18-month deal in North London after failing to guide the club into the Champions League, a year on from the sale of Gareth Bale to Real Madrid.
Having returned to management with relegation threatened Aston Villa, Sherwood has won just two of his seven league games in charge but has taken the club into the semi-finals of the FA Cup.
Sherwood returns to White Hart Lane this weekend with his Villa side three points above the relegation zone after the 3-3 draw with Queens Park Rangers and admits he career could have taken a different path had he taken up Levy's offer to remain at Spurs.
"Daniel Levy has known me for a lot of years," said Sherwood. "It wasn't as if I fell off my chair and said, 'you're joking'. I knew [the sacking] was coming.
"I was disappointed I was leaving, but Daniel wanted me to stay in some kind of capacity. But I'm a manager now. He didn't say what job. I was looking around eyeing up the broom cupboards and all that!
"I respect the decision made by the board. I thank them because not being there now is the reason I am at this fantastic football club.
"I've been in communication with Daniel since I've left, I haven't got a problem. He's the chairman and he makes those decisions based on what he thinks is in their best interest."
Levy replaced Sherwood with Mauricio Pochettino from Southampton and despite the club reaching the League Cup final where they were beaten by Chelsea, Tottenham have endured an uncertain season under the Argentine's guidance.
The club trail fourth place Manchester City by seven points with just seven games to go and though Levy is unlikely to make another managerial alteration he will not be content with another season without qualification for Europe's premier club competition.
Having been at Tottenham on and off as a player, assistant coach, technical director and manager since 1999, Sherwood has been exposed to Levy's style of leadership and has revealed an insight into his fearsome reputation.
"I was there the day he walked into the club [in 2001]," he said. "He got rid of George Graham and brought in Glenn Hoddle in the week building up to the FA Cup semi-final.
"He is not frightened of making decisions at crucial times. You have to say Tottenham is a far better club now than when he found it.
"I have a lot of good friends at the club, I will shake their hands. Once I get in that dressing room I'll be telling our boys to do whatever they can to win."