The Blues were reduced to nine men and required a penalty shootout to win their third round FA Cup tie against Norwich City on Wednesday, 17 January, following a 1-1 draw after extra-time.
It means Antonio Conte's men have now played out five successive draws, with just two wins in their last eight games in all competitions. Summer-signing Alvaro Morata has notably been a cause for concern for Chelsea as the Spaniard is currently goalless in his last six games for the club.
With a need for attacking reinforcements, Carroll has emerged as a shock target for the reigning Premier League champions. Chelsea have already opened talks with the Hammers over a six-month loan deal but have been told only a permanent transfer will be considered this month, with West Ham demanding a fee of £20m ($27.5m).
Wise believes the 29-year-old is not the answer as he would come to Stamford Bridge with a lot of question marks hanging over his head.
"He's a handful on his day, he really is. [But] no, I don't think he is [the answer]," Wise told Sky Sports. "The reason being he doesn't play enough games, has a lot of injuries, is 29 years old, doesn't score enough goals and therefore, looking at that point of view, no.
"Chelsea need a striker that likes to go in behind to stretch teams. He won't be that person if they play with one up front. He will be more of a hold-up and with the players that they've got, they come off more. They always need that one person who is really going to stretch to give them the space."
The former Chelsea captain also stated that Carroll would only be a bit-part player at the club and would not challenge Morata enough for his place in the starting eleven.
"I think there's more out there that score more goals because they'll get a lot of chances and play, and really push Morata," Wise added.
"That's what he needs and I think Andy won't push him as far because he'll only play a certain amount of games, and I think you want a player that's really going to push Morata that's going to score goals and put him under pressure."