The Labour party is at its worst position in nearly 50 years, David Miliband had said.
The former leadership candidate for Labour said the party is now weaker than it was in the 1980s and must face up to the "historic" challenge ahead.
Miliband's comments come after Labour's defeat in the Copeland byelection to the Conservative Party, which marked the first time a governing party had taken a seat from an opposition party in a byelection since 1982.
"I'm obviously deeply concerned that Labour is further from power than at any stage in my lifetime," Miliband said in an interview with the Times on Saturday (25 February).
Trudy Harrison of the Conservative Party beat her Labour counterpart Gillian Troughton by more than 2,100 votes to take the seat in Cumbria.
Jeremy Corbyn has come under intense pressure after the defeat and was urged not to "pass the buck".
The Unison General Secretary Dave Prentis told the Guardian: "The blame for these results does not lie solely with Jeremy Corbyn, but he must take responsibility for what happens next.
"Nurses, teaching assistants, care workers and ordinary people everywhere need a Labour government. Jeremy has to show he understands how to turn things around and deliver just that."
Despite being pressured, Corbyn remains defiant and said he would not be standing down.
"I was elected to lead this party," he said. "We will continue our campaigning work on the NHS, on social care, on housing."
The loss in Copeland overshadowed Labour's victory in the Stoke-on-Trent byelection, where Gareth Snell stubbed out Ukip leader Paul Nuttall's hopes of becoming an MP.
But despite calls for Corbyn to step down, both Labour's supporters and political analysts cannot see any viable replacement for him. Speaking to BBC News, Professor John Curtice of Strathclyde University said that "it's not entirely clear Labour's problem would be solved", unless a new potential leader is found from within Labour.
When asked by the Times, Miliband ruled out a return to Westminster, but kept his options open.
"I honestly don't know what I'm going to do. It's hard to see, but what's the point of saying never?"