Klopp saw his side give away a two-goal lead against Arsenal to settle for a point as the match ended in a 3-3 draw at the Emirates. In the Champions League, Liverpool surrendered a 3-0 lead against Sevilla and came home from their trip to Spain with a point after the final scoreline was 3-3.
Liverpool are fourth in the table with 35 points, one point ahead of fifth-place Tottenham. They are already out of the Carabao Cup and their chances of winning the title look impossible at this stage of the season.
Manchester City are flying under Guardiola and have lost only one competitive fixture in all competitions so far this season. They are on top of the table with 55 points after 19 games, leading second-place Manchester United by 13 points.
Neville believes Liverpool do not have the defence to win the Premier League of the Champions League. The former United defender believes a clearout that allows weak links leave is the only way going forward if the Anfield club aims to be successful.
"Get rid of the players. You look at what Pep Guardiola did last season. He brought Claudio Bravo in for big money. He wasn't good enough and couldn't adapt to the Premier League. He got rid of him," Neville told Sky Sports.
"Ederson comes in and looks fantastic. The full-backs weren't good enough at City, he got rid of them, spent £130m on three new ones and just dealt with the issue.
"He has got to deal with the issue and Jurgen Klopp is the only one who can do that. It is staring everyone in the face. What I would say is that the toughest part of coaching a football team is making them inventive going forward. Liverpool are fantastic going forward. If he fixes that defence he has got a real chance.
"At Borussia Dortmund, he had Neven Subotic and Mats Hummels with Lukasz Piszczek at right-back. He had a good back-four, a good base to build from. You look here: Ragnar Klavan? Really? DejanLovren has disappointed me even though he was fantastic at Southampton. Joe Gomez is young so I think he's got some development to do.
"Simon Mignolet we have talked about, Loris Karius we have talked about. And then you talk about the left-backs, you play James Milner there, you play Andrew Robertson there, you play Alberto Moreno there. Can you trust them in the biggest games?
"The reason why we are talking like this is because we are talking about Liverpool winning the Premier League and the Champions League. That's the measure of the club historically.
"So if you look at that and you say, 'Do I want Moreno at left-back? Do I want Robertson at left-back? Do I want Klavan as my centre-back playing at Real Madrid away or Barcelona away?' You can't. You can't have those players in those games. It's impossible. So they've just got to go.
"It's sad and you don't like saying that because they are probably good pros and they'll have great careers. They are good lads but they shouldn't be playing here at this level at a club that is trying to win the league.
"Jurgen Klopp did an interview beforehand where he told us that his defence was getting better. I think they will always make you feel like that when they are playing, with respect, Bournemouth or Newcastle or Burnley. They will always make you feel like they can keep a clean sheet in those games.
"But in the really big moments when a goal goes in against a good team like Sevilla or Arsenal away, they will let you down."