The former England captain swapped Old Trafford for the Santiago Bernabeu after his relationship with Sir Alex Ferguson became damaged beyond repair. Having suffered an injury in the early stages of the 2002-03 season, Beckham struggled to regain a place in the starting XI, with Ferguson preferring to deploy Ole Gunnar Solskjaer as right winger instead.
The simmering tension between Ferguson and Beckham threatened to boil over in February 2003 when, following a defeat to Arsenal at Old Trafford in the FA Cup, the former struck a boot in the dressing room that flew across the dressing room and ended up hitting the latter above the eye.
Beckham required stitches and the publicity gathered by the incident only sped up the divorce, as he brought an 11-year spell with the Red Devils to an end.
Speaking to exclusively to United fan channel Stretty News TV, Neville said Beckham's departure did not come as a surprise.
"I knew it was coming to an end a little bit from the previous 12 months," Neville said. "I was aware of the things that were going on behind the scenes.
"I was disappointed when he left but I was also disappointed when Nicky Butt left.
"I was disappointed when my brother left, I was disappointed when Roy Keane left and I was disappointed when Denis Irwin left. They were a massive part of our lives and our careers at United."
While Beckham's departure was not surprising, Neville admitted it left a gaping hole in a United side that had just clinched a fourth Premier League title in five seasons.
"It wasn't just the lads around me - David, Phil and Nicky and such like," he said.
"When a player of that magnitude left there was a void. They're a big miss.
"To be honest with you, at the time 'sad' probably wasn't the right word.
"I was probably relieved a little bit. For him, for the club - it could never end badly, that relationship. David had done so much for the club and the club had done so much for David."
After his stint in Madrid, where he won the league title in the 2006-07 season, Beckham went on to play for Los Angeles Galaxy for five years, with two loan spells at AC Milan sandwiched in between, before ending his career at PSG.
Neville, who retired from football halfway through the 2010-11 campaign, revealed Beckham had always harboured this hope of playing abroad and did not begrudge his former teammate's decision to leave Manchester.
"You can't look at it and think that he was wrong," he added.
"He's richer for those experiences.
"His time had probably come at United where he was right to leave. There was a bit of me that was relieved that that tension, that little bit of an underlying thing that was there had gone.
"He was a brilliant player and I loved playing with him."