Arsenal required a last-minute penalty from Santi Cazorla to secure a 2-1 win against Southampton at the Emirates Stadium. The Saints took the lead in the first half through Dusan Tadic, whose 20-yard free-kick crept past Petr Cech and into the net.
The Gunners - who handed debuts to Shkodran Mustafi and Lucas Perez - responded shortly afterwards through a spectacular overhead kick from Laurent Koscielny and immediately pushed for a match-winning goal. However, they were forced to wait until the 90th minute, when they were awarded a penalty after Jose Fonte fouled substitute Olivier Giroud in the box. Cazorla took the subsequent spot-kick and secured all three points for the hosts.
At the Bet365 Stadium, meanwhile, striker Harry Kane broke his goal drought in Tottenham Hotspur's 4-0 win against Stoke City. The England international added the fourth goal in what was a thoroughly one-sided match, which also featured a brace from Son Heung-min and another from Dele Alli.
Stoke's misery was compounded by manager Mark Hughes being sent to the stands in the first half after losing his cool on the touchline.
At the London Stadium, Watford came from two goals down to seal an unlikely 4-2 win against Slaven Bilic's side. The hosts took the lead through a Michail Antonio brace, before goals from Odion Ighalo, Troy Deeney, Etienne Capoue and Jose Holebas completed a remarkable turnaround.
At the Riverside, Crystal Palace's Wilfried Zaha ended Middlesbrough's unbeaten start to the season and, in turn, gave the Eagles their first win of the campaign. Christian Benteke scored his first goal for his new club in the first half before Daniel Ayala levelled the score with a header. But Zaha - who was recently linked with a move to Tottenham Hotspur - put Palace back in front after finishing James McArthur's through ball.
At Turf Moor, Robert Snodgrass scored a 95th-minute free-kick to rescue a point for Hull City against Burnley. The midfielder found the net from 20 yards following Ben Mee's foul on Tom Huddlestone, after Steven Defour had earlier put the hosts in front.
Meanwhile, Bournemouth's Callum Wilson scored a brilliant late goal to earn the Cherries victory over West Brom, their first Premier League win of the campaign. Wilson met Adam Smith's cross with a deft flick to give the home team the victory with just over ten minutes remaining.
In the early kick off, Manchester City won the Manchester derby courtesy of a 2-1 win against United at Old Trafford. Pep Guardiola's men, who produced a stunning opening 40 minutes against their eternal rivals, took the lead through man-of-the-match Kevin De Bruyne, who calmly slid the ball past David de Gea in a one-on-one situation.
City doubled their lead through 19-year-old forward Kelechi Iheanacho, who poked the ball home from close range after a De Bruyne effort struck the post. However, United were handed a lifeline in the game shortly before the break when City debutant Claudio Bravo dropped a cross in his area and Zlatan Ibrahimovic volleyed the ball into the net.
Thereafter, United boss Jose Mourinho introduced teenage striker Marcus Rashford and midfielder Ander Herrera, which served to change the complexion of the game. But neither was able to find an equalising goal and City - who always posed a threat on the break - were able to hang on for a win.