The Cityzens leapfrogged Liverpool into top spot after beating the Saints thanks to Erling Haaland's first half strike.
The Norwegian's goal ensured Southampton's winless start continued, with Russell Martin's side rooted to the bottom of the table.
However, Guardiola was complimentary of the South Coast club, claiming they are 'one of the best teams in the Premier League' in one area.
The Spaindard told BBC Match of the Day: 'The way they played, they are so demanding. They defend with the ball.
Pep Guardiola praised Southampton following Manchester City's 1-0 win on Saturday
'You can defend with the ball and they are really good [at it], one of the best in the Premier League.
'They didn't create many chances, but we created a lot. Unfortunately we could not score the second. Big credit to them - it wasn't because we were sloppy or flat, I don't have that feeling.
'I think we were there and sometimes they were good. We have to accept that.'
Southampton sit bottom with just one point after losing eight of their opening nine matches on their return to the top flight.
However, Martin insists he's 'proud' of his players and that they will learn from the latest defeat.
'I'm proud of them. The courage they had and the build-up was incredible. We had some big moments, so I'm proud of that as well', the Southampton boss said.
'I think we had more [chances] than most coming here. The level of courage and intensity to run and play football in our way was incredible.
'We will take a lot of belief from today. Not just the defending, the character after going 1-0 down early and the character to play in the way we want to.
'If we are going to lose, at least we will do it in a way that we can be proud of.'