What do you mean “skill against skill”?

Like your boxing example.
On lichess.org new accounts gets the average Elo rating from where they can work their way up or down. In sports one can simply go to a club and start off with the beginners or where they or a coach think they could manage and work their way up.

Another way of interpreting your comment is dismissing differences and simply making people all compete against each other. This would obviously exclude all women from most sports so you’d have to be a pretty committed misogynist to think that was preferable to gendered sports.

This wouldn’t be misogynist but bullying because heavyweight men/women would beat up featherweight men/women too.