Yeah, but think your examples through.

Refusing to treat because of a religion is illegal and there is no way for a doctor to justify it.

But being a doctor that only treats men or only treats women is not illegal, and many exist.

My barber only services men.

If he came out and said that he doesn’t cut women because they are inferior or whatever, he would quickly be fined or jailed.

But because he says he specializes in men’s hair, it’s fine.

And maybe he secretly is a misogynist, but there is no way to know.