Your confusion is showing. Internal concepts of gender are not misogynistic. They simply are. You keep on getting back to the cultural expectation argument. You need to get beyond that. It ignores real biology.

When very young children insist they are a gender that does not conform to their sex, they are not expressing a cultural expectation. They are firmly proclaiming who they are in spite of social standards, not because of them.

Sometimes feelings are just feelings and not based upon social conditioning. When a woman feels uncomfortable with a male appearing person in a dressing room, that does not mean that society has created that attitude.