Should just call them XX or XY because no matter what surgery or identity they think they have you can't change biology.
It's pedantic but not everyone is XX or XY. Women can be XY, and it's not all that rare.
But for general usage, look to the chromosomes. I believe the same as you but people will bring up people with genetic abnormalities. Women can also have just an X and no second chromosome.
It's a complex thing, but letting a man in a dress or having women "bind" their chests to pass as male is not the best solution, I feel.