Okay, ignoring the fact that they have the sex chromosomes backwards, biosex genetics are not binary either. You can have trisomies or more, like XXX, XXY, XYY, or even XXYY. Or you can have only one X chromosome with no pairing chromosome.
Or, you know how in high school you learned about meiosis and crossing over? How it's why most children of the same parents are genetically different from each other? Well, you can get accidental crossover between the X and Y chromosomes, namely the sex determining regions. That's how you can get people with XX but develop biological male characteristics, and vice versa. It's also why there are biologically intersex people and "hermaphrodites" (not sure of that word is PC, please let me know if it isn't).
None of this have anything to do with gender identity though, just biological sex. But surprise surprise, science has also found real neurological, congenital, and yes, genetic factors, that contribute to a person's likelihood of identifying as transgender, nonbinary, etc.
More info here, with sources: https://yewtu.be/watch?v=MitqjSYtwrQ