Narrowly avoided harassment yesterday despite being closeted

Hi. I'm a closeted 17 year old non-binary person, and yesterday I was alone walking through the quad at school when a group of guys maybe twenty feet away yelled “Hey, ‘They/Them’!” I didn’t think they were talking to me because I’m not out and I have terrible space awareness so I barely heard them. But when I didn’t respond they just said to each other “Nah, I don’t think he’s a [slur].” 

So like… on the one hand my ignorance saved me, but on the other hand WHAT THE ACTUAL F*** AND HOW THE F*** DID THEY KNOW ABOUT MY IDENTITY?!

The only way I can imagine is that either a) someone saw something in my notebook (which I try to keep clear of anything trans-related) or b) they just made a lucky guess. To be honest the second choice wouldn't be that crazy considering I wear a bright purple jacket all the time and act generally strange, but I'm still extremely nervous about how they would've known. I don't even know these people, so were they told by someone else? Is this a rumor that's being spread throughout the school??? Normally I wouldn't give a f*** if someone spread some rumor about me, but this time around it could legitimately put me in a dangerous situation (especially considering the US' current political situation), and I'm really nervous. 

Also, this is the first time I've ever heard someone actually use a trans slur (I have always stayed away from people like that, even before figuring out my identity), and it stung way more than I would've expected it to. But mostly, I'm just really nervous. Even though California is a very liberal state, the current federal government is devolving into something really dangerous and it seems these people might be emboldened by this.

Anyway, I just needed to vent this somewhere, and I just wanted to see if anyone had any advice to stay safe and avoid accidentally outing myself, assuming every transphobe on campus doesn't all know already...