In some ways I can relate with you on this one, @skathie, as I am a social anxiety sufferer.
You're right, it is just a feeling and it's very good that you have challenged it on a rational basis. I found that was an important step for me and whenever I slip, it's often the first thing that I've forgotten. With regards to dealing with it, I think it's important to try and put things into a context.
When you see that person coming the other way in the corridor, what is the worst that could happen? When you really break it down, the worst thing that could happen really isn't that bad and you'll probably find that you can cope with it. But, equally, when you confront it and continue to walk bravely down the corridor, you'll find that things won't turn out the way that you dreaded in the first place.
Ultimately however, it's up to you what approach that you take to your anxiety but I find the following is true. Some people like to talk about having "outs" if a conversation does strike up. I'm not a big fan of that for my own social anxiety because in a way, it's a get-out-of-jail-free card.. or in other words.. it's an unconscious admittance that "I can't cope with the situation so this is how I'll get out of it". For all of the years that I tried that approach, my anxiety only got worse. For the year that I took the opposite approach, I found my anxiety became more manageable. Note the wording there.. the feeling still exists but it's more manageable than previously.
Ultimately, however, our anxiety trigger.. our anxiety symptoms.. and therefore the ways in which we can successfully manage anxiety differ among all of us. Have a look around the anxiety forum, especially the Anxiety Support Resources forum sub-category. You may just find something that makes sense for you there :)