Can an employeer refuse to hire me because their policy is no goatee's?

I had an interview cut short because I was informed that the company does not allow goatee's at this time. They do allow mustaches though. I was confused by this because the store manager had a mustache that was bigger than my goatee. When I went in for the interview my goatee was well trimmed. I feel as if I was discriminated. Do I have any legal recourse with this? I understand if a company has a policy like "We do not allow you to drink on the job" that makes sense. But because I have hair on my chin I'm not qualified to assist customers?