First, I'm not sure you can call it a Mosque, or say that it is located "at" ground zero. But lets lay that aside for a second. Here are some things that conservatives traditionally value, that would seem to say, "let them do it."

1. Religious freedom.
2. The rights of property owners.
3. Decision making on the local level.

Many of the people who are fighting against this also fight for religious displays on public property, no matter how "insensitive" their opponents tell them it is.

I can imagine a US church attempting to buy and build a place for worship in Baghdad, and local lraquis protesting that they should not. What do you think the pundits would say about that? Likely, we would assert that anybody can build any church, anywhere, if they pay for it.

If the anti-mosque folks win this debate, they will likely come to regret it when Christians are in the minority (likely, soon) and the same arguments they've made are used against them.

You treat people the way you want to be treated, regardless of how they treat you.