There are a lot of allergic people who keep cats and survive. My team captain's boyfriend is severely asthmatic and allergic, and she has come up with all kind of coping mechanisms. Her cats are both long-haired cats, and one of them, she just shaves every few months, the poor thing. She also vacuums religiously and they got rid of all their carpeting and have leather couches because cat hair doesn't stick to leather. But they have a much smaller house so there are fewer places for poor allergic Walt to retreat.
Most important is KEEP THE THING OUT OF YOUR BEDROOM. You need a cat-free space to retreat to. Be really strict about this, even to the point of not keeping dirty laundry in there that might have cat hair on it.
I mean, that's only if he's coming inside at all-- but really, restricting what rooms of the house he can go into is a really good idea if he ever comes inside. Keep doors shut and so on.
A good-quality HEPA air filter in the cat-free space will also help. I know these assholes fired me, but their HEPA filters are for serious-- www.austinair.com -- and they're good for chemical sensitivities too. If you're interested in one I'll give you my old sales manager's contact info-- he's still a nice dude and could cut you a deal.
www.allergybuyersclub.com is a good website with various allergy coping mechanisms-- faithfully brushing the cat (preferably outdoors) to keep shedding and dander to a minimum, but they also sell like these baby-wipe-like things that you can wipe the cat down with now and then to get dander off it so it doesn't become airborne.
Does your kitty have a name? He looks a lot like Chita so I am automatically a sucker for him and want you guys to totally keep him. The more proactive you are about keeping him from annoying you, the longer you can keep him without, you know, dying. So really I'm being selfish in telling you all this, but also I don't want you to die.