Is there a problem with taking it in over the winter?
If I had an established citrus like you have (well I have, I just keep it permanently in the conservatory but I'm in NE Scotland!), I'd hate for it to catch a bad winter and die.

Maybe with global warming you will be able to leave it out after 2050 or so! In the meantime I'm afraid you have to take it in.
If you had a sheltered urban courtyard facing south, you might get away it with it once or twice, but then along would come a cold winter like this year's and it would be goodbye lemon.
If you like 'tropical' and unusual fruit, why not try a feijoa or Acca, as these can survive our winters outside.

There are some pretty hardy Banana plants around as well. Look for hardy Musa species. You'd probably still have to wrap them up though.
http://www.cooltropicalplants.com/Cold-hardy-bananas.html