Depends how many leaves are dropping - they do lose some leaves, so as long as you've got half the leaves still on the tree, and new ones coming, it's fine. It probably prefers being outside, but it can't be, obviously, during the winter. I have a client who has a lemon tree and keeps it in the shed all winter - by the time spring arrives, it's often got no leaves at all, but it revives once repotted and put outside and fed, perhaps pruned a bit if necessary.