A few possibilities, Mercedes:
A healthy citrus sets about 10 times as many fruit as it can bring to maturity. The extras drop off at intervals through the summer, until the right number is left, usually around August. If just a few are falling at a time, that's probably what's going on.
Extreme variation in watering or feeding will harm the foliage relatively little, but often kills the young fruit. Make sure you are watering before they wilt, and follow package directions exactly when feeding. If this has been a problem, 90%-100% of the young fruit will fall off at once, usually a few days after a drought or feeding.
If it was in an unheated greenhouse or conservatory, the recent cold weather may have damaged the young fruit. Fruit dropping on the top and outsides of the tree, while inner fruit remain, is a good indicator of cold damage. One reason why our old citrus groves here spent millions of dollars on devices to prevent frost damage!