So it does bear fruit, but the olives are tiny? In that case, if you increase watering once the plant flowers, and keep it up once small fruits have formed, you may actually manage to get proper olives IF the weather remains suitable. They need to be kept well supplied with water during this time so that the olives can swell, and particularly if growing in a pot. They taste quite different from shop bought olives though...