I'm just wondering what you mean by a 'cornwall' hardy olive tree. Olea europea is olive, and there isn't one that's native to Cornwall. They are not fully hardy, and need a sheltered, warm and sunny spot to thrive. They may look a bit sickly after winter, but usually recover as the summer goes on, although this year, with the weather being as it is, recovery may take a bit longer.
How long have you had your tree and is it in a pot or the ground?