I'm sure an acorn is the fruit of the oak tree

Yes, they are mostly the classic egg shape to a greater or lesser degree though some are more top shaped (especially the Red Oak section of the genus).

If the fruits are diseased though they can look like a knarled craggy conker (but green)

Or are you refering to the oak gall caused by an insect?
They are spherical and about the size of your fingernail.

You are either on about the ACORN or the Oak Apple .
The Oak Apple is a golf ball with "warts" , (i call them) all over it but other wise the Acorn is not very big and is the main fruit of the Oak tree

The Acorn is the ONLY fruit of the oak, the oak apple is a gall induced by a small insect related to Ichneumon Wasps.