Cauliflowers are particularly sensitive to any check to growth, caused by dryness, for example, or a cold snap followed by warm weather.
The only way to be sure of getting large well hearted caulis during the summer months is to have a regular, frequent watering regime which you follow religiously, with plenty of nitrogen or organic fertiliser.
They are far easier to grow for a spring/early summer crop or a late autumn one, when the weather is cooler.
Another problem can be cabbage root fly which damages the roots and causes the plant to abort in the same way.