My experience with oleander is that it doesn't do well the further north you go.

Given that we've had a mostly cool summer, I wonder if it just hasn't been warm enough for it where you are.

It needs plenty of water at the time of the buds opening, i've noticed that mine does the bud dropping thing if it's too dry.
Mine's in a sheltered south facing border that's bone dry, great for overwintering but not so good for flowers, it's starting to open the buds a wee bit now so i'm keeping my fingers crossed !

Lack of warmth and/or sun would prevent buds from forming at all, though prolonged cold, gray weather can deep six buds that formed in friendlier weather. I'm leaning in Louise's direction. Although Oleanders are drought tolerant in the ground in mild climates, in pots they are real water suckers.