An attack of the dreaded Plum moth. All you can do is destroy the affected fruits. Then next year get a Plum moth pheromone trap and spray against them when you see the adults getting trapped. Dratted annoying though. Sympathy extended.

They could also be sawfly. The easiest way to distinguish between the two is, plum moth caterpillars are pink with a brown head, sawflies are green with a black head. If it's sawfly then you need to treat the tree next season with an insecticide about a week or so after the petals have fallen prior to the fruit forming.

Our neighbours have about 5 plum trees, and every single fruit is similarly affected - not one edible one. I think it has been a particularly bad year. We used pheromone traps for codling moths in our apple trees, and the spraying regime for this and plum moth seems to be very similar.