Unfortunately, Victoria is a variety that is especially prone to Plum Rust - rake up affected leaves and burn or dispose of in the rubbish bin, leave none laying around. If you have anemones growing in your garden, inspect those for rust - often the source is rust ridden anemones, the two are linked. Keep the tree well fed and watered next year to maintain vigour. No treatment available otherwise. Not sure about the fruits being sugary and shrivelled - drought?