How can I deter worms from under block paving

I take it you layed on soil not sand?

You can't stop worm activity, I'm afraid, there is no way you can achieve that. I do wonder, though, what you put down to retain the edges of your block paving - there should have been a concrete/shingle/sand mix laid all along the edges of the paving, a good 4/6 inches deep and about 2 inches wide minimum, which should retain not only the paving, but the sand you have presumably laid it on. Worm activity would then not be able to affect the level of your paving, and actually, I doubt the worms are affecting it anyway - its because there's no functioning retaining material, by the sound of it.