Go out after dark with a torch and pick them off and stamp on them.

Or if they are very invasive try Provado

If in mangeable pots, try a soil change and bump off the magotts if you can find them in the old soil.....robins love em. If pot is bigger......difficult. There is quite a history on GoY answers to questions saying Provado affecting the honey bee population. So far I have given 1 drench early in season before they get too active, and just as the plant is stirring. I am choosing to do no more than that because I see no honey bees so far this year. I might have seen one today, and it wasn't very well. Tenderly placed in the garden. By the way welcome to GoY.