there will be many plants that are not food sources for catterpillars. avoid nasturtiums as they are adored by cabbage whites and their near cousins. decide on the plants you like then google to see if there are any butterflies that use them.

plastic flowers will do the trick

Regular spraying with Bacillus thuringiensis v. kurstaki will kill the caterpillars, while harming little else. I can't recommend it for everyone, since the moths and butterflies that they become are such friends of the gardeners.