Physalis pruinosa drops its fruit inside the little papery seed case when the fruit is ripe and a pale yellow colour. I guess the physalis you are referring to is the Cape Gooseberry.. Physalis peruviana, where the fruit should be golden yellow. The latter tends to fruit when it's grown into a very large plant (around a metre high and wide) and I find this is often not before mid to late August in a polytunnel, unless I have overwintered the plant. So I wonder if yours is just fruiting prematurely on too small a plant?
If this isn't the case and the plant is already very large, then it sounds like it may be too dry at the roots.

Thanks.