According to the Plants For a Future database it's not poisonous.
http://www.pfaf.org/user/Plant.aspx?LatinName=Leycesteria+formosa
The berries are variously described as "very sweet with a treacle-like taste" or "Bitter and not very desirable". Perhaps it depends how ripe the berries are and it may vary from plant to plant.

Lots of common garden plants are poisonous - it may be best to teach the grandchildren not to eat anything without checking.

Thankyou! for response this plant just appeared had no idea what it was but we think it is qiute unusual. Have just discovered its name from a friend, we moved it this year and it took really well.

They self seed very easily. Perhaps one of your neighbours had one - the seeds are spread by birds. Another name for the shrub is Pheasant Berry as lots of them were planted to give food and shelter for game birds. You may well find that you have seedlings coming up in strange places in future. They're easy to get out when small and if they're somewhere where you don't mind having a large bush growing you could leave them.