I think its an Arum Italicum or lords and ladies.

Or Arum maculatum. By the sound of your question you didn't see the leaves. Maculatum has black spots on the leaves whereas Italicum looks like Italian marble.

It's known by many common names including Jack in the pulpit, wake robin, cuckoopint (rhymes with 'mint') and up to 100 others.

Those berries are toxic, by the way.

Either let them be and allow them to self seed or pick the berries remove the seeds and plant up to grow more...

That's if you want more of them.... I'd pull it up and get rid of it, but it's a matter of opinion. I had written "taste" but as they're poisonous, changed it!
I know it as Wild Arum and regard it as a weed.

I do not view this as a weed - sorry Beattie!

I love it, especialyl Italicum marmoratum. It is certainly cheering up your ivy.

I'm sure "weed or wanted" depends on how much you have of whatever it is, and whether it spreads more than you want it to.

My garden would be over run with them, so I do remove each and everyone yet still they come.

Same here Drc. I dig out the tubers too, if I find them.

:))