You'll have to harden them off first then, if they've been inside all this time - do that by placing outside during the day for increasing periods of time over 2 weeks, then pick a milder night to leave them out all night, then move them further away from the house or wall till they're out in the open and fully hardened, then plant - if they were hardened off you could plant them now, but obviously it'll be mid to last week in March, I'd imagine, before you do, though if your conservatory is unheated at night, it won't take so long to harden them.

Just to add to Bamboo's advice, I'd check out your soil type and treat the planting site accordingly. We are on very heavy alkaline clay and found the bought quince specimen we got died in just over a year, while the ones we grew from seed are all struggling and look very unhealthy in the summer with blighted leaves.
So should you be on heavy clay like us, you need to do something to improve the soil.

Is that cos you're in France, though, Bertie? It grows brilliantly here in West London on clay so solid you could make pots with it.