there are lots of lovely plants but beauty is in the eye of the beholder. I like Pyracanthra, evergreen, small creamy white flowers in late spring into gorgeous berries through the autumn.

then there is Camellia, but that is more of an upright shrub. you will get lots of suggestions I am sure. others holly/berberris/osmanthus/ [depends on the type of flowers you want]

It also depends on your soil (you need acid for Camellias and Rhodos) and the way the fence faces (N/S/E/W).
Ceanothus are a lovely blue, or try Crinodendron hookerianum - the lantern tree (stunning).

Some are pretty when they flower but dull when they dont, not sure if you mean pretty all the time or just when they flower. I have Viburnum tinus, and Escallonia 'Apple Blossom' both very pretty when they flower but somewhat dull most of the year. Do look at Cotoneasters and Ceanothus but these may be too tall for you?
I find my Aralia fatsia japonica, and Euonymus fortunei Emerald 'n' Gold ' have an all year round interest?