Hard to say - when you say 'shaley rock', that implies a fine grained sedimentary rock, which is usually mineral rich - what matters is whether there is soil beneath this material, and how far down any subsoil actually is, i.e., is there soil beneath a few inches of the shale sufficient for large roots to grow? Your builders may know the answer, assuming they've done footings, laid drains and services, or done any piling. If the shale extends downwards for more than 2 or 3 feet, I wouldn't be expecting to grow any large plants, rather, it'd be better to add soil over the top and grow much smaller plants.