Fears for building societies as private lenders take up market space