I’m not too sure what you plan on doing with the roof area (well I think it’s the roof… anyway… that blue sextion, bottom right). You could possibly overlap some of the UVs there and save some space and scale some of the other smaller detailed areas up. From what I can tell there seems to be some texture space being wasted.
With regards to the white wall texture that you want to scale down: are the areas that you want to use it on going to be unique (ie. have dirt on, or any other attribute that will set them apart from the other pieces of white wall)? If not, I can suggest that you possibly devote a large area of that map for those UVs that will share that wall texture. Then you should be able to use a larger, less scaled piece of wall on you unwrap.
Also I might help if you can post a render or screenshot of the building.
About scaling down photos, you really need to be careful there. From a large image down to something quite a bit smaller can give bad results. What you will probably have to do is repaint and touch up some areas.
Anyway hope what i mentioned makes some sense, and helps you out.