Currently the building symbolizer uses a darkened version of "building-fill" in order to color the walls of the extruded form. It just multiplies the building fill color by 0.8.
See this line and this line.
It would be great if there were a way to designate the fill color of the extrusion walls, perhaps with something like "building-wall-fill".
/cc @ajashton - do you think this would be useful?
Yes, building-wall-fill sounds good.
Related, it would also be nice to specify the width of the line that this symbolizer draws around the extruded polygon.