4 marble factories: 12,124 k material x 40 squares
40 squares for the factories, but each factory needs 1048 population.
Your level 21 residences give 901 population in 12 squares (75 pop/square), so each factory needs about 14 squares of population.
Also, each level 19 factory needs 276 culture, and each level 21 residence needs 382 culture (31.8/square).
In the Orcs chapter, you can probably build culture buildings at around 120 culture/square (event buildings are generally better, of course).
So adding it all up, your marble production for 4 factories uses :
40 squares (factories) + 9 squares (culture for factories) + 56 squares (residences) + 15 squares (culture for residences) = 120 squares total.
Obviously, your factories can be upgraded, but each upgrade costs more and more in population and culture = taking even more space.
It's true that your score would go down if you sold four T1 factories and replaced them with the set buildings - but you'd have 76 more squares to build other things (which could be higher-scoring per square than T1), so in total you'd come out ahead.
Edit: However, you make good points in 2 and 4 - spells change the calculation a bit, and the devs could change the production numbers around in the interests of "balance" at any time. By then, there will probably be a new event and a new set of buildings to collect anyway ...