I'm not against a modular approach at all, in fact if it's done right it could work out great for guys like me trying to make PCB's available. It worked wonders for the 3D printer market, where essentially all the 3D printers at the hobby/home level are based around a couple basic standards of board layouts, physical layouts, and software sets.
Generally form follows function. A great starting point would be to define which features would go on which module, and once that's defined we should standardize connector and signal types between the modules. This way we're not all re-inventing the connectivity standards from iteration to iteration, and modules from different sources would be easily interchangeable. I'd put a high preference for connectors that have waterproof styles available, with non-corroding housings.
But the first thing I think we should agree on is which side of the Pi board the header should go
"MAYBE" @Michael Lane actually has the right idea. Put them all on top on a single board. Then you could have a series of pigtails to whatever type of connector arrangement you could want. Possibilities for connector positions are endless then.