I had a really hard time finding the kind of info to satisfy this curiosity. Most architecture histories focus on notable buildings and stylistic trends, not neighborhoods or development. I imagine the perfect representation of what I'm looking for is a real time, 3d map that you could play through history from the founding of philadelphia to the current day. But this doesn't really exist (although philageohistory.org is getting close).
Anyway, I've found a really good book on housing development in Philadelphia, focusing on the early 19th century. It's called Making Houses, Crafting Capitalism: Builders in Philadelphia 1790-1850 by Donna J. Rilling. I'm only two chapters in, but I'm surprised how interesting it is. I'm really enjoying the specific stories, the who and why of the men who built metropolitan Philly. One bit I'm particularly interested in: Philadelphia's system of 'ground rent' made it easier for small entrepreneurs to build, vs. boston and NYC, where the established merchant capitalists conducted most of the residential development. It's a nice little contrast of economic incentive that provides a nice understanding of why this city is how it is.