I have been wondering whether it is possible to develop 3D from Street view. Yes, some clever people already tried it and succeeded.
1. From Street View to 3D : http://googlemapsmania.blogspot.com.au/2016/04/from-street-view-to-3d.html
2. 3d world modeling with three.js : http://peter.pisljar.si/#!/en/projects/threejs_world
3. Large-Scale Multi-View 3D Reconstruction from Google Street-View Images: https://www.youtube.com/watch?v=mWlb_Sp2F8g
4. Photo city : http://www.photocitygame.com/
5. How to overlay a 3D model on interactive Google Street View panoramas :
5: From Google Street View to 3D City models : https://www.researchgate.net/publication/224135244_From_Google_Street_View_to_3D_City_models