Updating digital models of existing commercial buildings using deep learning




Czerniawski, Thomas

Journal Title

Journal ISSN

Volume Title



The vast scale of buildings poses many challenges to the flow of information. Automatically creating and updating building information models will require the adoption of computer vision systems capable of parsing a comprehensive set of building components all the while being resilient to imperfections inherent in large-scale data collection. Existing visual recognition methods have essentially been limited to recognizing floors, walls, ceilings, doors, and windows. To address this limitation, I show how a deep convolutional neural network can be trained to semantically segment RGB-D (i.e. color and depth) images into thirteen building component classes using a new annotated dataset called 3DFacilities. The dataset was designed using a common building taxonomy to ensure comprehensive semantic coverage and was collected from a diversity of buildings to ensure intra-class diversity. Transfer learning, class balancing, and prevention of overfitting are used to effectively overcome the dataset’s borderline adequate class representation. The second challenge is imperfect data collection. In updating building information models, changes in the built environment need to be detected by identifying differences between a building information model and a collected point cloud. However, association of subsets of building information models and point clouds becomes ambiguous when the point cloud is incomplete due to resource limited scanning paths, specular surfaces, and occlusion. To address this challenge, I show how point cloud completion using a hierarchical deep variational autoencoder improves the accuracy of change detection. The final challenge involves facilitating widespread adoption of building information modeling and innovative computer vision solutions. To address this challenge, I show how a framework inspired by concepts from reinforcement learning, can be used to quantify the otherwise nebulous cultural factors dictating an organization’s propensity for development. The utility of the framework is demonstrated by analyzing audio data from semi-structured interviews with facility managers. The contributions presented herein demonstrate how information systems can be made to contend with the immense spatial and temporal scale of buildings and how organizations must develop the requisite culture to successfully adopt them


LCSH Subject Headings