Aiming at the rapid development of urbanization, the identification between cities and cities is getting weaker and weaker, and the concept of urban landmarks is becoming more and more popular, This paper proposes a building recognition method based on deep learning. Using the improved Faster R-CNN algorithm as the training model, first, we input the image to be identified into the CNN network and extract the feature block diagram. Then, the model predicts the regional recommendations of the location of the target building through the RPN network, and maps these regional recommendations to the feature block diagram, the RoI Pooling layer converts these regional recommendations into fixed-size feature blocks. Finally, we use non-maximum suppression to remove similar results from the prediction bounding box to get the predicted bounding box, the category and probability of the target building in the border. The experimental results show that the recognition rate of landmark buildings can reach 908% under the condition that the training data set is sufficient. Compared with other models, the model has a good recognition effect.