Objective Mosaic disease is a common symptom in apples. Especially under the condition of large temperature differences between day and night the onset of mosaic disease is rapid which can lead to an increase in the rate of defoliation resulting in a large reduction in apple production and huge economic losses. The number of mosaic disease spots is too high and the scale of the spots varies resulting in problems such as low accuracy of disease identification. Based on this a method that introduces transfer learning and capsule networks was proposed to improve the disease identification rate. Methods Firstly this method expanded and enhanced the obtained mosaic disease dataset and used the Labelme tool to annotate images marking the lesion area and leaf area respectively. Secondly the weight of the trained VGG16 model was transferred to the coding part of U-net through the transfer learning technology and the capsule network was introduced so that the whole network had a stronger feature extraction ability. Then the VGG16 model and capsule network were trained. Finally the trained network model was semantically segmented and test results were output. Results The experimental results showed that the accuracy rate of the original dataset was 87. 51% and the accuracy rate after the introduction of transfer learning was improved to 91. 78% an improvement of 4. 88% the accuracy of introducing the capsule network was improved to 90. 04% an increase of 2. 89% after the introduction of transfer learning and capsule network the accuracy rate was improved to 93. 42% an improvement of 6. 75%. In addition the training time of each round of the model was also improved by 2 s after the introduction of transfer learning. Conclusion According to the experimental results it can be proved that the proposed model after the introduction of transfer learning and capsule network has a certain improvement in identification accuracy compared with the traditional model. Furthermore this method also reduces the model training time in each round and the overall segmentation performance is better.