Objective This study aims to improve the efficiency of cement production realize real-time fault diagnosis of cement roller press bearings and solve the problem that it is difficult to obtain a large amount of labeled fault data for training in the engineering field. Methods A Transformer model using continuous wavelet convolution layers is proposed. The model enhances the network?? s learning ability for mechanical fault features by using wavelet kernels instead of traditional convolutional kernels in the first layer of the feature extraction network. It then utilizes mask-training selfsupervised learning to enable the network to learn useful relevant features from a large amount of unlabeled data. Additionally the model is trained using a designed contrastive loss transfer learning strategy to learn domain-general discriminative features from relevant public data. Finally through fine-tuning the model completes training of the cement roller press fault diagnosis network. Results Experimental validation shows that this model effectively accelerates diagnostic progress. With the self-supervised learning and transfer learning training strategies the model achieves an accuracy of 78% in fine-tuning with only 1% of the training set data approaching the performance of supervised training using 100% of the data with only 10% of the training set data. Conclusion This method effectively addresses the challenge of acquiring large amounts of labeled fault data in engineering practice improving diagnostic accuracy and maintaining diagnostic precision with limited target data.