Objective This study addresses the challenges of annotating data for specific tweet sentiment analysis tasks and the issues arising from inaccurate annotations leading to unsatisfactory classification results. A method for machine annotation of data was proposed to investigate the performance differences in sentiment classification of tweets annotated by human annotators and the ChatGPT model. Methods In this study for a unified labeling system tweet data was annotated both manually and using the ChatGPT model interface followed by sentiment classification using a BERT- TextCNN hybrid deep learning model on both datasets. Results Experimental results indicated that the manually annotated dataset exhibited higher overall accuracy and reliability. However for certain tweet data the ChatGPT model with its richer knowledge base can produce more objective and scientifically interpretable annotations showing certain advantages in sentiment classification results. Both human and machine annotation methods have their strengths and weaknesses. Therefore it can be concluded that machine annotation is a feasible labeling method for text sentiment classification tasks. Conclusion In practical applications it is advisable to flexibly choose and combine both annotation methods based on task requirements and fully leverage the strengths of these two methods to achieve better analytical performance and outcomes.