Aiming at solving the problems of high false detection rate and low reliability of traditional single feature-based fatigue detection methods, which cannot adapt to the complex and changing driving environment, a fatigue driving detection method that fuses drivers multiple facial features such as driver's eyes and mouth is proposed. Compared with the existing face detection models, the gradient-based learning framework proposed here is more effective for side face detection and can better meet the detection time requirements. Meanwhile, the improved LeNet-5 neural network model is used to classify the smiles in videos, which excludes the interference of expression changes on fatigue driving detection. Finally, in order to reduce the influence of head posture deflection on fatigue feature extraction, the Euler angle-based feature correction algorithm was introduced. The detection results of the YawDD fatigue driving dataset show that the fatigue driving detection based on multi-feature fusion in different postures can not only effectively reduce the influence of head deflection on fatigue driving detection, but also has higher robustness than the traditional fatigue detection methods.