Objective In order to study the effect of the screening parameters of the vibrating screen on the screening efficiency of the vibrating screen and the motion law of the screening particles in the screening process the parameters of the vibrating screen were optimized. Methods A three-dimensional model of the vibrating screen was established the screening process of the vibrating screen was simulated by using discrete element analysis software EDEM and multiple groups of vibrating screen screening tests were designed by using the orthogonal test method. The effect of the three kinematic parameters of the vibrating screen namely amplitude vibration frequency and vibration direction angle on the screening efficiency was analyzed by the discrete unit method. Multivariate nonlinear fitting was performed on the orthogonal test results and the ant colony algorithm was used to find the vibration parameters corresponding to the optimal screening efficiency based on the fitting function. Results The results showed that the increase in vibration frequency led to the excessively large beating times of particles in the screening box and the particles were always in the bouncing state which reduced the probability of particles passing through sieve holes and the screening efficiency. The influence of vibration frequency on screening efficiency was consistent with that of amplitude. With the increase of vibration frequency the jitter of the screen surface was intensified the force on particles was increased and the time of particles in the air was increased which led to the decrease in the probability of particles entering the sieve holes and the decrease of screening efficiency. When the vibration direction angle increased the screening efficiency increased first and then decreased.When the direction angle was too small the particles popped out in the direction almost parallel to the screen the total contact time with the screen decreased and the sand particles passing through the sieve holes decreased. When the direction angle was too large the particles popped perpendicular to the sieve surface the screening speed became slower the particles tended to accumulate on the sieve surface and the chance of contact between the particles that could pass through the sieve holes and the sieve surface decreased during the screening process. Finally the particles passing through the sieve holes were reduced and the screening efficiency decreased. Conclusion When the amplitude is 2 mm the frequency is 18 Hz and the vibration direction angle is 44° the screening effect of the vibrating screen is the best. This study has a certain guiding significance for the optimization design of the vibrating screen.