In order to improve the prediction effect of chemical oxygen demand (COD) in sewage treatment, a BP neural network prediction model optimized by elephant herding optimization (EHO) is proposed. By combining the separation operation in the elephant herding optimization algorithm with the particle swarm optimization (PSO) algorithm with improved weight, the individuals with poor fitness in the population can be effectively removed to further improve the algorithm’s ability to find the optimal value. The improved elephant herding optimization algorithm is used to optimize the BP neural network for the first time to better approximate the data and improve the prediction accuracy of the prediction model. Finally, the simulation results clearly show that the improved BP neural network has higher prediction accuracy than the traditional BP neural network and the general wavelet neural network. The prediction model based on the improved elephant herding optimizlation algorithm and BP neural network can accurately predict the chemical oxygen demand of effluent in sewage treatment to a certain extent, which can meet the general requirements of predicting the chemical oxygen demand of effluent and which has certain research value.