Guiding poor farmers to participate in characteristic agriculture development is of significance to realizing total poverty eradication in poor area. This paper empirically analyzes the influence factors of farmers’ participation in characteristic agricultural project by Probit model and Multinomial Probit model, based on survey data of 410 farmers households in Wuling Mountain Area. The results show that the proportion of agricultural income and agricultural acreage are important factors to promote farmers to participate in characteristic agriculture projects. Poverty status has a significant inhibitory effects. Policy support has an opposite effect on farmers’ willingness and behavior to participate. Compared with the farmers without participation, poverty status, geographical location and policy support have a positive effect on farmers in the selection of transient participation. Family population size, agricultural acreage and policy support are important factors to promote farmers to conduct sustainable participation and poverty status has an opposite effects. Therefore, the government should strengthen the policy support for poor farmers to improve farmers’ participation expectations. Meanwhile, the government also should guide farmers to actively transfer land and develop professionally.