With the vigorous development of fund industry, fund has become an important part of China’s capital market. The increasingly fierce competition in the fund market makes the risk behavior of fund managers attract much attention. Existing literature mainly studies the influence of competition on fund managers’ risk-taking behavior from the perspective of fund performance ranking based on the analytical paradigm of “performance-capital flow-return-risk-taking”. However, few literatures study the influence of market competition intensity on fund managers’ risk-taking behavior, let alone relevant empirical evidence. This paper argues that competition among peer funds (funds with similar styles) has a significant impact on fund managers’ risk-taking behavior. Under the circumstance of uncertain fund investment returns and the substitution effect of peer funds, the intensification of peer competition will enhance the cover-up of investors’ fund style preference on fund performance preference, leading to lower investors’ pursuit of performance, weakening the incentive effect of the “performance-capital flow-return” mechanism, and inhibiting fund managers’ risk-taking motives and behaviors. In a situation where the outcome of risk-taking behavior is uncertain and competition has diminishing returns, increased peer competition will lead to a decline in fund performance, reduce the expected return of risk-taking behavior, and increase the risk of fund managers being dismissed, thereby inhibiting fund managers’ risk-taking motivation and behavior. Based on the data of China open-end funds from 2005 to 2020, it is found that the number of peer funds is significantly negatively correlated with the risk adjustment ratio, the concentration degree of heavy holdings and the degree of market deviation, which indicates that the intensification of peer competition will reduce the risk level of fund portfolio. The fund performance is negatively correlated with the average fund flow of the peer fund in the next period, indicating that the substitution effect of the peer fund is obvious. The increase in the number of peer funds will weaken the positive effect of fund’s excess returns on fund flows, indicating that the intensification of peer competition will reduce investors’ pursuit of performance. The partial mediating effect of fund performance on peer competition on fund managers’ risk-taking behavior is obvious, indicating that the intensification of peer competition can inhibit fund managers’ risk-taking behavior by reducing fund performance. Compared with the existing literature, this paper mainly expands and deepens this study from three aspects: the first is to theoretically discuss the inhibition effect of increasing fund market competition intensity on fund managers’ risk-taking behaviors; the second is to identify the peer fund through the three-dimensional style space network and construct a method to describe the competitive intensity of fund market based on style similarity; the third is to provide empirical evidence for the inhibition effect of fund managers’ risk-taking behaviors by using the data of China’s open-end funds. This paper explains the inhibitory effect of intensified peer competition on fund managers’ risk-taking behaviors and its influencing mechanism, and provides experience and policy inspiration for further optimizing the structure of the fund market and standardizing the management of the fund market. It is necessary to strengthen the macro-control and supervision of fund declaration and fund operation, so as to maintain the fund market competition within a moderate range, and effectively exert the improvement effect of peer competition on fund managers’ behavior, thereby promoting the healthy development of the fund industry and the growth of people’s wealth.