Phosphorus (P) is not only an essential nutrient required for plant growth, but also a pollutant with eutrophication. This brings a significant challenge to farmers, the main executive agent of farming. The study develops a model to determine farmers’ behaviors impacting P security of farming, and analyzes drivers of these behaviors. The results illustrate the farmers’ behaviors including sowing method, application of compound fertilizer, application of phosphate fertilizer, harvesting method, application of seed, and disposal of livestock manure have greatly impacted the P security in the farming system, in turn. Concerning with the factors obviously driving these key behaviors, farmers’ income, prices of goods, application skill, acceptance of skill training, mechanization level, and regulation of waste disposal, etc. are explored, which are classified into four categories: market and purchasing, personal characteristics, planting condition, and regulation. Thus, the study highlights the opportunities for managing P, including improving mechanization level, raising farmers’ environmental awareness, increasing efficiency of fertilizer application, optimizing planting condition, and improving capacity of waste treatment. Moreover, conducting further research on impact mechanism from various stakeholders, more environmental impacts, large temporal and spatial scales, and high quality of data may be necessary in the future.