The process of pursuing high-quality economic development involves many moving parts, and economic agglomeration (EA) is an essential element that can significantly affect green total factor productivity (GTFP). This study uses SBM and GML models to calculate GTFP of 99 cities in the Yellow River Basin from 2004 to 2017. Then, based on the panel data of these cities, the SYS-GMM and the mediating effect model are used to empirically test the influence and action path of EA on GTFP. The conclusions are as follows: There is an evident inverted U-shaped link between EA and GTFP in the YRB. The development level of the upper and middle reaches of the YRB is lower than the “ agglomeration inflection point ” (CNY 1.231 billion/KM²), while the downstream region has crossed this inflection point and has a crowding effect. In addition, EA affects GTFP through labor pool, intermediate input sharing and knowledge technology spillover. This study concludes that, in order to boost GTFP in the YRB, diversified strategies based on regional development are required to avoid the crowding impact.