Ports play an important role in transportation networks, but the ecological efficiency of many ports cannot reach an effective level. To improve the ecological efficiency of ports, we need to analyze the current ecological efficiency of ports and propose policy recommendations based on the results. In this paper, we first employed the super-efficiency SBM model to calculate the ecological efficiency of the four major ports in the Yangtze River Delta port group from 2011 to 2021. The results showed that the average comprehensive ecological efficiency of the four major ports was 0.647, which did not reach the effective level. The overall ecological efficiency curve was U-shaped. Then we used the Tobit model to analyze the key factors affecting the port’s ecological efficiency. We found that the economic development of port cities could promote the improvement of port ecological efficiency. The application level of emerging technologies also had a positive impact, but the impact was weak. However, expanding the scale of the port would reduce its ecological efficiency. At last, we provided some policy recommendations.