To alleviate the plight of municipal solid waste management, penalties were implemented in China. However, it is unclear how penalty affects residents’ waste separation behavior. This study aims to analyze the impact of penalty on residents’ waste separation behavior and the role of personal norms and trust in authorities. A theoretical model of the impact of penalty on residents’ waste separation behavior was established based on deterrence theory and normative activation model. Using online survey data from 628 respondents in Zhengzhou, China, an empirical analysis was performed by employing the moderated mediation model. The results showed that residents’ perceived penalty certainty and penalty severity positively influenced their waste separation behavior through the mediation effect of personal norm. Penalty certainty had a direct impact on waste separation behavior, whereas penalty severity had no direct impact on waste separation behavior. Trust in authorities positively moderated the mediation effect of personal norm. The higher the trust in authorities, the stronger mediating effect of personal norm. Some recommendations were proposed to improve residents’ waste separation behavior under the mandatory policy. This paper presents a new perspective and theoretical guidelines for the local government to improve residents’ waste separation behaviors in China and offers useful insights into waste separation management for other countries.