The assessment of ecological vulnerability plays an important role in the protection and management of scenic tourism sites. In this study, using remote sensing image data in the same month of 2000 and 2017, ecological vulnerability in the Tianchi Scenic Area (TSA) was analyzed by calculating the remote sensing ecological index (RSEI), and the influencing factors of ecological vulnerability were detected using the geographical detector model. From 2000 to 2017, the values of wetness, dryness, and temperature in the study area increased, and the value of greenness and RSEI decreased. Spatial correlations of the RSEI were significant, showing strong spatial agglomeration and alternating cold spots-hot spots-cold spots-hot spots from north to south. The impact factors of land use type, temperature, and precipitation showed greater contribution to ecological vulnerability. In addition, the local government implemented policies such as prohibition of mining, grazing prohibition, and landscape protection, which decreased ecological vulnerability in the study area. This study provides a reference for the formulation of ecological vulnerability protection measures in the TSA and theoretical basis for defining the monitoring indicators for heritage sites around the world.