Rapid urbanization has led to increased land surface temperature (LST) and severe urban heat islands (UHIs). The impacts of land use/land cover (LULC) on LST have been extensively studied. However, the differences between land use and land cover and their implications in an urban environment are often overlooked. Taking the example of Shanghai in China, this study aimed to study the effects from land use types and land cover patterns on LST along urbanization gradients. The LST and LULC data of the study area were obtained from a Landsat ETM+ image and Map world Shanghai, respectively. Then, landscape metrics were selected and calculated for analyzing the land cover patterns. Correlation analysis and regression analysis were undertaken to determine the relationship between LST and land cover patterns at the land use level. The results showed that it was inadequate to treat land cover as the single factor affecting LST. Furthermore, LST values did not simply decrease along the urbanization gradients at the land use level. Even though land cover patterns significantly affected LST, land cover variables related to LST varied greatly among the various land use types. The findings in our study provide additional knowledge for optimizing land cover patterns associated with different land use types, which may mitigate the adverse impacts of UHIs at a fine scale.