Satellite remote sensing has low spatial resolution and is easily affected by weather, making monitoring difficult in real-time. Near-ground lidar equipment is expensive, and data processing is complex, so its scope of application is narrow. This paper uses UAV (Unmanned Aerial Vehicle) remote sensing images and hyperspectral recognition models to estimate the impact of cadmium stress (CD stress) on soybean yield, analyze the physiological and ecological characteristics of soybeans, and provide a scientific basis for crop management and environmental pollution control. This paper uses UAV hyperspectral images to collect soybean field data and calibrate CD stress through soil and plant samples. After preprocessing the data, the vegetation index and other features were extracted to analyze the effect of CD stress on soybean growth. A hyperspectral recognition model was constructed to predict the effect of CD stress on soybean yield. The experimental results showed that increased cadmium concentration inhibited soybean photosynthesis and sugar metabolism and aggravated plants’ oxidative damage. The red edge and short-wave infrared bands showed obvious reflectivity changes. Based on the spectral characteristics, a hyperspectral recognition model was constructed. The transformer model was used to effectively classify and identify soybeans under CD stress and predict their yield changes. In the model evaluation, the Transformer model showed excellent performance with an accuracy of 92.8%, a recall rate of 95%, and an F1 score of 92.1%. The 10-fold cross-validation showed that the model performed stably on different data sets, and the accuracy and recall rates remained high. Using UAV remote sensing images combined with the hyperspectral recognition model for soybean physiological and ecological characterization can effectively capture the characteristics of CD stress and provide scientific prediction data for classification and yield estimation.