ORIGINAL RESEARCH
Hybrid Transformer-TimesNet Model for Accurate Prediction of Industrial Air Pollutants: A Case Study of the Xinyang Industrial Zone, China
,
 
,
 
 
 
 
More details
Hide details
1
Guangdong University of Science and Technology, Dongguan, China
 
2
Shaoguan University, Shaoguan, China
 
 
Submission date: 2024-11-22
 
 
Final revision date: 2025-01-31
 
 
Acceptance date: 2025-03-25
 
 
Online publication date: 2025-05-07
 
 
Corresponding author
Siyuan He   

School of Management, Guangdong University of Science and Technology, Dongguan, China
 
 
Guanwei Jang   

Shaoguan University, Shaoguan, China
 
 
 
KEYWORDS
TOPICS
ABSTRACT
Air pollution in industrial zones poses significant threats to environmental and public health, especially in developing regions, highlighting the necessity for accurate forecasting to guide pollution management. This study presents an Optuna-enhanced hybrid Transformer-TimesNet model aimed at improving time series forecasting of six critical pollutants (SO2, NO2, CO, O3, PM10, and PM2.5) in Xinyang Industrial Zone, Xiamen, China. Utilizing air quality data from 2019 to 2023, the model combines the Transformer’s strength in capturing long-range dependencies with TimesNet’s expertise in handling complex temporal patterns. Advanced preprocessing techniques were employed to address both linear and non-linear data components, and Optuna was used for hyperparameter tuning, enhancing model stability and predictive accuracy. Comparative experiments demonstrated the hybrid model’s superior performance against traditional statistical methods, machine learning models, and deep learning approaches, evaluated through metrics such as MAE, RMSE, SMAPE, and R2. The model’s capability to accurately capture long-term pollutant trends underscores its reliability and validity as a predictive tool for policymakers and environmental managers. These results contribute to theoretical advancements in environmental monitoring and offer practical solutions for public health protection and pollution mitigation, demonstrating the potential of hybrid deep learning models in addressing complex forecasting challenges.
CONFLICT OF INTEREST
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
REFERENCES (53)
1.
CHEN S., OLIVA P., ZHANG P. Air pollution and mental health: evidence from China. AEA Papers and Proceedings, 114, 423, 2024. https://doi.org/10.1257/pandp.....
 
2.
KAUR J., PARMAR K.S., SINGH S. Autoregressive models in environmental forecasting time series: a theoretical and application review. Environmental Science and Pollution Research, 30 (8), 19617, 2023. https://doi.org/10.1007/s11356....
 
3.
LUO J., GONG Y. Air pollutant prediction based on ARIMA-WOA-LSTM model. Atmospheric Pollution Research, 14 (6), 101761, 2023. https://doi.org/10.1016/j.apr.....
 
4.
NATH P., SAHA P., MIDDYA A.I., ROY S. Long-term time-series pollution forecast using statistical and deep learning methods. Neural Computing and Applications, 1, 2021. https://doi.org/10.1007/s00521....
 
5.
GARG N., SONI K., SAXENA T.K., MAJI S. Applications of Autoregressive integrated moving average (ARIMA) approach in time-series prediction of traffic noise pollution. Noise Control Engineering Journal, 63 (2), 182, 2015. https://doi.org/10.3397/1/3763....
 
6.
GREKOV A.N., VYSHKVARKOVA E.V., MAVRIN A.S. Forecasting and Anomaly Detection in BEWS: Comparative Study of Theta, Croston, and Prophet Algorithms. Forecasting, 6 (2), 343, 2024. https://doi.org/10.3390/foreca....
 
7.
MÉNDEZ M., MERAYO M.G., NÚÑEZ M. Machine learning algorithms to forecast air quality: a survey. Artificial Intelligence Review, 56 (9), 10031, 2023. https://doi.org/10.1007/s10462....
 
8.
SETHI J.K., MITTAL M. An efficient correlation based adaptive LASSO regression method for air quality index prediction. Earth Science Informatics, 14 (4), 1777, 2021. https://doi.org/10.1007/s12145....
 
9.
OBIDALLAH W.J. Artificial intelligence modeling and simulation of membrane-based separation of water pollutants via ozone Process: Evaluation of separation. Thermal Science and Engineering Progress, 51, 102627, 2024. https://doi.org/10.1016/j.tsep....
 
10.
JI Y., ZHI X., WU Y., ZHANG Y., YANG Y., PENG T., JI L. Regression analysis of air pollution and pediatric respiratory diseases based on interpretable machine learning. Frontiers in Earth Science, 11, 1105140, 2023. https://doi.org/10.3389/feart.....
 
11.
ZHANG X., DING C., WANG G. An AutoregressiveBased Kalman Filter Approach for Daily PM2. 5 Concentration Forecasting in Beijing, China. Big Data, 12 (1), 19, 2024. https://doi.org/10.1089/big.20....
 
12.
KOTHANDARAMAN D., PRAVEENA N., VARADARAJKUMAR K., MADHAV RAO B., DHABLIYA D., SATLA S., ABERA W. Intelligent forecasting of air quality and pollution prediction using machine learning. Adsorption Science & Technology, 2022, 5086622, 2022. https://doi.org/10.1155/2022/5....
 
13.
HE S., WU J., WANG D., HE X. Predictive modeling of groundwater nitrate pollution and evaluating its main impact factors using random forest. Chemosphere, 290, 133388, 2022. https://doi.org/10.1016/j.chem....
 
14.
ESSAMLALI I., NHAILA H., EL KHAILI M. Supervised Machine Learning Approaches for Predicting Key Pollutants and for the Sustainable Enhancement of Urban Air Quality: A Systematic Review. Sustainability, 16 (3), 976, 2024. https://doi.org/10.3390/su1603....
 
15.
KE G., MENG Q., FINLEY T., WANG T., CHEN W., MA W., YE Q., LIU T.Y. Lightgbm: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30, 2017.
 
16.
DOROGUSH A.V., ERSHOV V., GULIN A. CatBoost: gradient boosting with categorical features support. arXiv preprint arXiv, 1810, 11363, 2018.
 
17.
MA J., YU Z., QU Y., XU J., CAO Y. Application of the XGBoost machine learning method in PM2.5 prediction: a case study of Shanghai. Aerosol and Air Quality Research, 20 (1), 128, 2020. https://doi.org/10.4209/aaqr.2....
 
18.
WANG J., LU Y., XIN C., YOO C., LIU H. Kernel PLS with AdaBoost ensemble learning for particulate matters forecasting in subway environment. Measurement, 204, 111974, 2022. https://doi.org/10.1016/j.meas....
 
19.
CHAO B., GUANG QIU H. Air pollution concentration fuzzy evaluation based on evidence theory and the K-nearest neighbor algorithm. Frontiers in Environmental Science, 12, 1243962, 2024. https://doi.org/10.3389/fenvs.....
 
20.
LIAO Q., ZHU M., WU L., PAN X., TANG X., WANG Z. Deep learning for air quality forecasts: a review. Current Pollution Reports, 6, 399, 2020. https://doi.org/10.1007/s40726....
 
21.
BROWNLEE J. Deep learning for time series forecasting: predict the future with MLPs, CNNs and LSTMs in Python. Machine Learning Mastery, 2018.
 
22.
ULPIANI G., DUHIRWE P.N., YUN G.Y., LIPSON M.J. Meteorological influence on forecasting urban pollutants: Long-term predictability versus extreme events in a spatially heterogeneous urban ecosystem. Science of the Total Environment, 814, 152537, 2022. https://doi.org/10.1016/j.scit....
 
23.
AGGARWAL A., TOSHNIWAL D. A hybrid deep learning framework for urban air quality forecasting. Journal of Cleaner Production, 329, 129660, 2021. https://doi.org/10.1016/j.jcle....
 
24.
SARAVANAN D., KUMAR K.S. Improving air pollution detection accuracy and quality monitoring based on bidirectional RNN and the Internet of Things. Materials Today: Proceedings, 81, 791, 2023. https://doi.org/10.1016/j.matp....
 
25.
LUO J., ZHANG Z., FU Y., RAO F. Time series prediction of COVID-19 transmission in America using LSTM and XGBoost algorithms. Results in Physics, 27, 104462, 2021. https://doi.org/10.1016/j.rinp....
 
26.
AHMED A.A.M., JUI S.J.J., SHARMA E., AHMED M.H., RAJ N., BOSE A. An advanced deep learning predictive model for air quality index forecasting with remote satellite-derived hydro-climatological variables. Science of The Total Environment, 906, 167234, 2024. https://doi.org/10.1016/j.scit....
 
27.
LIANG Y.C., MAIMURY Y., CHEN A.H.L., JUAREZ J.R.C. Machine learning-based prediction of air quality. Applied Sciences, 10 (24), 9151, 2020. https://doi.org/10.3390/app102....
 
28.
TSOKOV S., LAZAROVA M., ALEKSIEVA-PETROVA A. A hybrid spatiotemporal deep model based on CNN and LSTM for air pollution prediction. Sustainability, 14 (9), 5104, 2022. https://doi.org/10.3390/su1409....
 
29.
WU C.L., SONG R.F., ZHU X.H., PENG Z.R., FU Q.Y., PAN J. A hybrid deep learning model for regional O3 and NO2 concentrations prediction based on spatiotemporal dependencies in air quality monitoring network. Environmental Pollution, 320, 121075, 2023. https://doi.org/10.1016/j.envp....
 
30.
KIM J., WANG X., KANG C., YU J., LI P. Forecasting air pollutant concentration using a novel spatiotemporal deep learning model based on clustering, feature selection and empirical wavelet transform. Science of the Total Environment, 801, 149654, 2021. https://doi.org/10.1016/j.scit....
 
31.
ZHOU H., ZHANG S., PENG J., ZHANG S., LI J., XIONG H., ZHANG W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, 35 (12), 11106, 2021. https://doi.org/10.1609/aaai.v....
 
32.
REZA S., FERREIRA M.C., MACHADO J.J., TAVARES J.M.R. A multi-head attention-based transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networks. Expert Systems with Applications, 202, 117275, 2022. https://doi.org/10.1016/j.eswa....
 
33.
WU N., GREEN B., BEN X., O'BANION S. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv, 2001, 08317, 2020.
 
34.
ZENG A., CHEN M., ZHANG L., XU Q. Are transformers effective for time series forecasting? In Proceedings of the AAAI Conference on Artificial Intelligence, 37 (9), 11121, 2023. https://doi.org/10.1609/aaai.v....
 
35.
LIANG Y., XIA Y., KE S., WANG Y., WEN Q., ZHANG J., ZHENG Y., ZIMMERMANN R. Airformer: Predicting nationwide air quality in china with transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, 37 (12), 14329, 2023. https://doi.org/10.1609/aaai.v....
 
36.
ZHANG Z., ZHANG S. Modeling air quality PM2.5 forecasting using deep sparse attention-based transformer networks. International Journal of Environmental Science and Technology, 20 (12), 13535, 2023. https://doi.org/10.1007/s13762....
 
37.
AL-QANESS M.A., DAHOU A., EWEES A.A., ABUALIGAH L., HUAI J., ABD ELAZIZ M., HELMI A.M. ResInformer: residual transformer-based artificial time-series forecasting model for PM2. 5 concentration in three major Chinese cities. Mathematics, 11 (2), 476, 2023. https://doi.org/10.3390/math11....
 
38.
WU H., HU T., LIU Y., ZHOU H., WANG J., LONG M. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv, 2210, 02186, 2022.
 
39.
ZHANG X., YANG K., ZHENG L. Transformer Fault Diagnosis Method Based on TimesNet and Informer. Actuators, 13 (2), 74, 2024. https://doi.org/10.3390/act130....
 
40.
ZHOU T., MA Z., WEN Q., WANG X., SUN L., JIN R. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, 27268, 2022.
 
41.
ZUO C., WANG J., LIU M., DENG S., WANG Q. An ensemble framework for short-term load forecasting based on timesnet and tcn. Energies, 16 (14), 5330, 2023. https://doi.org/10.3390/en1614....
 
42.
SALMAN D., DIREKOGLU C., KUSAF M., FAHRIOGLU M. Hybrid deep learning models for time series forecasting of solar power. Neural Computing and Applications, 1, 2024. https://doi.org/10.1007/s00521....
 
43.
LOSHCHILOV I. Decoupled weight decay regularization. arXiv preprint arXiv, 1711, 05101, 2017.
 
44.
KINGMA D.P. Adam: A method for stochastic optimization. arXiv preprint arXiv, 1412, 6980, 2014.
 
45.
AKIBA T., SANO S., YANASE T., OHTA T., KOYAMA M. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2623, 2019. https://doi.org/10.1145/329250....
 
46.
SNOEK J., LAROCHELLE H., ADAMS R.P. Practical bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 25, 2012.
 
47.
EMMERICH M.T., DEUTZ A.H. A tutorial on multiobjective optimization: fundamentals and evolutionary methods. Natural Computing, 17, 585, 2018. https://doi.org/10.1007/s11047....
 
48.
TAX D.M., DUIN R.P. Support vector data description. Machine Learning, 54, 45, 2004. https://doi.org/10.1023/B:MACH....
 
49.
KHAN S.S., MADDEN M.G. One-class classification: taxonomy of study and review of techniques. The Knowledge Engineering Review, 29 (3), 345, 2014. https://doi.org/10.1017/S02698....
 
50.
HUANG Y., ZHOU Z., WANG Z., ZHI X., LIU X. TimesNet-PM2.5: Interpretable TimesNet for Disentangling Intraperiod and Interperiod Variations in PM2. 5 Prediction. Atmosphere, 1 (11), 1604, 2023. https://doi.org/10.3390/atmos1....
 
51.
CHEUNG Y.W., LAI K.S. Lag order and critical values of the augmented Dickey-Fuller test. Journal of Business & Economic Statistics, 13 (3), 277, 1995. https://doi.org/10.2307/139218....
 
52.
WU Y., HU J., IRFAN M., HU M. Vertical decentralization, environmental regulation, and enterprise pollution: an evolutionary game analysis. Journal of Environmental Management, 349, 119449, 2024. https://doi.org/10.1016/j.jenv....
 
53.
ELBAZ K., HOTEIT I., SHABAN W.M., SHEN S.L. Spatiotemporal air quality forecasting and health risk assessment over smart city of NEOM. Chemosphere, 313, 137636, 2023. https://doi.org/10.1016/j.chem....
 
 
CITATIONS (3):
1.
Sales Forecasting of FAW-Audi Based on a Hybrid LSTM-XGBoost Model
Siyuan He, Weiguo Liang, Yuhao Liu, Chen Gjun Chen
2025 8th International Conference on Big Data and Artificial Intelligence (BDAI)
 
2.
A CEEMDAN-GNN-transformer hybrid model for air quality index forecasting: a case study of Chang’an town, Dongguan, China
Siyuan He, Yuhao Liu, Jin Peng, Dean Xu, Yu Wang
Frontiers in Environmental Science
 
3.
Transformers for Multivariate Time Series Forecasting: Comprehensive Analysis, Challenges, Research Opportunities, and Future Prospects
M. K. Saravana, M. S. Roopa, J. S. Arunalatha, K. R. Venugopal
IEEE Access
 
eISSN:2083-5906
ISSN:1230-1485
Journals System - logo
Scroll to top