Intelligent Automated Feature Engineering and Model Optimization Using Reinforcement Learning for Large-Scale Predictive Analytics Applications
DOI:
https://doi.org/10.63282/3050-9246.IJETCSIT-V7I3P108Keywords:
Automated Feature Engineering, Reinforcement Learning, Automl, Model Optimization, Predictive Analytics, Hyperparameter Optimization, Feature Selection, Machine Learning, Intelligent OptimizationAbstract
Feature engineering, feature selection, model selection and hyperparameter tuning are typically large and manual tasks that can be costly in terms of computational resources, time to develop and suboptimal predictive performance in large-scale predictive analytics applications. Typically, conventional automatic machine learning technologies can minimize the need for manual participation, but often require a pre-designed search strategy that may not be able to adjust well to complex and evolving optimization spaces. In this work, these drawbacks are overcome by proposing an intelligent reinforcement-learning-based framework for the simultaneous task of feature engineering and predictive model optimization. The proposed approach treats the feature transformation, feature selection, model selection, and hyperparameter configuration as sequential decision-making problems, allowing a reinforcement learning agent to explore different combinations of them and learn optimal strategies based on the reward feedback when they are performed. The proposed framework combines automated feature generation and selection with adaptive model and hyperparameter optimization to find optimal predictive configurations that balance predictive performance and computational efficiency. The framework is tested with big datasets of predictive analytics and is contrasted to traditional feature engineering, standard machine learning optimization, and automated optimization methods. The measure of the performance depends on the prediction task, and it is evaluated by predictive accuracy, precision, recall, F1-score, RMSE, computational time, convergence rate, and resource utilization. Experimental results show that reinforcement learning is effective in reducing unnecessary combinations of features, as well as in tuning better models and selecting them efficiently, while maintaining competitive prediction performance with lower optimization costs. The main contribution is an integrated intelligent optimization framework, combining automated feature engineering and model optimization with reinforcement learning, enabling the efficient predictive analytics on large and complex datasets.
References
[1] Fayaz, S. A., Jahangeer Sidiq, S., Zaman, M., & Butt, M. A. (2022). Machine learning: An introduction to reinforcement learning. Machine learning and data science: Fundamentals and applications, 1-22.
[2] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Hassabis, D. (2015). Human-level control through deep reinforcement learning. nature, 518(7540), 529-533.
[3] Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., ... & Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971.
[4] Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
[5] Vincent, F. L., Peter, H., Riashat, I., Marc, G. B., & Joelle, P. (2018). An introduction to deep reinforcement learning. Foundations and trends® in machine learning, 11(3-4), 219-354.
[6] Thornton, C., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2013, August). Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 847-855).
[7] Olson, R. S., Urbanowicz, R. J., Andrews, P. C., Lavender, N. A., Kidd, L. C., & Moore, J. H. (2016, March). Automating biomedical data science through tree-based pipeline optimization. In European conference on the applications of evolutionary computation (pp. 123-137). Cham: Springer International Publishing.
[8] Olson, R. S., & Moore, J. H. (2016, December). TPOT: A tree-based pipeline optimization tool for automating machine learning. In Workshop on automatic machine learning (pp. 66-74). PMLR.
[9] Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., & Hutter, F. (2015). Efficient and robust automated machine learning. Advances in neural information processing systems, 28.
[10] Zöller, M. A., & Huber, M. F. (2021). Benchmark and survey of automated machine learning frameworks. Journal of artificial intelligence research, 70, 409-472.
[11] Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011, January). Sequential model-based optimization for general algorithm configuration. In International conference on learning and intelligent optimization (pp. 507-523). Berlin, Heidelberg: Springer Berlin Heidelberg.
[12] Falkner, S., Klein, A., & Hutter, F. (2018, July). BOHB: Robust and efficient hyperparameter optimization at scale. In International conference on machine learning (pp. 1437-1446). PMLR.
[13] Eggensperger, K., Müller, P., Mallik, N., Feurer, M., Sass, R., Klein, A., ... & Hutter, F. (2021). HPOBench: A collection of reproducible multi-fidelity benchmark problems for HPO. arXiv preprint arXiv:2109.06716.
[14] Lindauer, M., Eggensperger, K., Feurer, M., Biedenkapp, A., Deng, D., Benjamins, C., ... & Hutter, F. (2022). SMAC3: A versatile Bayesian optimization package for hyperparameter optimization. Journal of Machine Learning Research, 23(54), 1-9.
[15] Snoek, J., Larochelle, H., & Adams, R. (2012). Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25.
[16] Brochu, E., Cora, V. M., & De Freitas, N. (2010). A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599.
[17] Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & De Freitas, N. (2015). Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1), 148-175.
[18] Yang, Z., & Zhang, A. (2021). Hyperparameter optimization via sequential uniform designs. Journal of Machine Learning Research, 22(149), 1-47.
[19] Merrill, E., Fern, A., Fern, X., & Dolatnia, N. (2021). An empirical study of bayesian optimization: Acquisition versus partition. Journal of Machine Learning Research, 22(4), 1-25.
[20] Gijsbers, P., Bueno, M. L., Coors, S., LeDell, E., Poirier, S., Thomas, J., ... & Vanschoren, J. (2024). Amlb: an automl benchmark. Journal of Machine Learning Research, 25(101), 1-65.
[21] Bouadi, M., Alavi, A., Benbernou, S., & Ouziri, M. (2024). Semantic-Guided RL for Interpretable Feature Engineering. arXiv preprint arXiv:2410.02519.
[22] Hutter, F., Kotthoff, L., & Vanschoren, J. (2019). Automated machine learning: methods, systems, challenges (p. 219). Springer Nature.
[23] Brahmandam, L. M. K. (2024). An Empirical Evaluation of the Medallion Architecture on Databricks and Apache Spark with Snowflake: Throughput, Latency, and Cost for Batch and Real-Time Ingestion Patterns. International Journal of AI, BigData, Computational and Management Studies, 5(3), 197-206. https://doi.org/10.63282/3050-9416.IJAIBDCMS-V5I3P122
[24] Paruchuri, J. K. (2024). Apache Kyuubi on Kubernetes: Building Elastic Multi-Tenant Spark SQL Platforms. INDO-CONTINENTAL ACADEMIC PUBLISHERS.
[25] Sunkara, R. (2026). Serverless Architecture Patterns for Enterprise AI Agents: ECS Fargate, OpenSearch k-NN, and DynamoDB for Knowledge-Grounded LLM Workflows. International Journal of AI, BigData, Computational and Management Studies, 7(2), 197-201. https://doi.org/10.63282/3050-9416.IJAIBDCMS-V7I2P129
[26] Gantikota, S. (2023). Reducing HL7 Processing Errors through Automated File Creation and Ingestion Pipelines: A Production Case Study in EHR Data Integration. International Journal of Emerging Trends in Computer Science and Information Technology, 4(4), 241-245. https://doi.org/10.63282/3050-9246.IJETCSIT-V4I4P125
[27] Sandra, K. (2022). Real-Time Stream Processing with Apache Flink vs Spark Structured Streaming: An Enterprise Comparison.
