A Hybrid Metaheuristic Approach for Code Smell Refactoring Sequencing in Object-Oriented Systems

Abstract

Code smells must be identified to assess whether a software system has architectural flaws that impede maintainability, extensibility, and overall quality improvement. A tried-and-true method for eliminating such irregularities while maintaining the same external functionality is refactoring. However, because of their interdependencies and various quality goals, figuring out the best order for refactoring operations is still a challenging issue. A hybrid metaheuristic optimization framework for the automated identification of refactoring sequences in object-oriented software systems is presented in this paper. The suggested method combines the sooty tern optimization algorithm (STOA) with the spotted hyena optimization (SHO) algorithm. In the early stages of the search, the hybrid strategy makes use of STOA’s high exploratory capabilities, while in the later stages, it makes good use of SHO’s potent exploitative strengths. A discrete refactoring sequence can be converted into a continuous optimization issue using a priority-driven encoding approach. Cohesion, coupling, complexity, and code smell minimization measures are combined to create a complete multi-objective fitness function. For object-oriented applications, experimental results show that the suggested STOA–SHO hybrid strategy works better than standalone metaheuristic strategies in terms of lowering code smells, speeding up convergence, and improving software quality indicators in a balanced and effective way.

Received: 27 May 2025 | Revised: 12 January 2026 | Accepted: 29 January 2026

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Author Contribution Statement

Ritika Maini: Conceptualization, Software, Validation, Formal analysis, Writing – original draft, Visualization. Navdeep Kaur: Data curation, Supervision, Project administration. Amandeep Kaur: Methodology, Investigation, Resources, Writing – review & editing.