Exploration-Exploitation Tradeoffs in Metaheuristics: A Review
DOI:
https://doi.org/10.24203/98kbmk88Keywords:
optimizationAbstract
A metaheuristic algorithm is an algorithmic framework at a high level, independent of problems, which offers a set of recommendations or strategies to develop heuristic optimization algorithms.
Evolutionary computation and other metaheuristics have long been focused on the trade-off between exploration and exploitation. This subject is present in a wide range of domains, including modeling and prediction, search and optimization, machine learning and cognition, and many more situations where uncertainty is present. The trade-off between exploration and exploitation is crucial for all optimization techniques. Efficient optimization and lower computing costs can be achieved by striking a fair balance between both.
The paper introduces a study that discusses metaheuristic algorithms and previous studies in three areas: 1) What elements of metaheuristic facilitate exploration and exploitation; 2) When and how exploration control is applied. 3) how to find a balance between the two.
References
E. Cuevas, P. Diaz, and O. Camarena, “Experimental Analysis Between Exploration and Exploitation,” in Intelligent Systems Reference Library, vol. 195, 2021, pp. 249–269. doi: 10.1007/978-3-030-58100-8_10.
K. Hussain, M. N. M. Salleh, S. Cheng, and Y. Shi, “On the exploration and exploitation in popular swarm-based metaheuristic algorithms,” Neural Comput. Appl., 2019, doi: 10.1007/s00521-018-3592-0.
J. Chen, B. Xin, Z. Peng, L. Dou, and J. Zhang, “Optimal contraction theorem for exploration-exploitation tradeoff in search and optimization,” IEEE Trans. Syst. Man, Cybern. Part ASystems Humans, 2009, doi: 10.1109/TSMCA.2009.2012436.
E. Bayona, J. E. Sierra-García, and M. Santos, “Comparative Analysis of Metaheuristic Optimization Methods for Trajectory Generation of Automated Guided Vehicles,” Electron., 2024, doi: 10.3390/electronics13040728.
M. Abdel-Basset, L. Abdel-Fatah, and A. K. Sangaiah, “Metaheuristic algorithms: A comprehensive review,” in Computational Intelligence for Multimedia Big Data on the Cloud with Engineering Applications, 2018. doi: 10.1016/B978-0-12-813314-9.00010-4.
D. A. Marín-Idárraga, J. M. Hurtado González, and C. Cabello Medina, “Factors affecting the effect of exploitation and exploration on performance: A meta-analysis,” BRQ Bus. Res. Q., 2022, doi: 10.1177/2340944420972707.
Z. Michalewicz and D. B. Fogel, How to Solve It: Modern Heuristics. 2004. doi: 10.1007/978-3-662-07807-5.
T. A. Feo, M. G. C. Resende, and S. H. Smith, “Greedy randomized adaptive search procedure for maximum independent set,” Oper. Res., 1994, doi: 10.1287/opre.42.5.860.
P. Moscato and C. Cotta Porras, “An Introduction to Memetic Algorithms,” Intel. Artif., 2003, doi: 10.4114/ia.v7i19.721.
J. Xu and J. Zhang, “Exploration-exploitation tradeoffs in metaheuristics: Survey and analysis,” in Proceedings of the 33rd Chinese Control Conference, CCC 2014, 2014. doi: 10.1109/ChiCC.2014.6896450.
B. Morales-Castañeda, D. Zaldívar, E. Cuevas, F. Fausto, and A. Rodríguez, “A better balance in metaheuristic algorithms: Does it exist?,” Swarm Evol. Comput., 2020, doi: 10.1016/j.swevo.2020.100671.
O. Samuel Sowole, “A Comparative Analysis of Search Algorithms for Solving the Vehicle Routing Problem,” in Search Algorithms - Applications for Pure Science and Industry [Working Title], 2023. doi: 10.5772/intechopen.112067.
T. Dokeroglu, E. Sevinc, T. Kucukyilmaz, and A. Cosar, “A survey on new generation metaheuristic algorithms,” Comput. Ind. Eng., 2019, doi: 10.1016/j.cie.2019.106040.
A. Hussain and Y. S. Muhammad, “Trade-off between exploration and exploitation with genetic algorithm using a novel selection operator,” Complex Intell. Syst., 2020, doi: 10.1007/s40747-019-0102-7.
E. Alba and B. Dorronsoro, “The exploration/exploitation tradeoff in dynamic cellular genetic algorithms,” IEEE Trans. Evol. Comput., 2005, doi: 10.1109/TEVC.2005.843751.
F. Vafaee, G. Turán, P. C. Nelson, and T. Y. Berger-Wolf, “Balancing the exploration and exploitation in an adaptive diversity guided genetic algorithm,” in Proceedings of the 2014 IEEE Congress on Evolutionary Computation, CEC 2014, 2014. doi: 10.1109/CEC.2014.6900257.
H. Zhang, J. Sun, T. Liu, K. Zhang, and Q. Zhang, “Balancing exploration and exploitation in multiobjective evolutionary optimization,” Inf. Sci. (Ny)., 2019, doi: 10.1016/j.ins.2019.05.046.
H. Yang, Z. Yang, Y. Yang, and L. Zhang, “An improved particle swarm optimization algorithm based on simulated annealing,” in 2014 10th International Conference on Natural Computation, ICNC 2014, 2014. doi: 10.1109/ICNC.2014.6975891.
R. K. Kincaid and A. Ninh, “Simulated Annealing,” in Springer Optimization and Its Applications, 2023. doi: 10.1007/978-3-031-38310-6_10.
K. Amine, “Multiobjective Simulated Annealing: Principles and Algorithm Variants,” 2019. doi: 10.1155/2019/8134674.
S. Chen, C. Xudiera, and J. Montgomery, “Simulated annealing with thresheld convergence,” in 2012 IEEE Congress on Evolutionary Computation, CEC 2012, 2012. doi: 10.1109/CEC.2012.6256591.
S. Singer and S. Singer, “Efficient Implementation of the Nelder–Mead Search Algorithm,” Appl. Numer. Anal. Comput. Math., 2004, doi: 10.1002/anac.200410015.
Bilal, M. Pant, H. Zaheer, L. Garcia-Hernandez, and A. Abraham, “Differential Evolution: A review of more than two decades of research,” Eng. Appl. Artif. Intell., 2020, doi: 10.1016/j.engappai.2020.103479.
M. G. Epitropakis, V. P. Plagianakos, and M. N. Vrahatis, “Balancing the exploration and exploitation capabilities of the Differential Evolution Algorithm,” in 2008 IEEE Congress on Evolutionary Computation, CEC 2008, 2008. doi: 10.1109/CEC.2008.4631159.
L. Wang, X. Zhou, T. Xie, J. Liu, and G. Zhang, “Adaptive Differential Evolution with Information Entropy-Based Mutation Strategy,” IEEE Access, 2021, doi: 10.1109/ACCESS.2021.3119616.
Â. A. R. Sá, A. O. Andrade, A. B. Soares, and S. J. Nasuto, “Exploration vs. Exploitation in differential evolution,” in AISB 2008 Convention: Communication, Interaction and Social Intelligence - Proceedings of the AISB 2008 Symposium on Swarm Intelligence Algorithms and Applications, 2008.
Y. Zhang, G. Chen, L. Cheng, Q. Wang, and Q. Li, “Methods to balance the exploration and exploitation in Differential Evolution from different scales: A survey,” Neurocomputing, 2023, doi: 10.1016/j.neucom.2023.126899.
Z. Cai, X. Yang, M. C. Zhou, Z. H. Zhan, and S. Gao, “Toward explicit control between exploration and exploitation in evolutionary algorithms: A case study of differential evolution,” Inf. Sci. (Ny)., 2023, doi: 10.1016/j.ins.2023.119656.
M. N. M. Salleh et al., “Exploration and exploitation measurement in swarm-based metaheuristic algorithms: An empirical analysis,” in Advances in Intelligent Systems and Computing, 2018. doi: 10.1007/978-3-319-72550-5_3.
A. M. Jabbar, “Controlling the Balance of Exploration and Exploitation in ACO Algorithm,” J. Univ. BABYLON Pure Appl. Sci., 2018, doi: 10.29196/jub.v26i4.678.
Y. Liu, B. Cao, and H. Li, “Improving ant colony optimization algorithm with epsilon greedy and Levy flight,” Complex Intell. Syst., 2021, doi: 10.1007/s40747-020-00138-3.
T. Kumazawa, M. Takimoto, and Y. Kambayashi, “Exploration Strategies for Model Checking with Ant Colony Optimization,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2021. doi: 10.1007/978-3-030-88081-1_20.
M. Gendreau, “An Introduction to Tabu Search,” in Handbook of Metaheuristics, 2006. doi: 10.1007/0-306-48056-5_2.
R. Chandran, S. Rakesh Kumar, and N. Gayathri, “Genetic algorithm-based tabu search for optimal energy-aware allocation of data center resources,” Soft Comput., 2020, doi: 10.1007/s00500-020-05240-9.
S. Hanafi, Y. Wang, F. Glover, W. Yang, and R. Hennig, “Tabu search exploiting local optimality in binary optimization,” Eur. J. Oper. Res., 2023, doi: 10.1016/j.ejor.2023.01.001.
A. G. Gad, “Particle Swarm Optimization Algorithm and Its Applications: A Systematic Review,” Arch. Comput. Methods Eng., 2022, doi: 10.1007/s11831-021-09694-4.
K. J. Binkley and M. Hagiwara, “Balancing exploitation and exploration in particle swarm optimization: Velocity-based reinitialization,” Trans. Japanese Soc. Artif. Intell., 2008, doi: 10.1527/tjsai.23.27.
B. Nakisa, M. N. Rastgoo, and M. J. Norodin, “Balancing exploration and exploitation in particle swarm optimization on search tasking,” Res. J. Appl. Sci. Eng. Technol., 2014, doi: 10.19026/rjaset.8.1117.
M. Ben Ghalia, “Particle swarm optimization with an improved exploration-exploitation balance,” in Midwest Symposium on Circuits and Systems, 2008. doi: 10.1109/MWSCAS.2008.4616910.
Y. Zhang and X. Kong, “A particle swarm optimization algorithm with empirical balance strategy,” Chaos, Solitons Fractals X, 2023, doi: 10.1016/j.csfx.2022.100089.
W. Liu, Z. Wang, N. Zeng, Y. Yuan, F. E. Alsaadi, and X. Liu, “A novel randomised particle swarm optimizer,” Int. J. Mach. Learn. Cybern., 2021, doi: 10.1007/s13042-020-01186-4.
T. Stützle and R. Ruiz, “Iterated local search,” in Handbook of Heuristics, 2018. doi: 10.1007/978-3-319-07124-4_8.
H. R. Lourenço, O. C. Martin, and T. Stützle, “Iterated local search: Framework and applications,” in International Series in Operations Research and Management Science, 2019. doi: 10.1007/978-3-319-91086-4_5.
H. N. K. Al-Behadili, K. R. Ku-Mahamud, and R. Sagban, “Hybrid ant colony optimization and iterated local search for rules-based classification,” J. Theor. Appl. Inf. Technol., 2020.
F. Zhao, X. He, G. Yang, W. Ma, C. Zhang, and H. Song, “A hybrid iterated local search algorithm with adaptive perturbation mechanism by success-history based parameter adaptation for differential evolution (SHADE),” Eng. Optim., 2020, doi: 10.1080/0305215X.2019.1595611.
D. Karaboga and B. Basturk, “A powerful and efficient algorithm for numerical function optimization: Artificial bee colony (ABC) algorithm,” J. Glob. Optim., 2007, doi: 10.1007/s10898-007-9149-x.
E. Kaya, B. Gorkemli, B. Akay, and D. Karaboga, “A review on the studies employing artificial bee colony algorithm to solve combinatorial optimization problems,” 2022. doi: 10.1016/j.engappai.2022.105311.
W. F. Gao and S. Y. Liu, “A modified artificial bee colony algorithm,” Comput. Oper. Res., 2012, doi: 10.1016/j.cor.2011.06.007.
G. Zhu and S. Kwong, “Gbest-guided artificial bee colony algorithm for numerical function optimization,” Appl. Math. Comput., 2010, doi: 10.1016/j.amc.2010.08.049.
P. W. Tsai, J. S. Pan, B. Y. Liao, and S. C. Chu, “Enhanced artificial bee colony optimization,” in International Journal of Innovative Computing, Information and Control, 2009.
W. sheng Xiao, G. xin Li, C. Liu, and L. ping Tan, “A novel chaotic and neighborhood search-based artificial bee colony algorithm for solving optimization problems,” Sci. Rep., 2023, doi: 10.1038/s41598-023-44770-8.
K. M. Passino, “Biomimicry of Bacterial Foraging for Distributed Optimization and Control,” IEEE Control Syst., 2002, doi: 10.1109/MCS.2002.1004010.
C. Guo, H. Tang, B. Niu, and C. Boon Patrick Lee, “A survey of bacterial foraging optimization,” Neurocomputing, 2021, doi: 10.1016/j.neucom.2020.06.142.
M. Chen, Y. Ou, X. Qiu, and H. Wang, “An Effective Bacterial Foraging Optimization Based on Conjugation and Novel Step-Size Strategies,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020. doi: 10.1007/978-3-030-57884-8_32.
H. Chen, Y. Zhu, and K. Hu, “Self-adaptation in bacterial foraging optimization algorithm,” in Proceedings of 2008 3rd International Conference on Intelligent System and Knowledge Engineering, ISKE 2008, 2008. doi: 10.1109/ISKE.2008.4731080.
B. Pang, Y. Song, C. Zhang, H. Wang, and R. Yang, “Bacterial foraging optimization based on improved chemotaxis process and novel swarming strategy,” Appl. Intell., 2019, doi: 10.1007/s10489-018-1317-9.
X. S. Yang, “A new metaheuristic Bat-inspired Algorithm,” in Studies in Computational Intelligence, 2010. doi: 10.1007/978-3-642-12538-6_6.
T. Agarwal and V. Kumar, “A Systematic Review on Bat Algorithm: Theoretical Foundation, Variants, and Applications,” 2022. doi: 10.1007/s11831-021-09673-9.
A. H. Gandomi and X. S. Yang, “Chaotic bat algorithm,” J. Comput. Sci., 2014, doi: 10.1016/j.jocs.2013.10.002.
X. S. Yang and S. Deb, “Cuckoo search via Lévy flights,” in 2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings, 2009. doi: 10.1109/NABIC.2009.5393690.
A. Sharma, A. Sharma, V. Chowdary, A. Srivastava, and P. Joshi, “Cuckoo search algorithm: A review of recent variants and engineering applications,” in Studies in Computational Intelligence, 2021. doi: 10.1007/978-981-15-7571-6_8.
H. Malik, A. Iqbal, P. Joshi, S. Agrawal, and I. B. Farhad, Metaheuristic and Evolutionary Computation : Algorithms and Applications. 2021.
S. K. Nayak, B. R. Senapati, and D. Mishra, “Balancing exploration and exploitation: Unleashing the adaptive power of automatic cuckoo search for meta-heuristic optimization,” Intell. Decis. Technol., 2024, doi: 10.3233/IDT-idt230275.
R. Salgotra, U. Singh, and S. Saha, “New cuckoo search algorithms with enhanced exploration and exploitation properties,” Expert Syst. Appl., 2018, doi: 10.1016/j.eswa.2017.11.044.
G. Kanagaraj, S. G. Ponnambalam, and N. Jawahar, “A hybrid cuckoo search and genetic algorithm for reliability-redundancy allocation problems,” Comput. Ind. Eng., 2013, doi: 10.1016/j.cie.2013.08.003.
K. Hussain, M. N. Mohd Salleh, Y. A. Prasetyo, and S. Cheng, “Personal best Cuckoo search algorithm for global optimization,” Int. J. Adv. Sci. Eng. Inf. Technol., 2018, doi: 10.18517/ijaseit.8.4.5009.
Z. Huang, Z. Gao, L. Qi, and H. Duan, “A Heterogeneous Evolving Cuckoo Search Algorithm for Solving Large-Scale Combined Heat and Power Economic Dispatch Problems,” IEEE Access, 2019, doi: 10.1109/ACCESS.2019.2933980.
X. S. Yang, “Firefly algorithm, stochastic test functions and design optimization,” Int. J. Bio-Inspired Comput., 2010, doi: 10.1504/IJBIC.2010.032124.
X. S. Yang and X. He, “Firefly algorithm: recent advances and applications,” Int. J. Swarm Intell., 2013, doi: 10.1504/ijsi.2013.055801.
A. H. Gandomi, X. S. Yang, S. Talatahari, and A. H. Alavi, “Firefly algorithm with chaos,” Commun. Nonlinear Sci. Numer. Simul., 2013, doi: 10.1016/j.cnsns.2012.06.009.
I. Brajević and P. Stanimirović, “An improved chaotic firefly algorithm for global numerical optimization,” Int. J. Comput. Intell. Syst., 2018, doi: 10.2991/ijcis.2018.25905187.
I. Brajević and J. Ignjatović, “An upgraded firefly algorithm with feasibility-based rules for constrained engineering optimization problems,” J. Intell. Manuf., 2019, doi: 10.1007/s10845-018-1419-6.
M. Sababha, M. Zohdy, and M. Kafafy, “The enhanced firefly algorithm based on modified exploitation and exploration mechanism,” Electron., 2018, doi: 10.3390/electronics7080132.
J. A. Villaruz, B. D. Gerardo, A. O. Gamao, and R. P. Medina, “Scouting Firefly Algorithm and its Performance on Global Optimization Problems,” Int. J. Adv. Comput. Sci. Appl., 2023, doi: 10.14569/IJACSA.2023.0140350.
G. Li, P. Liu, C. Le, and B. Zhou, “A novel hybrid meta-heuristic algorithm based on the cross-entropy method and firefly algorithm for global optimization,” Entropy, 2019, doi: 10.3390/e21050494.
S. Mirjalili, S. M. Mirjalili, and A. Lewis, “Grey Wolf Optimizer,” Adv. Eng. Softw., 2014, doi: 10.1016/j.advengsoft.2013.12.007.
Y. Liu, A. As’arry, M. K. Hassan, A. A. Hairuddin, and H. Mohamad, “Review of the grey wolf optimization algorithm: variants and applications,” Neural Comput. Appl., 2024, doi: 10.1007/s00521-023-09202-8.
N. Mittal, U. Singh, and B. S. Sohi, “Modified Grey Wolf Optimizer for Global Engineering Optimization,” Appl. Comput. Intell. Soft Comput., 2016, doi: 10.1155/2016/7950348.
M. Kohli and S. Arora, “Chaotic grey wolf optimization algorithm for constrained optimization problems,” J. Comput. Des. Eng., 2018, doi: 10.1016/j.jcde.2017.02.005.
V. K. R. Aala Kalananda and V. L. N. Komanapalli, “Hybrid evolutionary grey wolf optimizer for constrained engineering problems and multi-unit production planning,” Evol. Intell., 2024, doi: 10.1007/s12065-024-00909-8.
S. Joshi and J. C. Bansal, “Grey Wolf gravitational search algorithm,” in IWCI 2016 - 2016 International Workshop on Computational Intelligence, 2017. doi: 10.1109/IWCI.2016.7860371.
N. Singh and H. Hachimi, “A New Hybrid Whale Optimizer Algorithm with Mean Strategy of Grey Wolf Optimizer for Global Optimization,” Math. Comput. Appl., 2018, doi: 10.3390/mca23010014.
X. Yu, W. Y. Xu, X. Wu, and X. Wang, “Reinforced exploitation and exploration grey wolf optimizer for numerical and real-world optimization problems,” Appl. Intell., 2022, doi: 10.1007/s10489-021-02795-4.
G. Shial, S. Sahoo, and S. Panigrahi, “An Enhanced GWO Algorithm with Improved Explorative Search Capability for Global Optimization and Data Clustering,” Appl. Artif. Intell., 2023, doi: 10.1080/08839514.2023.2166232.
W. Long, J. Jiao, X. Liang, S. Cai, and M. Xu, “A Random Opposition-Based Learning Grey Wolf Optimizer,” IEEE Access, 2019, doi: 10.1109/ACCESS.2019.2934994.
Z. W. Geem, J. H. Kim, and G. V. Loganathan, “A New Heuristic Optimization Algorithm: Harmony Search,” Simulation, 2001, doi: 10.1177/003754970107600201.
A. Sadollah, H. Sayyaadi, D. G. Yoo, H. M. Lee, and J. H. Kim, “Mine blast harmony search: A new hybrid optimization method for improving exploration and exploitation capabilities,” Appl. Soft Comput. J., 2018, doi: 10.1016/j.asoc.2018.04.010.
A. A. Alomoush, A. R. A. Alsewari, K. Z. Zamli, A. Alrosan, W. Alomoush, and K. Alissa, “Enhancing three variants of harmony search algorithm for continuous optimization problems,” Int. J. Electr. Comput. Eng., 2021, doi: 10.11591/ijece.v11i3.pp2343-2349.
J. Wang, H. Ouyang, Z. Zhou, and S. Li, “Harmony Search Algorithm Based on Dual-Memory Dynamic Search and Its Application on Data Clustering,” Complex Syst. Model. Simul., 2023, doi: 10.23919/CSMS.2023.0019.
W. Wu, H. Ouyang, A. W. Mohamed, C. Zhang, and S. Li, “Enhanced harmony search algorithm with circular region perturbation for global optimization problems,” Appl. Intell., 2020, doi: 10.1007/s10489-019-01558-6.
S. Mirjalili and A. Lewis, “The Whale Optimization Algorithm,” Adv. Eng. Softw., 2016, doi: 10.1016/j.advengsoft.2016.01.008.
X. Wu, S. Zhang, W. Xiao, and Y. Yin, “The Exploration/Exploitation Tradeoff in Whale Optimization Algorithm,” IEEE Access, 2019, doi: 10.1109/ACCESS.2019.2938857.
N. Rana, M. S. A. Latiff, S. M. Abdulhamid, and H. Chiroma, “Whale optimization algorithm: a systematic review of contemporary applications, modifications and developments,” 2020. doi: 10.1007/s00521-020-04849-z.
J. Du, J. Hou, H. Wang, and Z. Chen, “Application of an improved whale optimization algorithm in time-optimal trajectory planning for manipulators,” Math. Biosci. Eng., 2023, doi: 10.3934/mbe.2023728.
R. Xu, C. Zhao, J. Li, J. Hu, and X. Hou, “A Hybrid Improved-Whale-Optimization–Simulated-Annealing Algorithm for Trajectory Planning of Quadruped Robots,” Electron., 2023, doi: 10.3390/electronics12071564.
T. Tian, Z. Liang, Y. Wei, Q. Luo, and Y. Zhou, “Hybrid Whale Optimization with a Firefly Algorithm for Function Optimization and Mobile Robot Path Planning,” Biomimetics, 2024, doi: 10.3390/biomimetics9010039.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Bayadir Abbas Al-Himyari, Hiba Al-khafaji , Noor Fadel Hussain
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
- Papers must be submitted on the understanding that they have not been published elsewhere (except in the form of an abstract or as part of a published lecture, review, or thesis) and are not currently under consideration by another journal published by any other publisher.
- It is also the authors responsibility to ensure that the articles emanating from a particular source are submitted with the necessary approval.
- The authors warrant that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required.
- The authors ensure that all the references carefully and they are accurate in the text as well as in the list of references (and vice versa).
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Attribution-NonCommercial 4.0 International that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
- The journal/publisher is not responsible for subsequent uses of the work. It is the author's responsibility to bring an infringement action if so desired by the author.
