Why Tuning the Control Parameters of Metaheuristic Algorithms Is So Important for Fair Comparison?
Abstract
Although metaheuristic optimization has become a common practice, new bio-inspired algorithms often suffer from a priori ill reputation. One of the reasons is a common bad practice in metaheuristic proposals. It is essential to pay attention to the quality of conducted experiments, especially when comparing several algorithms among themselves. The comparisons should be fair and unbiased. This paper points to the importance of proper initial parameter configurations of the compared algorithms. We highlight the performance differences with several popular and recommended parameter configurations. Even though the parameter selection was mostly based on comprehensive tuning experiments, the algorithms' performance was surprisingly inconsistent, given various parameter settings. Based on the presented evidence, we conclude that paying attention to the metaheuristic algorithm's parameter tuning should be an integral part of the development and testing processes.
References
Awad, N., Ali, M., Suganthan, P., Liang, J., and Qu, B. Problem definitions and evaluation criteria for the cec 2017 special session and competition on single objective real-parameter numerical optimization. Technical Report, Nanyang Technological University, Singapore 34, 2016.
Bergh, F., and Engelbrecht, A. A study of particle swarm optimization particle trajectories. Information Sciences 176 (2006), 937-971.
Eberhart, R., and Shi, Y. Particle swarm optimization: developments, applications and resources. In Proceedings of the 2001 Congress on Evolutionary Computation (2001), pp. 81-86.
Faris, H., Aljarah, I., Mirjalili, S., Castillo, P., and Merelo, J. Evolopy: An open-source nature-inspired optimization framework in python. In Proceedings of the 8th International Joint Conference on Computational Intelligence (2016), pp. 171-177.
Hansen, N., Auger, A., Finck, S., and Ros, R., 2010. Real-Parameter Black-Box Optimization Benchmarking 2010: Experimental Setup. INRIA.
Harrison, K., Ombuki-Berman, B., and Engelbrecht, A. Optimal parameter regions for particle swarm optimization algorithms. In 2017 IEEE Congress on Evolutionary Computation (CEC) (2017), pp. 349-356.
Huang, C., Li, Y., and Yao, X. A survey of automatic parameter tuning methods for metaheuristics. IEEE Transactions on Evolutionary Computation 24 (2020), 201-216.
Kennedy, J., and Eberhart, R. Particle swarm optimization. In Proceedings of ICNN'95 - International Conference on Neural Networks (1995), pp. 1942-1948.
LaTorre, A., Molina, D., Osaba, E., Ser, J. D., and Herrera, F. Fairness in bio-inspired optimization research: A prescription of methodological guidelines for comparing meta-heuristics. arXiv (2020), arXiv:2004.09969.
Maca, P., and Pech, P. The inertia weight updating strategies in particle swarm optimisation based on the beta distribution. Mathematical Problems in Engineering (2015), 1-9.
Matousek, R. GAHC: Improved genetic algorithm. Studies in Computational Intelligence 129 (2008), 507-520.
Matousek, R., Popela, P., and Kudela, J. Heuristic approaches to stochastic quadratic assignment problem: Var and cvar cases. MENDEL 23, 1 (Jun. 2017), 73-78.
Mo, Y., Ma, Y., and Zheng, Q. Optimal choice of parameters for firefly algorithm. In 2013 Fourth International Conference on Digital Manufacturing Automation (2013), pp. 887-892.
Savsani, P., and Savsani, V. Passing vehicle search (pvs): A novel metaheuristic algorithm. Applied Mathematical Modelling 40 (2016), 3951-3978.
Sorensen, K. Metaheuristics - the metaphor exposed. International Transactions in Operational Research 22 (2015), 3-18.
Wolpert, D., and Macready, W. No free lunch theorems for optimization. IEEE transactions on evolutionary computation 1 (1997), 67-82.
Xue, F., Cai, Y., Cao, Y., Cui, Z., and Li, F. Optimal parameter settings for bat algorithm. International Journal of Bio-Inspired Computation 7 (2015), 125-128.
Yang, X.-S., Ed. Nature-Inspired Metaheuristic Algorithms. Luniver Press, 2008.
Yang, X.-S. A new metaheuristic bat-inspired algorithm. arXiv (2010a), arXiv:1004.4170.
Yang, X.-S. Firefly algorithm, levy flights and global optimization. In Research and Development in Intelligent Systems XXVI (2010b), pp. 209-218.
Yang, X.-S., and Deb, S. Cuckoo search via levy flights. In 2009 World Congress on Nature Biologically Inspired Computing (NaBIC) (2009), pp. 210-214.
Yang, X.-S., and Deb, S. Engineering optimisation by cuckoo search. arXiv (2010), arXiv:1005.2908.
Copyright (c) 2020 MENDEL
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
MENDEL open access articles are normally published under a Creative Commons Attribution-NonCommercial-ShareAlike (CC BY-NC-SA 4.0) https://creativecommons.org/licenses/by-nc-sa/4.0/ . Under the CC BY-NC-SA 4.0 license permitted 3rd party reuse is only applicable for non-commercial purposes. Articles posted under the CC BY-NC-SA 4.0 license allow users to share, copy, and redistribute the material in any medium of format, and adapt, remix, transform, and build upon the material for any purpose. Reusing under the CC BY-NC-SA 4.0 license requires that appropriate attribution to the source of the material must be included along with a link to the license, with any changes made to the original material indicated.
