Published:
2022-06-24Issue:
Vol. 19 No. 1 (2022): Tekhnê JournalSection:
ArticlesHow to choose an activation function for deep learning
Cómo elegir una función de activación para el aprendizaje profundo
Keywords:
Activation function, deep learning, neural network, nonlinearity (en).Keywords:
Aprendizaje profundo, función de activación, no linealidad, red neuronal (es).Downloads
Abstract (en)
Activation functions are important in each layer of the neural network because they allow the network to learn complex relationships between the input data and the output data. They also introduce nonlinearity into the network, which is essential for learning patterns in data. Activation functions play a critical role in the training and optimization of deep learning models, and choosing the right activation function can significantly impact the model’s performance. This article presents a summary of the features of these functions.
Abstract (es)
Las funciones de activación son importantes en cada capa de la red neuronal porque permiten a la red aprender relaciones complejas entre los datos de entrada y los de salida. También introducen la no linealidad en la red, que es esencial para aprender patrones en los datos. Las funciones de activación desempeñan un papel fundamental en el entrenamiento y la optimización de los modelos de aprendizaje profundo, y la elección de la función de activación adecuada puede influir significativamente en el rendimiento del modelo. Este artículo presenta un resumen de las características de estas funciones.
References
Alonso, A., Peña, A., & Martínez, F. (2021). Autonomous identification of high-contact surfaces from convolutional neural networks. Journal of Physics: Conference Series, 2135(1), 012001.
Brownlee, J. (2021). Machine learning Mastery. https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/
Chang, Y.-W., & Tsai, C.-Y. (2017). Apply deep learning neural network to forecast number of tourists. 2017 31st International Conference on Advanced Information Networking and Applications Workshops (WAINA). https://doi.org/10.1109/waina.2017.125
Deng, L., Hinton, G., & Kingsbury, B. (2013). New types of deep neural network learning for speech recognition and related applications: An overview. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. https://doi.org/10.1109/icassp.2013.6639344
Ibrahim, M. F. I., & Al-Jumaily, A. A. (2018). Auto-encoder based deep learning for surface electromyography signal processing. Advances in Science, Technology and Engineering Systems Journal, 3(1), 94–102. https://doi.org/10.25046/aj030111
Jacinto, E., Martínez, F., & Martínez, F. (2022). Performance evaluation of temporal and frequential analysis approaches of electromyographic signals for gestures recognition using neural networks. International Journal of Advanced Computer Science and Applications, 13(3), 1–8.
Kim, J.-Y., & Cho, S.-B. (2019). Evolutionary optimization of hyperparameters in deep learning models. 2019 IEEE Congress on Evolutionary Computation (CEC). https://doi.org/10.1109/cec.2019.8790354
Maguolo, G., Nanni, L., & Ghidoni, S. (2019). Ensemble of convolutional neural networks trained with different activation functions. arXiv, 1–13.
Martínez, F., Hernández, C., & Rendón, A. (2017). A study on machine learning models for convergence time predictions in reactive navigation strategies. Contemporary Engineering Sciences, 10(25), 1223–1232.
Martínez, F., Martínez, F., & Jacinto, E. (2020). Performance evaluation of the nasnet convolutional network in the automatic identification of covid-19. International Journal on Advanced Science, Engineering and Information Technology, 10(2), 662.
Martínez, F., Martínez, F., & Montiel, H. (2020). Low cost, high performance fuel cell energy conditioning system controlled by neural network. TELKOMNIKA (Telecommunication Computing Electronics and Control), 18(6), 3116–3122.
Martínez, F., Montiel, H., & Martínez, F. (2022). A machine learning model for the diagnosis of coffee diseases. International Journal of Advanced Computer Science and Applications, 13(4), 1–8.
Martínez, F., Penagos, C., & Pacheco, L. (2020). Scheme for motion estimation based on adaptive fuzzy neural network. TELKOMNIKA (Telecommunication Computing Electronics and Control), 18(2), 1030–1037.
Montiel, H., Jacinto, E., & Martínez, F. (2021). A double-loop hybrid approach for the recognition of fissures in bone structures. ARPN Journal of Engineering and Applied Sciences, 16(11), 1151–1156.
Montiel, H., Martínez, F., & Jacinto, E. (2017). Visual patterns recognition in robotic platforms through the use of neural networks and image processing. International Journal of Applied Engineering Research, 12(18), 7770–7774.
Pomerat, J., Segev, A., & Datta, R. (2019). On neural network activation functions and optimizers in relation to polynomial regression. 2019 IEEE International Conference on Big Data (Big Data). https://doi.org/10.1109/bigdata47090.2019.9005674
Ramachandran, P., Zoph, B., & Le, Q. V. (2017). Searching for activation functions. arXiv, 1–13.
Rendón, A., & Martínez, F. (2021). Intelligent sensor for thermal process control using convolutional neural network. Journal of Physics: Conference Series, 1993(1), 012027.
Rendón, A., Martínez, F., & Hernández, C. (2017). Deep regression model for predictive control in a vegetable waste carbonization plant. Contemporary Engineering Sciences, 10(21), 1047–1055.
Szandala, T. (2021). Bio-inspired neurocomputing (A. K. Bhoi, P. K. Mallick, C.-M. Liu, & V. E. Balas, Eds.; Vol. 903). Springer Singapore. https://doi.org/10.1007/978-981-15-5495-7
Zhu, D., Wu, X., & Yang, T. (2022). Benchmarking deep auroc optimization: Loss functions and algorithmic choices. arXiv, 1–32.
Zhu, H., Yang, X., & Wang, Y. (2018). Prediction of daily entrance and exit passenger flow of rail transit stations by deep learning method. Journal of Advanced Transportation, 2018(1), 1–11. https://doi.org/10.1155/2018/6142724
Zhu, X., Li, J., Zhu, M., Jiang, Z., & Li, Y. (2018). An evaporation duct height prediction method based on deep learning. IEEE Geoscience and Remote Sensing Letters, 15(9), 1307–1311. https://doi.org/10.1109/lgrs.2018.2842235
How to Cite
APA
ACM
ACS
ABNT
Chicago
Harvard
IEEE
MLA
Turabian
Vancouver
Download Citation
Visitas
Downloads
License
Copyright (c) 2023 Albert I. Rodríguez P., Xiomara D. Buitrago R.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Those authors who have published with this journal agree to the following terms:
- The authors retain their copyright and guarantee to the journal right of first publication of his work, which will be simultaneously subject to license recognition of Creative Commons that allows others to share the work as long as its author is indicated and its first publication in this journal.
- Authors may adopt other agreements nonexclusive distribution version of the work as long as indicated that the initial publication is in this journal.
- It allows and encourages authors disseminate their work via the Internet before and during the submission process, which can produce interesting exchanges and increase citations of the published work.