جامعة المعرفة
المصدر المعتمد للمعلومات البحثية والمخرجات هو قاعدة بيانات سكوبس (Elsevier Scopus) مع تدقيق بيانات الباحثين من عمادة البحث العلمي.
مقالوصول مفتوح20234 استشهاد

Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning

International Journal of Advanced Computer Science and Applications · مج 14 (2) · 2023

الملخّص

A convolutional neural network (CNN) is a subset of machine learning as well as one of the different types of artificial neural networks that are used for different applications and data types. Activation functions (AFs) are used in this type of network to determine whether or not its neurons are activated. One non-linear AF named as Rectified Linear Units (ReLU) which involves a simple mathematical operations and it gives better performance. It avoids rectifying vanishing gradient problem that inherents older AFs like tanh and sigmoid. Additionally, it has less computational cost. Despite these advantages, it suffers from a problem called Dying problem. Several modifications have been appeared to address this problem, for example; Leaky ReLU (LReLU). The main concept of our algorithm is to improve the current LReLU activation functions in mitigating the dying problem on deep learning by using the readjustment of values (changing and decreasing value) of the loss function or cost function while number of epochs are increased. The model was trained on the MNIST dataset with 20 epochs and achieved lowest misclassification rate by 1.2%. While optimizing our proposed methods, we received comparatively better results in terms of simplicity, low computational cost, and with no hyperparameters.

التفاصيل

المجلة
International Journal of Advanced Computer Science and Applications
الناشر
Science and Information Organization
المجلد
14
العدد
2
الصفحات
—
الترقيم الدولي ISSN
2156-5570
السنة
2023
اللغة
en
الترخيص
cc-by
الوصول المفتوح
diamond
الاستشهادات
4
المصدر
scopus

الموضوعات

MNIST databaseComputer scienceSigmoid functionActivation functionConvolutional neural networkHyperparameter

Neural Networks and Applications (Computer Science) · Advanced Neural Network Applications (Computer Science) · Stochastic Gradient Optimization Techniques (Computer Science)

استشهد بهذا

APA
Atoum, I. A. (2023). Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning. International Journal of Advanced Computer Science and Applications, 14(2). https://doi.org/10.14569/ijacsa.2023.0140212
Harvard
Atoum, I. A. 2023, 'Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning', International Journal of Advanced Computer Science and Applications, vol. 14(2). Available at: https://doi.org/10.14569/ijacsa.2023.0140212
Vancouver
Atoum IA. Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning. International Journal of Advanced Computer Science and Applications. 2023;14(2).
BibTeX
@article{atoum20232089,
  title = {Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning},
  author = {Ibrahim A. Atoum},
  year = {2023},
  journal = {International Journal of Advanced Computer Science and Applications},
  volume = {14},
  number = {2},
  publisher = {Science and Information Organization},
  doi = {10.14569/ijacsa.2023.0140212},
  url = {http://dx.doi.org/10.14569/ijacsa.2023.0140212}
}
RIS
TY  - JOUR
AU  - Ibrahim A. Atoum
TI  - Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning
PY  - 2023
JO  - International Journal of Advanced Computer Science and Applications
VL  - 14
IS  - 2
DO  - 10.14569/ijacsa.2023.0140212
UR  - http://dx.doi.org/10.14569/ijacsa.2023.0140212
ER  - 

مخرجات ذات صلة