An Efficient Training Algorithm of Restricted Boltzmann Machines
Abstract
The paper deals with an actual applied problem related to the artificial neural networks training. An approach to the solution based on the idea of random search is proposed. An original training algorithm that implements Boltzmann annealing has been developed and its convergence in probability to the global optimum has been proved. It is also shown that the proposed algorithm can be easily modified to train any artificial neural network. Thus, it has a good prospect for solving applied problems using neural network technologies in general. Experimental studies have been carried out, in which, using the example of compressing color raster images problem, the proposed algorithm was compared with the known adaptive moment algorithm - one of the best gradient methods for training neural networks. Image compression was performed using an ensemble of n Gauss-Bernoulli restricted Boltzmann machines. The use of an ensemble of n machines in combination with a specially developed parallelization procedure made it possible to reduce the computational complexity of the training process and increase the speed of the proposed algorithm. As a result of experiments, it was shown that the proposed approach is not inferior to gradient methods in terms of speed. Moreover, the developed training algorithm turned out to be more than twice as effective as the adaptive moment algorithm in terms of the quality of the solution obtained.
Copyright information
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
About this article
Publication Date
27 February 2023
Article Doi
eBook ISBN
978-1-80296-960-3
Publisher
European Publisher
Volume
1
Print ISBN (optional)
-
Edition Number
1st Edition
Pages
1-403
Subjects
Hybrid methods, modeling and optimization, complex systems, mathematical models, data mining, computational intelligence
Cite this article as:
Matskevich, V. V., & Stasiuk, V. A. (2023). An Efficient Training Algorithm of Restricted Boltzmann Machines. In P. Stanimorovic, A. A. Stupina, E. Semenkin, & I. V. Kovalev (Eds.), Hybrid Methods of Modeling and Optimization in Complex Systems, vol 1. European Proceedings of Computers and Technology (pp. 296-303). European Publisher. https://doi.org/10.15405/epct.23021.36