مقاله انگلیسی رایگان در مورد ارائه مدولار شبکه های عصبی لایه بندی شده – الزویر 2018

Layered neural networks have recently been applied to various tasks (Bengio, Courville, & Vincent, 2013; LeCun, Bengio, & Hinton, 2015), including image processing (Krizhevsky, Sutskever, & Hinton, 2012; Tompson et al., 2014), speech recognition (Hinton et al., 2012; Sainath et al., 2013), natural language processing (Collobert, 2011; Sutskever, Vinyals, & Le, 2014), and bioinformatics (Leung et al., 2014; Xiong et al., 2015). Although they have simple layered structures of units and connections, they outperform other conventional models by their ability to learn complex nonlinear relationships between input and output data. In each layer, inputs are transformed into more abstract representations under a given set of the model parameters. These parameters are automatically optimized through training so that they extract the important features of the input data. In other words, it does not require either careful feature engineering by hand, or expert knowledge of the data. This advantage has made layered neural networks successful in a wide range of tasks, as mentioned above. However, the inference provided by a layered neural network consists of a large number of nonlinear and complex parameters, which makes it difficult for human beings to understand it. More complex relationships between input and output can be represented as the network becomes deeper or the number of units in each hidden layer increases, however interpretation becomes more difficult. The large number of parameters also causes problems in terms of computational time, memory and over-fitting, so it is important to reduce the parameters appropriately. Since it is difficult to read the underlying structure of a neural network and to identify the parameters that are important to keep, we must perform experimental trials to find the appropriate values of the hyperparameters and the random initial parameters that achieve the best trained result. In this paper, to overcome such difficulties, we propose a new method for extracting a global and simplified structure from a layered neural network (For example, Figs. 5 and 11). Based on network analysis, the proposed method defines a modular representation of the original trained neural network by detecting communities or clusters of units with similar connection patterns. Although the modular neural network proposed by Azam (2000) and Jacobs et al. (1991) has a similar name, it takes the opposite approach to ours. In fact, it constructs the model structure before training with multiple split neural networks inside it. Then, each small neural network works as an expert of a subset task. Our proposed method is based on the community detection algorithm. To date, various methods have been proposed to express the characteristics of diverse complex networks without layered structures (Estrada & Velázquez, 2005; Meunier & Paugam-Moisy, 2006; Newman, 2006; Newman & Girvan, 2004; Newman & Leicht, 2007), however, no method has been developed for detecting the community structures of trained layered neural networks.