Extension neural network is a pattern recognition method found by M. H. Wang and C. P. Hung in 2003 to classify instances of data sets. Extension neural network is composed ofartificial neural network and extension theory concepts. It uses the fast and adaptive learning capability of neural network and correlation estimation property of extension theory by calculating extension distance.
Extension theory was first proposed by Cai in 1983 to solve contradictory problems. While classical mathematic is familiar with quantity and forms of objects, extension theory transforms these objects to matter-element model.
where in matter, is the name or type, is its characteristics and is the corresponding value for the characteristic. There is a corresponding example in equation 2.
where and characteristics form extension sets. These extension sets are defined by the values which are range values for corresponding characteristics. Extension theory concerns with the extension correlation function between matter-element models like shown in equation 2 and extension sets. Extension correlation function is used to define extension space which is composed of pairs of elements and their extension correlation functions. The extension space formula is shown in equation 3.
where, is the extension space, is the object space, is the extension correlation function, is an element from the object space and is the corresponding extension correlation function output of element. maps to a membership interval. Negative region represents an element not belonging membership degree to a class and positive region vice versa. If is mapped to, extension theory acts like fuzzy set theory. The correlation function can be shown with the equation 4.
where, and are called concerned and neighborhood domain and their intervals are and respectively. The extended correlation function used for estimation of membership degree between and, is shown in equation 5.
Extension Neural Network
Extension neural network has a neural network like appearance. Weight vector resides between the input nodes and output nodes. Output nodes are the representation of input nodes by passing them through the weight vector. There are total number of input and output nodes are represented by and, respectively. These numbers depend on the number of characteristics and classes. Rather than using one weight value between two layer nodes as in neural network, extension neural network architecture has two weight values. In extension neural network architecture, for instance, is the input which belongs to class and is the corresponding output for class. The output is calculated by using extension distance as shown in equation 6. Estimated class is found through searching for the minimum extension distance among the calculated extension distance for all classes as summarized in equation 7, where is the estimated class.
Learning Algorithm
Each class is composed of ranges of characteristics. These characteristics are the input types or names which come from matter-element model. Weight values in extension neural network represent these ranges. In the learning algorithm, first weights are initialized by searching for the maximum and minimum values of inputs for each class as shown in equation 8 where, is the instance number and is represents number of input. This initialization provides classes' ranges according to given training data. After maintaining weights, center of clusters are found through the equation 9. Before learning process begins, predefined learning performance rate is given as shown in equation 10 where, is the misclassified instances and is the total number of instances. Initialized parameters are used to classify instances with using equation 6. If the initialization is not sufficient due to the learning performance rate, training is required. In the training step weights are adjusted to classify training data more accurately, therefore reducing learning performance rate is aimed. In each iteration, is checked to control if required learning performance is reached. In each iteration every training instance is used for training.
Instance, belongs to class is shown by: Every input data point of is used in extension distance calculation to estimate the class of. If the estimated class then update is not needed. Whereas, if then update is done. In update case, separators which show the relationship between inputs and classes, are shifted proportional to the distance between the center of clusters and the data points.
The update formula:
To classify the instance accurately, separator of class for input moves close to data-point of instance, whereas separator of class for input moves far away. In the above image, an update example is given. Assume that instance belongs to class A, whereas it is classified to class B because extension distance calculation gives out. After the update, separator of class A moves close to the data-point of instance whereas separator of class B moves far away. Consequently, extension distance gives out, therefore after update instance is classified to class A.
