Artificial Neural Networks

Information about Artificial Neural Networks

Published on May 3, 2010

Author: aSGuest44017



ADNANCE CONTROL TECHNOLOGY AND APPLICATIONSNEURAL NETWORKS FOR CONTROLGROUP 3 PRESENTATION : ADNANCE CONTROL TECHNOLOGY AND APPLICATIONSNEURAL NETWORKS FOR CONTROLGROUP 3 PRESENTATION PRESENTED BY: - SAMUEL SUBHASH - AJAY KUMAR - RANJITH KUMAR HISTORICAL SYNOPSIS : HISTORICAL SYNOPSIS Neural networks came into picture before the advent of computers but it appears to be a recent development. As this field required more financial and professional support, minimal researches have been made. Minsky and Papert were the two pioneers who developed a convincing thesis which overcame the limitations proposed by others. Warren McCulloch and Walter Pits were the two neurophysiologists who developed the first artificial neuron in 1943. Artificial Neural Networks (ANN) : Artificial Neural Networks (ANN) Neural network inspired by biological nervous systems, such as our brain. The key element of this idea is the novel structure of the information processing system. Essentially a function approximation Transforms inputs into outputs to the best of its ability What Are Artificial Neural Networks? : Composed of many “neurons” that co-operate to perform the desired function. What Are Artificial Neural Networks? Types Neural NetworkArchitectures : Types Neural NetworkArchitectures Many kinds of structures, main distinction made between two classes: a) Feed- forward (a directed acyclic graph (DAG): links are unidirectional, no cycles - There is no internal state other than the weights. b) Recurrent: links form arbitrary topologies e.g., Hopfield Networks and Boltzmann machines. Recurrent networks: can be unstable, or oscillate, or exhibit chaotic behaviour e.g., given some input values, can take a long time to compute stable output and learning is made more difficult. PRACTICAL IMPLEMENTATION : PRACTICAL IMPLEMENTATION The usage of neural networks are as follows: Classification - Pattern recognition, feature extraction, image matching. Noise Reduction - Recognize patterns in the inputs and produce, noiseless outputs Prediction - Extrapolation based on historical data THEORITICAL BASIS : THEORITICAL BASIS WORKING OF NEURAL NETWORK: - The output of a neuron is a function of the weighted sum of the inputs plus a bias. The function of the entire neural network is simply the computation of the outputs of all the neurons ACTIVATION FUNCTIONS : ACTIVATION FUNCTIONS The activation functions act as control functions such that the output of a neuron in a neural network is between certain values (usually 0 and 1, or -1 and 1). There are three types of activation functions : 1) Threshold function (V) 2) Piecewise-Linear function 3) Sigmoid function Slide 9: Applied to the weighted sum of the inputs of a neuron to produce the output. Majority of NN’s use sigmoid functions - Smooth, continuous, and monotonically increasing (derivative is always positive) - Bounded range - but never reaches max or min Consider “ON” to be slightly less than the max and “OFF” to be slightly greater than the min Slide 10: The most common sigmoid function used is the logistic function - f(x) = 1/(1 + e-x) - The calculation of derivatives are important for neural networks and the logistic function has a very nice derivative - f’(x) = f(x)(1 - f(x)) Other sigmoid functions also used -hyperbolic tangent -arctangent The exact nature of the function has little effect on the abilities of the neural network. Where do the weights come from? : Where do the weights come from? The weights in a neural network are the most important factor in determining its function. Training is the act of presenting the network with some sample data and modifying the weights to better approximate the desired function. There are two main types of training Supervised Training ■ Supplies the neural network with inputs and the desired outputs. ■ Response of the network to the inputs is measured The weights are modified to reduce the difference between the actual and desired outputs. Unsupervised Training ■ Only supplies inputs ■ The neural network adjusts its own weights so that similar inputs cause similar outputs PRECEPTRONS : PRECEPTRONS First neural network with the ability to learn Made up of only input neurons and output neurons Input neurons typically have two states: ON and OFF Output neurons use a simple threshold activation function The Perceptron calculates a weighted sum of inputs and compares it to a threshold. If the sum is higher than the threshold, the output is set to 1, otherwise to -1. In basic form, can only solve linear problems - Limited Application Slide 13: The most common neural network model is the multilayer preceptron (MLP), this type of neural network is known as supervised network because it requires a desired output in order to learn. An extension of the perceptron - Multiple layers The addition of one or more “hidden” layers in between the input and output layers. -Activation function is not simply a threshold Usually a sigmoid function -A general function approximation Not limited to linear problems Information flows in one direction The outputs of one layer act as inputs to the next layer BACKPROPAGATION : BACKPROPAGATION Most common method of obtaining the many weights in the network - A form of supervised training The basic backpropagation algorithm is based on minimizing the error of the network using the derivatives of the error function - Simple - Slow -level to local minima issues Most common measure of error is the mean square error: E = (target – output)2 Partial derivatives of the error wrt the weights: -Output Neurons: let: δj = f’(netj) (targetj – outputj) j = output neuron ∂E/∂wji = -outputi δj i = neuron in last hidden Slide 16: -Hidden Neurons: let: δj = f’(netj) Σ(δkwkj) j = hidden neuron ∂E/∂wji = -outputi δj i = neuron in previous layer k = neuron in next layer Calculation of the derivatives flows backwards through the network, hence the name, backpropagation. These derivatives point in the direction of the maximum increase of the error function. A small step (learning rate) in the opposite direction will result in the maximum decrease of the (local) error function: wnew = wold – α ∂E/∂wold ;where α is the learning rate The learning rate is important - Too small Convergence extremely slow - Too large May not converge Momentum - Tends to aid convergence - Applies smoothed averaging to the change in weights: Δnew = βΔold - α ∂E/∂wold wnew = wold + Δnew Acts as a low-pass filter by reducing rapid fluctuations COUNTERPROPAGATION (CP) NETWORKS : COUNTERPROPAGATION (CP) NETWORKS Another multilayer feed forward network. Up to 100 times faster than back propagation Not as general as back propagation Made up of three layers: ► Input ► Kohonen ► Grossberg (Output) HOW DO THEY WORK? : HOW DO THEY WORK? Kohonen Layer: - Neurons in the Kohonen layer sum all of the weighted inputs received. -The neuron with the largest sum outputs a 1 and the other neurons output 0 Grossberg Layer: -Each Grossberg neuron merely outputs the weight of the connection between itself and the one active Kohonen neuron WHY TWO DIFFERENT TYPES OF LAYERS : WHY TWO DIFFERENT TYPES OF LAYERS More accurate representation of biological neural networks Each layer has its own distinct purpose: - Kohonen layer separates inputs into separate classes Inputs in the same class will turn on the same Kohonen neuron - Grossberg layer adjusts weights to obtain acceptable outputs for each class TRAINING A CP NETWORK : TRAINING A CP NETWORK Training the Kohonen layer - Uses unsupervised training -Input vectors are often normalized -The one active Kohonen neuron updates its weights According to the formula: wnew = wold + α(input - wold) where α is the learning rate The weights of the connections are being modified to more closely match the values of the inputs At the end of training, the weights will approximate the average value of the inputs in that class Training the Grossberg layer - Uses supervised training - Weight update algorithm is similar to that used in backpropagation Hidden Layers and Neurons : Hidden Layers and Neurons For most problems, one layer is sufficient Two layers are required when the function is discontinuous ● The number of neurons is very important: - Too few Underfit the data – NN can’t learn the details - Too many Overfit the data – NN learns the insignificant details Start small and increase the number until satisfactory results are obtained. OVERFITTING : OVERFITTING IMPLEMENTATION USING THE MATLAB SOFTWARE : IMPLEMENTATION USING THE MATLAB SOFTWARE The Neural Network are good at fitting functions and recognizing patterns. Neural Networks are composed of simple elements operating in parallel. Neural Networks are adjusted , or trained , so that a particular input leads to a specific target output. Neural Networks have been trained to perform complex functions in various field s, including pattern recognition, identification, classification, speech, vision and control systems. Neural Network can also be trained to solve the problems that are difficult for conventional computers or human beings. The toolbox emphasize the use of neural network concept that build up in the use of financial , engineering and other practical applications. Neural Network toolbox software for the control system describes three practical control system applications and other applications. SIMULINK : SIMULINK The neural network toolbox provides a set of blocks you can use to build neural networks in Simulink or which can be used by the function gensim to generate the Simulink version of any network you have created in MATLAB. BLOCK SET: The result is a window that contains three blocks. Each of these blocks contains additional blocks. Slide 25: TRANSFER FUNCTION BLOCKS: Double-click on the Transfer Functions block in the Neural window to bring up a window containing several transfer function blocks. Each of the blocks takes a net input vector and generates a corresponding output vector whose dimensions are the same as the input vector. Slide 26: NET-INPUT BLOCKS WEIGHT BLOCKS CONTROL SYSTEM BLOCKS FITTING FUNCTION : FITTING FUNCTION To define a fitting problem for the toolbox, arrange a set of Q input vectors as columns in a matrix. Then, arrange another set of Q target vectors (the correct output vectors for each of the input vectors) into a second matrix. For example, you can define the fitting problem for a Boolean AND gate with four sets of two-element input vectors and one-element targets as follows: inputs = [0 1 0 1; 0 0 1 1]; targets = [0 0 0 1]; targets = [0 0 0 1]; The next section demonstrates how to train a network from the command line USING COMMAND-LINE FUNCTIONS : USING COMMAND-LINE FUNCTIONS Let us assume an example with housing data set provided with tool box. Load the data, consisting of input vectors and target vectors, as follows: load house_dataset Create a network. For this example, you use a feed-forward network with the default tan-sigmoid transfer function in the hidden layer and linear transfer function in the output layer. This structure is useful for function approximation (or regression) problems. Use 20 neurons (somewhat arbitrary) in one hidden layer. The network has one output neuron, because there is only one target value associated with each input vector. Train the network. The network uses the default Levenberg-Marquardt algorithm for training. The application randomly divides input vectors and target vectors into three sets as follows: - 60% are used for training. - 20% are used to validate that the network is generalizing and to stop training before overfitting. - The last 20% are used as a completely independent test of network generalization. To train the network, enter: net=train(net,houseInputs,houseTargets); Slide 29: During training, the following training window opens. This window displays training progress and allows you to interrupt training at any point by clicking Stop Training. PERFORMANCE AND REGRESSION PLOT : PERFORMANCE AND REGRESSION PLOT USING THE NEURAL NETWORK FITTING TOOL GUI : USING THE NEURAL NETWORK FITTING TOOL GUI Open the Neural Network Fitting Tool with this command: nftool Click Next to proceed. Slide 32: Click Load Example Data Set in the Select Data window. The Fitting Data. Set Chooser window opens. Select Simple Fitting Problem, and click Import. This brings you back to the Select Data window. Click Next to display the Validate and Test Data window, shown in the following figure. The validation and test data sets are each set to 15% of the original data. Slide 33: 6. Click Next. The number of hidden neurons is set to 20. You can change this value in another run if you want. You might want to change this number if the network does not perform as well as you expect. Click Next. Slide 34: Click Train. This time the training continued for the maximum of 1000 iterations. Under Plots, click Regression. For this simple fitting problem, the fit is almost perfect for training, testing, and validation data. Slide 35: View the network response. For single-input/single-output problems, like this simple fitting problem, under the Plots pane, click Fit. ADVANTAGES AND DISADVANTAGES : ADVANTAGES AND DISADVANTAGES ADVANTAGES: A neural network can perform tasks that a linear program can not. When an element of the neural network fails, it can continue without any problem by their parallel nature. A neural network learns and does not need to be reprogrammed. It can be implemented in any application. It can be implemented without any problem. DISADVANTAGES: The neural network needs training to operate. The architecture of a neural network is different from the architecture of microprocessors therefore needs to be emulated. Requires high processing time for large neural networks. CONCLUSION : CONCLUSION The ability of neural networks to learn and generalize in addition to their wide range of applicability makes them very powerful tools. There is no need to understand the internal mechanisms of that task. They are also used for the real time systems. Finally, I would like to state that even though neural networks have a huge potential we will only get the best of them when they are integrated with computing, fuzzy logic and so on. REFERENCES : REFERENCES [Hayk94] S. Haykin, Neural Networks, New York, NY: Nacmillan College Publishing Company, Inc., 1994. Referred To matlab help THANK YOU : THANK YOU

Related presentations