1.1 Feed-Forward artificial neural networks

Artificial Neural Networks (ANN) are computing systems containing many simple nonlinear computing units or nodes, called neurons, interconnected by links. A feed-forward ANN has a large number of neurons arranged in layers. Generally, all neurons in a layer are connected to all neurons in adjacent layers through uni-direction links, which are represented by synaptic weights. In the three-layer perceptron, schematically depicted in Figure 3, the neurons are grouped in sequentially connected layers: the input, the output and the hidden layers. Each neuron in the hidden and output layer is activated by a non linear activation function that relies on the weighted sum of its inputs and the neuron parameter, called bias, b.

The output of a neuron in the output layer is:

where the h hidden units (processing elements) perform the weighting summation of the inputs xt and the nonlinear transformation by the sigmoid (log-sigmoid or tan-sigmoid) transfer function Y, (.)