ARTIFICIAL NEURAL NETWORKS

Without going into detail about the history, let's just say neural networks were invented in the sixties by mathematicians (back in those days, computer scientists were often mathematicians; my how things have changed) looking for ways to model processing in the human brain at the lowest level. They took what they knew about a human neuron as a model for creating an electronic neuron.
3.2.1 The Neuron
These researchers started by boiling down to how they viewed a neuron as working in light of existing knowledge. A neuron, they said, may have many inputs ("dendrites") that a hooked into the single output ("axon") of another neuron. Signals received by some dendrites would tend to activate the neuron, while signals on others would tend to suppress activation
*Pattern Matching
Here is where a lot of introductions stop short. Let me explain what it means for one of these neurons to "know" something before explaining how they work in concert with other such neurons.
The goal of a neuron of this sort is to fire when it recognizes a known pattern of inputs. Let's say for example that the inputs come from a black and white image 10 x 10 pixels in size. That would mean we have 100 input values. For each pixel that is white, we'll say the input on its corresponding dendrite has a value of -1. Conversely, for each black pixel, we have a value of 1. Let's say our goal is to get this neuron to fire when it sees a letter "A" in the image, but not when it sees any other pattern. Figure 3 below illustrates this:
"Firing" occurs when the output is above some threshold. We could choose 0 as the threshold, but in my program, I found 0.5 is a good threshold. Remember; our only output (axon) for this neuron is always going to have a value between -1 and 1. The key to getting our neuron to recognize an "A" in the picture is to set the weights on each dendrite so that each of the input-times-weight values will be positive. So if one dendrite (for a pixel) is expecting to match white (-1), its weight should be -1. Then, when it sees white, it will contribute -1 * -1 = 1 to the sum of all inputs. Conversely, when a dendrite is expecting to match black (1), its weight should be 1. Then, its contribution when it sees black will be 1 * 1 = 1. So if the input image is exactly like the archetype our single neuron has of the letter "A" built into its dendrite weights, the sum will be exactly 100, the highest possible sum for 100 inputs. Using our "sigma" function, this reduces to an output of 1, the highest possible output value, and a sure indication that the image is of an A.
Now let’s say one of the input pixels was inverted from black to white or vice-versa. That pixel's dendrite would contribute a -1 to the sum, yielding 98 instead of 100. The resulting output would be just a little under 1. In fact, each additional "wrong" pixel color will reduce the sum and hence the output.
If 50% of the pixels were "wrong" - not matching what the dendrites say they are expecting as input - the sum would be exactly 0 and hence the output would be 0. Conversely, if the entire image of the "A" were inverted, 100% of them would be wrong, the sum would be -100, and the output would be -1. Since 0 and -1 are obviously below our threshold of 0.5, we would say this neuron does not "fire" in these cases.
The most important lesson to take away from this simple illustration is that knowledge is encoded in the weights on the dendrites. The second most important lesson is that the output of this sort of thinking is "fuzzy". That is, our neuron compares input patterns and generates variable outputs that are higher the closer the inputs are to its archetypical pattern. So while there are definitive "match" (1) and "non-match" (-1) outputs, there is a whole range in between of somewhat matching.