how neural networks work: exploring principles and practical applications

How many algorithms have you implemented today? - If today is a weekday, then I will wake up at six, otherwise I will continue to sleep. - If a few of my colleagues have gathered at the coffee machine in the morning, I will pour myself coffee, otherwise I will go and look through my news feed. If you believe in the free choice of your immortal soul, forget it. You are controlled by a neural network. Every day, your neurons, hand in hand with synapses, decide what to do with you. At the end of the article, you will find a pleasant or not very understanding: there is no free will - only solid mathematics.

Why do neural networks need to recognize cats? Because they can

Techies are the laziest people on the planet. At first it is difficult for them to wash and they invent a washing machine, then it becomes tedious to wash with their hands and ... You understand. Worse than them only IT people. The crowning achievement of their laziness is an artificial neural network. It is almost like a real one, but only in a computer. There are three global applications for neural networks:

1. Classification. For example, if you choose a mug for morning coffee, then distribute all the mugs according to the parameters - what size it is, color, whether there is a print in the form of cats - and only then take the one you like the most. The neural network classifies faster and better than you. No offense.

2. Prediction. The neural network will not be able to predict your fate (and even if it can, it will not tell you), but it can predict the growth of stocks based on the situation in the stock market.

3. Recognition. This is the most interesting application of neural networks. How many of you determined your age from a photograph? There are neural networks that determine whether there is a cat in the photo. And there are even those that not only determine, but also drive someone else's cat off your lawn. But do not judge the moral qualities of all neural networks by one of its representatives.

Marshmallow or not marshmallow, that is the question

An artificial neural network, of course, is not an easy task. Let's leave the implementation to the experts, and we'll go to the top ourselves. Let's assemble our own neural network, which will decide whether to buy this pack of marshmallows for us. And the problem of mankind will be solved.

We have four parts: three neurons and one synapse. This is all we need to make any decision. There are three types of neurons: input, output, and hidden. Input neurons take on information, transmit it to the hidden ones. Those, in turn, think about the information and give it to the output neurons, which announce the decision.

Synapses are neural connections that allow neurons to communicate.

Imagine standing in front of a candy shelf in a store. At this point in your life, only three things are important:

1. It's winter now, and in winter, hot cocoa with marshmallows is especially beautiful.

2. You love pink.

3. But don't like chocolate.

Input neurons understand only two words from human language: yes and no. Therefore, let's take three input neurons, each of which is waiting for an answer to its own question:

1. Is it winter now?

2. Is the marshmallow pink?

3. Is it covered in chocolate?

Until the neurons know this, they need to be informed. And when they find out, they keep it inside themselves. Neurons store numbers in themselves - this is their language and they know it very well. In this language, they communicate with synapses. Each synapse (connection) has its own weight. In fact, this is the most important thing, because weight is everything.

Let's finally tell the input neurons what's going on: it's not winter outside, the marshmallows are pink and covered in chocolate. Let's also assume that the first synapse has a weight of 0.5, the second has a weight of 0, and the third has a weight of 0.5.

If the input neuron says "yes" to the synapse, then the synapse fires and passes on its weight. In our case, the second and third synapse will transmit 0 and 0.5, while the first will not transmit anything. The red output synapse will receive the final value. He will make a positive decision if he receives an amount equal to or greater than 0.5.

In our case, he gets 0 + 0.5 = 0.5 and decides to buy marshmallows. And now you are standing at the checkout, summer is blooming outside the window, in your hands is a pack of pink marshmallows in chocolate, and you understand that this is not at all what you wanted. Something went wrong in your life.

What did you expect? Neural networks need to be trained.

School for naughty neural networks

When a decision made by a neural network does not fit into any framework of the law and your morality, you need to do something about it. Some data that we set was incorrect, hence the following result. It’s really summer outside the window, you really are a lover of delicate pink shades and you haven’t flared up with a passion for chocolate. So something is wrong with the weight of the synapses. This is how neural networks are trained. After each check, the weight of the synapses that led to the error is changed. We are not IT people, not lazy щтуы, so we will do it manually. We will give the first and second synapses a weight of 0.5. If one of these conditions is met, we will take the marshmallow from the shelf. And we will give the third synapse a weight of -1.5, because under no circumstances do we want to see chocolate on our favorite dessert, God forbid. Checking.

The output synapse tells us the result is "-1" and advises against taking this marshmallow. Now the network is working properly. And now we have one good news and one bad news. The good news is that we now have a smart neural network that knows your marshmallow preferences. Well, the bad one is no sweets today.

Training neural networks is the second big pain after designing their architecture. Any change in the input data completely changes the value of all neurons. And if we can analyze three synapses, then manually tracking the values of a million synapses is simply impossible. We conclude that machine learning specialists do not eat their bread in vain.

Problem under the asterisk: hidden neurons

But our desires are much more complicated. Let's say you're having a bad day at work, and a couple of airy sweet marshmallow pillows could save not only your mood, but the whole world. In this case, you are ready to make concessions and buy marshmallows with chocolate. But only if it's winter outside and the marshmallows are pink. In summer, you are allergic to chocolate. But if without chocolate, then any will do. Weird requests, but let's see what we can do. And hidden neurons will help us. Now each incoming neuron is connected to the hidden one. And each connection has its own weight. Roughly speaking, hidden conditions help to implement additional conditions that need to be taken into account in the neural network.

There can be many hidden neurons that set new conditions for the neural network. This complication is called generalization. This is exactly what the human brain is capable of - generalizing many factors to make the only right decision.

How many neurons does a bee have

Our mind is certainly more complex. As well as any intelligent neural network is more complicated. But the general principles of communication between neurons and synapses remain unchanged and common to man and machine. It seems like this whole story with marshmallows does not sound difficult. And it seems that everything should not be so simple. The difficulty lies in the fact that there are millions of millions of such neurons. And to work with such huge amounts of information, complex mechanisms are needed.

A bit of interesting math: the average human has 1011 neurons. At this time, the frog has only 107 of them, and the bee has 106. In 2012, artificial neural networks could not even catch up with the frog in terms of the number of neurons. For example, the neural network for image recognition of those years contained 106 neurons. Like a bee. But we have nothing to rejoice at and there are two reasons for this. First, technology has come a long way since 2012. And secondly, even then such a neural network was smarter in its field not only of a frog, but also of a person, despite the number of neurons.

Our neurons are designed to solve many different tasks. They control the movement of the body, and the decisions made, and the desire to stroke the cat. In contrast, all the neurons of the neural network are aimed at only one task - to recognize the picture. And they are not distracted by trifles. To date, Nvidia's most hype neural network, which turns simple sketches into realistic images, already contains 1.7 * 109 neurons.

There are two morals to this whole story. First - now in the smoking room you can boast of understanding the operation of the simplest neural network. But probably no one will believe you due to the fact that everything can not be so simple. And the second is to understand that in our head, as in an artificial neural network, no magic happens. If you disassemble the huge mechanism into several parts, then everything does not seem so global. The principle of operation of our brain is electrical connections between neurons that communicate with each other, transmit signals and perfectly master their neural arithmetic.