| Oct 26, 2023 | Biraj sarmah |

The Inner Workings of Neural Network 1

understanding neural network

 This is a 3. It’s sloppily written and rendered at an extremely low resolution of 28 by 28 pixels, but your brain has no trouble recognizing it as a 3. And I want you to take a moment to appreciate how crazy it is that brains can do this so effortlessly. I mean, this, this, and this are also recognizable as 3s, even though the specific values of each pixel is very different from one image to the next.

understanding the working neural networks in 28by28 pixels image

The particular light sensitive cells in your eye that are firing when you see this three are very different from the ones firing when you see this three. But something in that crazy smart visual cortex of yours resolves these as representing the same idea, while at the same time recognizing other images as their own distinct ideas.

But if I told you, hey, sit down and write for me a program that takes in a grid of 28 by 28 pixels like this, An output’s a single number between 0 and 10, telling you what it thinks the digit is. Well, the task goes from comically trivial to dauntingly difficult. Unless you’ve been living under a rock, I think I hardly need to motivate the relevance and importance of machine learning and neural networks to the present and to the future.

But what I want to do here is show you what a neural network actually is, assuming no background, and to help visualize what it’s doing. Not as a buzzword, but as a piece of math. My hope is just that you come away feeling like the structure itself is motivated, and to feel like you know what it means when you read or you hear about a neural network quote unquote learning.

This video is just going to be devoted to the structure component of that, and the following one is going to tackle learning. What we’re going to do is put together a neural network that can learn to recognize handwritten digits.

This is a somewhat classic example for introducing the topic, and I’m happy to stick with the status quo here, because at the end of the article, I want to point you to a couple good resources where you can learn more, and where you can download the code that does this and play with it on your own computer.

neural networks

There are many, many variants of neural networks, and in recent years there’s been sort of a boom in research towards these variants, but in theseseries of post You and I are just going to look at the simplest plain vanilla form with no added frills. This is kind of a necessary prerequisite for understanding any of the more powerful modern variants.

And, trust me, it still has plenty of complexity for us to wrap our minds around. But even in this simplest form, it can learn to recognize handwritten digits, which is a pretty cool thing for a computer to be able to do. And at the same time, you’ll see how it does fall short of a couple hopes that we might have for it.

As the name suggests, neural networks are inspired by the brain. But let’s break that down. What are the neurons, and in what sense are they linked together? Right now, when I say neuron, all I want you to think about is a number. Specifically, a number between 0 and 1. It’s really not more than that. For example, the network starts with a bunch of neurons corresponding to each of the 28×28 pixels of the input image, which is 784 neurons in total.

Each one of these holds a number that represents the grayscale value of the corresponding pixel. Ranging from zero for black pixels up to one for white pixels. This number inside the neuron is called its activation. And the image you might have in mind here is that each neuron is lit up when its activation is a high number.

So all of these 784 neurons make up the first layer of our network.

Now, jumping over to the last layer, this has 10 neurons each representing one of the digits, the activation in these neurons. Again, some number that’s between 0 and 1 represents how much the system thinks that a given image corresponds with a given digit. There’s also a couple layers in between called the hidden layers, which for the time being should just be a giant question mark for how on earth this process of recognizing digits is going to be handled. 

In this network, I chose two hidden layers, each one with 16 neurons. And admittedly, that’s kind of an arbitrary choice. To be honest, I chose two layers based on how I want to motivate the structure in just a moment. And 16? Well, that was just a nice number to fit on the screen. In practice, there is a lot of room for experiment with a specific structure here.

The way the network operates, activations in one layer determine the activations of the next layer. And of course, the heart of the network, as an information processing mechanism, comes down to exactly how those activations from one layer bring about activations in the next layer. It’s meant to be loosely analogous to how in biological networks of neurons, some groups of neurons firing cause certain others to fire.

Now the network I’m showing here has already been trained to recognize digits, and let me show you what I mean by that. It means if you feed in an image, Lighting up all 784 neurons of the input layer, according to the brightness of each pixel in the image, that pattern of activations causes some very specific pattern in the next layer, which causes some pattern in the one after it, which finally gives some pattern in the output layer. Continue reading …

Readers Also Read This

Understanding machine Learning

Understanding Machine Learning

inner workings of gradient descent

The Inner Working of Neural Networks |2|

Text To Speech

Artificial Intelligence Has No Reason to Harm Us: Deep Dive Analysis

how to improve accuracy in machine learning

How To Improve Accuracy In ML


Join Discord

Unleash AI Text-to-Speech Excellence – Elevate Your Voice, Join Discord and Speak Your Mind with Cutting-Edge Technology!

Join a community of over 200k

Hear From Your Favorite


Or Subscribe for All Alerts