A Brief Algorithmic & Historical Analysis of Convolutional Neural Network’s
With neural networks being at the forefront of cutting edge data science they are one of the most promising and exciting things in the world of science in general. They would be the cause of a lot of your favorite sci-fi movies such as Irobot and the Terminator movies. The most basic neural network is a series of algorithms that try and recognize underlying relationships and patterns in a set of data. The name neural network loosely comes from trying to mimic the human brain. They can adapt to changing inputs, so the network generates the best possible result without the need to redesign the output criteria. The pioneers of the neural network, Warren McCulloch and Walter Pitts in 1943 created a computational model based on algorithms called threshold logic.
There are several different types of neural nets that can do vastly different and do amazing things; from recognizing patterns in human speech all the way to being able to tell a human’s mood based on their facial expression. But this article will focus on one of the most important and widely used neural networks, the Convolutional Neural Network. While this is not a research paper, I would like to high light some of the most important and groundbreaking parts of a Convolutional Neural Network for people who may be new to deep learning as I once was not long ago.
As stated above neural nets are very good at recognizing patterns, so with a CNN, you start with the input being an image and the output being a labeled decision, that can then be classified. This is possible because the image is passed through a series of filters. This is where the specificity lies, these filtering layers must at least have one convolutional layer. The reason for this type of architecture is this allows the neural net to be able to process more complex images. Real-world applications of CNN’s are:
- Facebook’s automatic tagging algorithms
- Tesla’s semi-automated driving car which is object detection
- Face recognition on social media
- Image analysis in healthcare
While one would think that a computer has to go through every pixel in a picture in order to understand it this is not the case, this would actually be very computationally expensive. Not only because there are lots of pixels but the computer would also have to assign RBG values to each pixel. Therefore pre-processing is needed, this is done in several layers that are found at the beginning of the neural net to reduce it into a smaller input before the images can be applied. The most important layers in a CNN are as you probably guessed, the convolutional layers. A Convolution is applied to small sections of an image, sampling the values of the pixels in this section and converting it into a single pixel. It is applied to the entire image in order to make a new image. The idea behind it is that the pixels in the new image incorporate information about the surrounding pixels. This then should reflect how well a feature is represented in that area. Convolutions have two predominant attributes, size, and stride. Stride is how many times to shift over the input matrix or image. Size is how large the filter is, e.g. 3x3.
Now we will shift focus to more of the historical focus of the CNN. The first notable success of convolutional neural nets was AlexNet which was developed by Alex Krizhevsky in 2012 at the University of Toronto. It was developed to solve the ImageNet challenge. This was an image classification problem with over 1.2 million images to classify into 1000 different categories. AlexNet won first place with an error rate of 15.3%, compared second place which was 26.2%. The winning network constituted of 5 convolutional layers, each followed by a ReLU activation layer then followed by three fully connected layers. Also, three of the five convolution-activation pairs were followed by max-pooling layers.
This is just a short introduction into one of the cornerstones of the future. Soon CNN’s will be apart of everyday life for most humans. If you have not yet worked with CNN’s I would highly recommend it and I hope this article motivates you to do so.
