Image Recognition at Speed Do you wonder how your phone’s camera is able to instantly identify you for face unlock, or how a self-driving car that is in motion can recognize a person in front of it, or how medical software is able to detect tiny tumors in your X-ray before a human doctor does?
Image recognizability is made possible by Convolutional Neural Networks (CNNs), and I want to explore these behind-the-scenes implementations to better understand how they actually work. They are used in your smartphone to recognize your face to unlock your phone. They are used in your self-driving car to instantly recognize a pedestrian in front of it. They are used in medical software to instantly recognize microscopic tumors in your X-rays before a human doctor can.
If you are curious about the possibilities of AI and are trying to understand the basics of how it works, then you are in the right place. Whether you are an aspiring data scientist, a software developer that is just starting to look into machine learning, or even a tech enthusiast, we will take a deep dive into all of the details that you need to understand about how it works to recognize objects in images.
We will go through CNN Explained in detail. The CNN architecture, several CNN applications, the CNN mathematical basis, and the CNN implementation will be described in this article.
CNN Explained: What is a Convolutional Neural Network?
First to get an idea of what they do we must look at how we as humans perceive images. A photo of a golden retriever is processed by your brain as a hierarchy of increasing complexity. The lowest level of processing is the edges, curves, and textures in the image. These are then grouped into features of objects in the image, i.e. the ears, the snout, and the tail of a dog. Eventually the highest level of processing in the human visual cortex recognizes the combination of all these features to recognize a dog.
Traditional computer vision algorithms for object recognition were written by engineers by hand and included complex rules for detection of visual features, which were manually engineered by developers (referred to as manual feature engineering).
A Convolutional Neural Network (CNN) completely automates this process.
Convolutional neural networks are a special kind of deep learning. They were designed to work with data that present themselves in a grid form, which images are an example of.A CNN consists of several layers. Every layer consists of several convolutions. Each convolution is followed by a ReLU function. Once in a while a layer consists of a pooling instead.
Raw Image (Pixels) [Convolution ReLU] [Pooling] [Fully Connected] Classification Output.
Why CNN in Deep Learning Changed Everything
Deep Learning’s CNN vs. Deep Learning’s MLP (Before Deep Learning’s CNN) – II
The number of parameters for a single fully connected layer is extremely large. Even a modest size image of 256x256 pixels with 3 color channels contains 196,608 pixels, and a single layer of such an image would require tens of millions of parameters. The amount of memory such a layer would require would already be too high for today’s hardware, not to mention the extreme amount of overfitting.
Loss of Spatial Structure: If a neural network is trained as a feedforward network (Multilayer Perceptron) the 2D images have to be transformed to 1D vectors, which completely discards the spatial relationships between pixels, i.e., two eyes are side by side and above the nose.
CNN in Deep Learning is the most commonly used computer vision algorithm today. The architecture of CNNs in deep learning preserves the spatial relationships in a given image by looking at the images in a 3D volume, i.e., images are of width, height, and depth (which are the color channels for a given pixel). Each of the depth (color channels) for a given pixel in the 3D volume can be thought of as a 2D feature map, and hence, the number of parameters is drastically reduced.
This is in contrast to the traditional neural network architectures for computer vision, where the images are first flattened into 1D vectors, and hence, spatial relationships are lost. The number of parameters is also drastically reduced in the convolutional neural networks for computer vision in Deep Learning using the following two properties of the convolutional neural networks (CNNs) in deep learning, i.e., parameter sharing and local connectivity.
Core Architecture: The Anatomy of Convolutional Neural Networks
CNNs in deep learning form the basis of computer vision. A typical CNN consists of a sequence of layers. We can describe the basic building blocks of CNNs.
┌───────────────────────────────────────────────────────────┐
│ INPUT IMAGE │
└─────────────────────────────┬─────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────┐
│ 1. CONVOLUTIONAL LAYER │
│ Applies small filters/kernels to extract features │
└─────────────────────────────┬─────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────┐
│ 2. RELU ACTIVATION LAYER │
│ Introduces non-linearity to capture complexity │
└─────────────────────────────┬─────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────┐
│ 3. POOLING LAYER │
│ Reduces spatial dimensions & computational load │
└─────────────────────────────┬─────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────┐
│ 4. FULLY CONNECTED LAYER │
│ Flattens feature maps & performs classification │
└─────────────────────────────┬─────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────┐
│ OUTPUT / SOFTMAX LAYER │
│ Generates probability scores for classes │
└───────────────────────────────────────────────────────────┘
The heart and soul of any convolutional neural network is the convolutional layer.
A kernel (or filter) is a small matrix of numbers, such as 3x3 or 5x5, which slides over the input image.
- Feature Map: The preceding output is then processed by multiple convolutions in the following layer to create a so-called Feature Map or also called an Activation Map which is then written to memory. The preceding feature maps now contain the specific visual features of the input image such as horizontal lines, edges, corners etc. as well as color gradients.
1. The Stride of the Filter
For every pixel within the filter (moving it pixel by pixel within the image) the filter performs multiplication of the corresponding elements of the image filter and itself (resulting in a partial sum). The resulting partial sum is added to the preceding total sum of pixels within the preceding feature maps (resulting in a total sum for the current feature map). The resulting total sum is stored within the corresponding feature map of the preceding layer. The filter is moved (either one by one or every other pixel) by a defined amount (the so-called stride) within the image to the right, followed by a subsequent move to the bottom.
- Padding: The amount of padding (i.e., adding border pixels with values usually set to zero) around the input image. This is to ensure that the output size of the convolutional operation (i.e., after max pooling) will have the same spatial dimensions as the input image, rather than being ‘cut down’ at the edges.
2. The Activation Function (ReLU)
Most of the common convolutional neural networks are making use of the Rectified Linear Unit (ReLU) function as an activation function in the convolutional as well as pooling layers. The function maps all negative values to 0. Thus in some cases they are not passed forward by the activation function.
\text{ReLU}(x) \max(0, x) .
$\text{ReLU}(x) = \max(0, x)$
ReLU. In fact the ReLU function reports only positive values and zeros out the rest. Also which ever negative values that may have been present are not reported by the activation function.
3. The Pooling Layer (Dimensionality Reduction)
As an image goes through more and more layers of convolution, the feature maps are becoming more and more detailed. However, doing full-resolution feature maps for dozens of layers would be expensive for the computer and prone to overfitting.
- The Pooling Layer for Dimensionality Reduction: Downsample in the case of CNN. In which we looked at the Convolutional Layer’s role in feature extraction in past sections. The Pooling Layer in a CNN which reduces the spatial dimensions (width and height) of the Feature Maps that come out of the Convolution. There are several methods for pooling.
- Max Pooling: This form of Down-sampling for the CNN (for Dimensionality Reduction) uses the maximum value within a defined set of regions (like a 2x2 window) for determining the feature information for the next layer of convolutional feature extraction.
- Average Pooling: Average pooling layers use average pooling in order to obtain a smoother representation of the input features. The average value of the values within the window is used.
- Average Pooling: we see that Average Pooling does what it does by taking the average of all pixels in the window. Thus Average Pooling brings out a smooth features of the input as opposed to the sharp peaks that Max Pooling does.
4. Fully Connected (FC) Layer
Input to the FC layer will be feature maps produced by the preceding layers of CNN. Thus, the CNN produces a feature map at the output of the preceding CNN layers. These feature maps, which are multidimensional, are fed through the layers of the CNN, and in each subsequent layer, the feature maps are flattened to form a single vector (or column vector) that is input to the next layer of the CNN.
Multidimensional feature maps from preceding convolutional and pooling layers are transformed in a flat feature vector of high dimensionality, which is input for subsequent fully connected layers.
5. Output / Softmax Layer
Softmax Layer: Since for our classification task with CNN we use Softmax as the output layer’s activation function which puts out a probability between 0 and 1 for each class in a multi class setting (for example P(cat) 0.8; P(dog) 0.2). Sigmoid activation function on the other hand is used for binary classification. Also we see Convolutional Neural Networks in a wide range of real world applications.
Real-World Applications of Convolutional Neural Networks
From academia to industries: the applications of CNNs in Deep Learning are numerous and cross boundaries of industries.Note: The list above is not exhaustive. There are many more applications of Convolutional Neural Networks.
Industry
Real-World Application
How CNNs Help
Healthcare & Medicine
Medical Imaging Diagnosis
It looks for diseases, lesions and fractures in CT, MRI and Xray images.
Automotive
Autonomous Vehicles & Drones
Identifies lanes, pedestrians, traffic signs, and obstacles in real-time.
Security & Devices
Facial Recognition & Biometrics
Authenticates user identities on smartphones and powers security surveillance systems.
Retail & E-Commerce
Retail & E-Commerce
Visual Search & Recommendation
Uploading a clothing photo to allow users to search for similar products online.
Manufacturing
Defect Inspection
Scans high-speed assembly line items to flag microscopic manufacturing flaws.
Convolutional Neural Networks Tutorial: Building a Simple CNN Flow
Convolutional Neural Networks Tutorial: Building a Simple CNN Flow
To better understand how CNNs are implemented in deep learning, we go through a step-by-step guide of how to design, train and deploy a CNN model using frameworks such as TensorFlow/Keras or PyTorch in a tutorial for building a simple CNN flow.
Step 1: Data Preparation & Augmentation
A machine learning model is only as good as the data that has been used to develop the model.
This is a basic scaling of the pixel values from the typical 0–255 RGB range used in most image processing programs to a 0–1 float range.
Apply Data Augmentation: Rotate images, flip them horizontally or zoom in/out on them in order to increase size of training data and prevent overfitting.
Step 2: Architecture Construction
Below is the simple pipeline to construct the basic CNN architecture using the TensorFlow/Keras API or PyTorch in Python:
Python
import tensorflow as tf
from tensorflow.keras import layers, models
# Initialize Sequential Model
model = models.Sequential([
# Layer 1: First Convolutional + ReLU + MaxPool
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(64, 64, 3)),
layers.MaxPooling2D((2, 2)),
# Layer 2: Second Convolutional + ReLU + MaxPool
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
# Layer 3: Flatten & Dense Layers
layers.Flatten(),
layers.Dense(128, activation='relu'),
# Output Layer for Binary Classification
layers.Dense(1, activation='sigmoid')
])
Step 3: Model Compilation
Next, we configure our network to learn, by choosing an optimizer, a loss function, and a set of evaluation metrics.
Python
model.compile(
optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy']
)
Step 4: Training & Backpropagation
Training & Backpropagation - Call model.fit() to feed image batches through the network.
- Forward Pass: Images pass through layers to create predictions.
- In terms of Loss Calculation: the model puts out a guess, which is then compared
- to the true value. The backpropagation/gradient descent algorithm uses the output information from the above steps to update the weights for each kernel. In each epoch, the model is improved.
Iconic Convolutional Neural Network Architectures
This is a CNN which was developed by Yann LeCun for recognizing handwritten digits.
- LeNet-5 set the stage for what today we have in modern CNNs. In
- 2012 AlexNet, which is a work by Alex Krizhevsky and Geoffrey Hinton, did very well in the ImageNet challenge which in turn caused the current deep learning boom. Also in
- 2014 we saw VGGNet which reported that very deep nets composed of mostly 3x3 convolutions did in fact perform the best.
- ResNet (2015): Residual connections for deep neural networks (hundreds of layers).
- Hybrid Vision Models: Utilize the edge extraction capabilities of CNNs in high speed, while leveraging the global context modeling of high efficiency Transformers for classification.
Ready to Accelerate Your Career in AI & Deep Learning?
While going through the above article “What are Convolutional Neural Networks”, the reader gets to know the basics of Convolutional Neural Networks for starting to learn about artificial intelligence, computer vision and machine learning engineering in general. This article aims to help readers to start their careers in AI.
Time to Act!
Get hands-on practice implementing a variety of custom CNNs and computer vision models, all with real-world applications to real problems. Learn how to implement a complete deep learning project from start to finish using the popular Python deep learning libraries, PyTorch and TensorFlow.
Frequently Asked Questions (FAQs)
1. Are convolutional neural networks only used for images?
Actually, Convolutional Neural Networks were originally designed to handle images and videos, but not only that… these types of data. Any sequential or grid-like data can be processed by a CNN, for example, 1D CNNs for audio signal processing, time-series forecasting, and natural language processing (NLP) for text classification, such as in the area of sentence classification.
2. What is the main difference between CNNs and traditional Neural Networks?
A traditional neural network treats each pixel in an image as input. All these pixels are then flattened into a 1D vector, which is then processed by the subsequent layers. However, this way the very important spatial relationships between pixels are destroyed. Also, the number of parameters would explode. Instead, the CNN uses shared weights (the filters) and local receptive fields to handle the spatial relationships between pixels, while keeping the number of parameters low.
3. Why are Pooling Layers crucial in a CNN?
Each of the Pooling Layers in a CNN perform downsampling of the corresponding Feature Maps. This reduces the number of parameters, memory usage, and computational cost required. In addition, the Pooling Layers enable detection of features by the CNN that are invariant to translational, rotational, and scaling changes in the images.
4. How do you prevent overfitting in Convolutional Neural Networks?
Common techniques to prevent overfitting in Convolutional Neural Networks include:
Data Augmentation: Rotating, cropping, and flipping training images.
Dropout: This trains the network with random subsets of the neurons (eg, 50% of them) in order to prevent them from ‘co-adapting’ in order to maintain accuracy.
Batch Normalization: Normalizes activations (from previous layers) to improve training and prevent learning from getting stuck in local optima.
Early Stopping: This stops training when the validation loss stops improving.
5. Can CNNs be used when you have a very small dataset?
That is true. However, with limited training data for your specific data set, Transfer Learning can allow you to reach high accuracy with only a few hundred images per class (as opposed to millions of training images, such as the images in the ImageNet data set used for training the pre-trained CNN models, e.g. ResNet, MobileNet, etc.).
Do visit our channel to learn More: SevenMentor
Dhanaji Kokare
Expert trainer and consultant at SevenMentor with years of industry experience. Passionate about sharing knowledge and empowering the next generation of tech leaders.