One of the main goals of deep learning systems is to build robust models that can make generalizations out of distribution data. One of the secret ingredients to building such models is a large dataset. Some of the recent progress that has been made in deep learning has been due to the availability of large datasets such as the image-net (which has more than 14 million images) for training computer vision systems and the Wikipedia corpus for training NLP systems.

In the case of deep learning, it is common to encounter a problem whereby the data to be used has limited size. In order for the data to be correctly generalized in your model, you will need more data and this is where data augmentation comes in. Also, in situations where you have poor model performance, one of the techniques used to improve model performance is Data Augmentation

This piece would clearly explain data augmentation and how it is carried out for deep learning when you have a limited dataset.

Before we go further, we need to define the two major terms we will discuss - deep learning and data augmentation.

What is deep learning?

Deep learning is a subset of ML- Machine Learning, that makes use of Artificial Neural Networks (ANNs) which is inspired by neurons found in the human brain to to build systems/models that are capable of learning and producing state of the art results in certain tasks.

What is data augmentation?

“Data Augmentation allows one to artificially increase the size of a dataset without having to collect more data. Data augmentation is key to improve the performance of our models.”

       -Santiago @svpino, Machine learning expert

Data augmentation is a technique in data training that involves increasing your training set by altering or making some modifications to the original dataset in order to improve model performance.

It  involves adding data to your model by modifying the already existing data in such a way that the main data is not completely or glaringly changed. The data could be images- by increasing the number of images in the dataset, you've carried out data augmentation. Some data augmentation techniques used in training computer vision models include flipping the images, adding some noise to the dataset (gaussian noise), resizing and cropping e.t.c.

Most times, the dataset may not be enough to carry other variations so samples from the existing data would have to be used to generate more data. Data augmentation can be an invaluable tool when working with a limited dataset.

The connection between deep learning and data augmentation

Lots of deep learning models require large datasets to produce good results, and this can be attributed to the large number of weights that need to be tuned during training.

Data collection is time and cost consuming in terms of funds, human effort, computational resources and the time involved to collect and process the data.

Having to collect more data when your dataset is having limited data as you have seen would be really strenuous. By cloning and tweaking your existing data, you've carried data augmentation which has positively compensated for the cost involved in further data collection.  

NB: Data augmentation is only relevant when you have limited data and also when you want to prevent overfitting during data training.

How data augmentation can be carried out practically in image data augmentation

Generally the most well known type of data augmentation is image data augmentation.

Below are a few techniques to carry out data augmentation on image data if your dataset has limited data.

1. Flip: Flipping images on horizontal or vertical axis

2. Rotation: Rotate an image with certain degree

3. Crop: Randomly, crop a section from given image and resize

4. Add Noise: Adding Gaussian noise to a given image

5. Color Jittering: Random color manipulation

Why data augmentation?

Asides eliminating further data collection, data augmentation provides other benefits like:

•It can help you scale the size of your original training set- By cloning your existing data, you automatically increase the data which was previously limited for your deep learning process.

• It provides better training examples for your model to learn from

• It provides ready data for all types of inputs during production

• It reduces data overfitting by providing higher prediction errors and increased diversity in your deep learning model.

• It increases generalization ability

• It solves class imbalance issues

In conclusion...

  • Data augmentation is used in deep learning when there is limited data to create more data from the existing data in the dataset.
  • Image data augmentation is the most common type of data augmentation, which involves rotation, flipping, cropping, sampling and other techniques to increase the variation of your data set.

Interested in upscaling your deep learning process with data augmentation, click here