Navigate this blog
In this post
Can Machines Decide What to Learn?
In my earlier blogs, we started exploring how machines think and learn, either through an explicitly guided mechanism or through an approach where the machine can identify patterns in the data without being explicitly guided, but within a controlled environment of parameters and hyperparameters. We also looked at the notion of operant conditioning, where the machine is taught to learn based on the consequences of its actions.
Most of these approaches require some level of guidance from domain experts, particularly when it comes to identifying the features that the machine should analyze to learn patterns and predict a category, a value or the next action. But what happens when the relevant features cannot be explicitly derived or are unknown? Can the machine identify or engineer them on its own?
The Evolution of Deep Learning
This is where deep learning, a specialized discipline within machine learning, comes into the picture. It builds upon the foundations of Artificial Neural Networks (ANNs), which have been around since the 1940s and 1950s, and takes them further through the use of neural networks with multiple layers.
The early work on neural networks included Frank Rosenblatt’s Perceptron in the late 1950s. However, the ability to train networks with multiple layers became more practical with the development and popularization of backpropagation in the 1980s. Researchers such as Geoffrey Hinton, Yann LeCun and Yoshua Bengio played significant roles in advancing neural networks and deep learning.
The field gained considerable momentum in the 2000s and especially after 2012, when deep neural networks began achieving impressive results in image recognition. Another interesting development came in 2014, when Ian Goodfellow and his colleagues introduced Generative Adversarial Networks (GANs), opening up new possibilities for generative modelling.
Letting Machines Learn What Matters
So, what is the purpose of moving towards this discipline? The idea was to let the machine learn which features or representations are useful for solving a given problem. While human experts can decide which combination of features (independent variables or predictors) may help the machine predict the output (dependent variable or target), it was time to see if the machine could identify features that were overlooked or unknown to human experts.
This was particularly useful for problems involving complex, non-linear relationships and perception-based data such as images, videos and text.
Let’s take an example where the input samples are images of dogs and cats, and the output is the class label namely, “dog” or “cat”. A human expert might suggest a set of features such as the shape of the ears, nose, distance between the eyes, nose and mouth, edges, colors and textures. Some experts could go even further and identify more specific features.

The machine would then be trained using these selected features to learn patterns and classify dogs and cats. But what happens when the training data does not adequately represent the variations within these categories? For example, a Chihuahua has distinctive facial proportions and large ears compared with many other dog breeds. Similarly, a Devon Rex cat has unusually large ears and distinctive facial features.
Human experts may have to revisit the features and determine whether the existing set is sufficient to capture these variations. This process could repeat every time the model encounters a new variation or a previously unseen pattern. The feature engineering process could become time-consuming and may also introduce human bias or errors. There could be a constant back and forth over which features really matter until some level of consensus is reached.
Taking Charge of Feature Engineering
To address this, researchers developed approaches that allow machines to learn useful features and representations directly from the input data. This is one of the key ideas behind deep learning.
Continuing with our example of classifying images of dogs and cats, the input image is represented as a grid of pixel values and processed through a series of layers in a deep neural network. In image-based networks, certain layers use mathematical filters (also called kernels) that move across different regions of the image to identify patterns. The initial layers may learn to recognize simple patterns such as edges and textures. As the information passes through successive layers, the network can learn more complex patterns, such as shapes, parts of objects and eventually develop representations that help distinguish one object from another.

These filters are not necessarily designed by human experts. Their values are learned during training through a process called backpropagation, which adjusts the network’s parameters based on the error in its predictions.
The result is that the machine can learn increasingly useful representations from the input data, rather than depending entirely on a predefined set of features supplied by human experts. If new variations of dogs or cats are introduced, the model can potentially learn to recognize them when it is retrained or fine-tuned with sufficiently representative examples.
How Deep Should a Network Be?
A neural network may have just a few layers, in which case it is commonly referred to as a shallow network. As the number of layers increases, the network can learn representations at multiple levels of abstraction. However, simply increasing the number of layers does not automatically make a network better. The architecture, training data and the way the network is trained all matter.
There are different ways in which these layers can be constructed, depending on the type of problem and input data.
One of the most basic types of layers is the Dense layer, also known as a fully connected layer. Each neuron in a Dense layer receives input from all the neurons in the preceding layer. These layers are commonly used in traditional feed-forward neural networks and often towards the end of other architectures to perform classification or regression.
Then came Convolutional Neural Networks (CNNs), which use convolutional filters to identify spatial patterns in data. They are widely used for image-based applications such as face recognition, object detection, medical image analysis, handwritten character recognition and image segmentation. Convolution operations can also be applied to one-dimensional data, such as audio signals and time-series, or three-dimensional data, such as volumetric medical scans and video.
Another architecture is the Recurrent Neural Network (RNN), which is designed to process sequential data. Unlike a conventional feed-forward network, an RNN maintains information from previous inputs while processing the next input in a sequence. This makes it useful for applications such as speech recognition, language modelling, text processing, machine translation and time-series forecasting. For example, an RNN can analyze historical sensor readings to predict future values or identify unusual patterns in a sequence of network events.
Over time, specialized architectures designed for generating new content, such as images, text and audio, have become extremely popular.
My Journey with Deep Learning
I began my journey with deep learning in 2018, during my doctoral research in the area of image forensics, specifically image splicing detection.
Around 2016, CNN-based approaches to image forensics were beginning to emerge, and the field was evolving rapidly. I was closely following the work of researchers from Drexel University, particularly Owen Mayer, who was exploring camera-based techniques for image forgery detection using CNNs.
I attempted to replicate some of this work under the guidance of Dr. V. Masilamani from IIITDM Kancheepuram.
My research later moved towards pixel-based analysis, where I developed a twin CNN architecture using feature-transfer learning for image splicing detection. The resulting model, called MissMarple, explored how features learnt from coarsely spliced image regions could help detect more subtle and visually imperceptible splicing forgeries. You can read the detailed research paper here: MissMarple: A Novel Socio-inspired Feature-transfer Learning Deep Network for Image Splicing Detection.
What I found interesting was how the machine learned to extract useful features and develop meaningful representations to address the problem at hand. But how much of this learning can we truly explain? Can machines go beyond making predictions and learning features?
(Note: Images used in this blog post were generated with the help of ChatGPT.)