Unsupervised Learning

When Sir Isaac Newton discovered the principle of gravity, he wasn’t experimenting in a laboratory with close supervision from his professor. Rather, he was sitting alone, unsupervised, under a tree when he happened to observe an apple suddenly fall to the ground. Newton carefully observed the plummeting fruit and, through his own curiosity and creativity, formulated the laws of motion and universal gravitation.

Sometimes we learn best through processes of exploration, experimentation, and self-directed inquiry. As children, we are encouraged to play without adult guidance to develop crucial problem-solving skills, creativity, and independence of thought (and because parents don’t have unlimited time for play). Later in life, this autonomous learning can lead to significant discoveries and innovations, just like Newton and gravity. 

When computers are left to their own devices, they can be pretty creative too.

Unsupervised learning is a type of machine learning that does not require human supervision. The input data does not have labels and so the goal is for the model to identify patterns, structures, and relationships within the data without explicit instructions on what to look for. In other types of machine learning, such as supervised learning, the model is trained on a dataset which contains input-output pairs that have been labeled by a human. 

You may be thinking, why are there different types of machine learning? Well, some types of machine learning are better suited to certain tasks than others. Unsupervised learning algorithms are excellent at handling complex processing tasks, such as organizing large datasets into clusters. They are also very effective at uncovering hidden patterns in data and can identify key features that help categorize information.¹ In fact, unsupervised learning can discover patterns that may not be immediately obvious to humans.

Machine Learning: A subfield of AI that focuses on the development of algorithms and models that allow computers to learn from and make predictions or decisions based on data, without being explicitly programmed to do so. The term was first coined in 1959 by Arthur Samuel, an IBM employee and pioneer in the field of computer gaming and AI.   

Deep learning: A subset of machine learning that involves the use of neural networks with many layers (hence "deep") to model and understand complex patterns in data. It enables computers to perform tasks such as image and speech recognition, natural language processing, and autonomous driving by learning from large amounts of data.

Unsupervised Learning: A type of machine learning where the model is trained on data that has no labels or predefined outcomes. The goal is to identify patterns, structures, or relationships in the data without any prior knowledge of what those patterns might be.

Reinforcement Learning: A type of machine learning where an agent learns to make decisions by interacting with an environment. The agent takes action, observes the outcomes, and receives rewards or penalties. The goal is to learn a policy that maximizes cumulative rewards over time.

Anomaly detection: The process of identifying rare items, events, or observations which raise suspicions by differing significantly from the majority of the data.

Clustering: A type of unsupervised learning technique used to group a set of data points into distinct clusters based on their similarities. Each cluster contains data points that are more similar to each other than to those in other clusters, helping to identify patterns, structures, and relationships within the data.

Association: A type of pattern discovery where the goal is to identify relationships or associations between items in a dataset. This approach is particularly useful for analyzing a dataset where items are frequently bought, used, or occur together.

Dimensionality Reduction: A process of simplifying data that has many attributes (or features) so that it becomes easier to work with. The random variables are reduced to a set of principal variables.

Artificial neural network: A computational model loosely inspired by the way biological neural networks in the human brain process information. It consists of interconnected units called neurons or nodes, organized into layers: an input layer, one or more hidden layers, and an output layer.

Large-scale convolutional neural network: An advanced type of deep learning model designed to process data with a grid-like topology, such as images.

Google describes unsupervised learning as “the curious pupil,”² referring to the model’s similarity with the way young children learn autonomously through creativity and exploration. Moving away from the belief that rigorously training algorithms can achieve artificial general intelligence to complete a wide variety of tasks, many scientists, such as Turing Award winners Yann LeCun and Yoshua Bengio, believe that the key to human-level machine intelligence is independent learning strategies.³ That is, When the limits of learning are not defined by human trainers, something rather magical can occur. 

Unsupervised (and supervised) learning can lead to a subset of machine learning called deep learning. If machine learning enables computers to learn from data without being explicitly programmed for specific tasks, then deep learning takes this a step further. Deep learning employs a subset of machine learning algorithms that utilize layers of artificial neural networks designed to mimic aspects of human brain function.

Artificial neural networks are computational models inspired by the human brain's interconnected neural cells. They consist of layers of nodes, or 'neurons,' that process information and can learn complex patterns in data. 

In 2012, a breakthrough occurred in the realm of deep learning when a large-scale convolutional neural network called AlexNet (named after its lead architect Alex Krizhnevsky) dominated a contest called the ImageNet classification competition. 

The algorithm was highly successful at detecting objects and classifying images on a large scale, but what was most impressive was how it did this. AlexNet could interpret images by building increasingly complex internal representations of its features. On the bottom layers, AlexNet placed low-level features such as textures and edges which it then combined at the top level to produce high-level concepts such as wheels or dogs.  

This is similar to how we process information in our brains; we first process edges and textures in primary sensory areas before working up to more complex objects like faces in secondary areas. That is, we don’t immediately perceive images in all their complexity. 

Yann LeCun: Renowned French-American computer scientist and a pioneer in the field of artificial intelligence, particularly known for his contributions to deep learning and computer vision. LeCun is the Chief AI Scientist at Meta. 

 Yoshua Bengio: A prominent Canadian computer scientist and one of the leading figures in artificial intelligence (AI), especially known for his contributions to deep learning. Bengio is the founder and scientific director of the Montreal Institute for Learning Algorithms (MILA), one of the world’s leading research centers for deep learning.

Alan Turing: British mathematician, logician, cryptanalyst, and computer scientist who played a foundational role in the development of modern computing and artificial intelligence. He is best known for the concept of the Turing Test and his work during World War II at Bletchley Park where he led efforts to decrypt German codes. 

Alex Krizhnevsky: Canadian computer scientist renowned for his work on artificial neural networks and deep learning. Along with his colleagues at the University of Toronto, Krizhnevsky designed AlexNet, a powerful visual-recognition neural network hailed as a breakthrough for AI.

Arguably the most important feature of unsupervised learning is the element of discovery. While the goal of supervised learning is to use already known data to make predictions about new data, the objective of unsupervised learning is to actually explore that original data and gain new insights.⁵ The machine learning itself determines what is different or interesting from the data set. 

Like children, unsupervised machine learning models are also quite adaptable. They can adjust to new data without needing to be retrained, just like a toddler can quickly adapt to new toys or stimuli. In fields such as cybersecurity or fraud detection, this enables new patterns and anomalies to be detected efficiently, potentially saving a lot of time and money. 

There’s also the issue of money. Compared to supervised learning, which requires a human expert to label the datasets from which the algorithm will learn, the training datasets for unsupervised learning don’t need manual annotation by a human expert. While the general rule is that supervised learning costs more, the computational requirements and difficulty interpreting the results in unsupervised learning can also lead to significant costs.

With the benefits of unsupervised learning come several drawbacks. While the approach may save human experts thousands of hours labeling training data before it is fed into the algorithm, unsupervised learning models are computationally complex and need powerful tools for working with large amounts of unclassified data. Where you save on human costs, you end up spending on computer power. 

Unsupervised learning, despite its name, is not entirely devoid of human oversight. The term 'unsupervised' primarily means that the learning process does not utilize labeled data; however, this does not guarantee that the outcomes are always accurate or useful. In fact, unsupervised models can sometimes produce highly inaccurate or misleading results due to their autonomous nature. For instance, if the data contains noise or irrelevant features, the model might incorporate these into its analysis, leading to erroneous pattern recognition or biased insights. Therefore, expert validation is crucial. Data scientists must rigorously analyze and interpret the outcomes to ensure that any patterns or structures identified by the model are both statistically significant and contextually relevant. This process often involves sophisticated statistical tools and a deep understanding of the domain to distinguish between genuine insights and spurious correlations.

One of the main challenges with unsupervised learning models lies in their interpretability. These models are designed to detect patterns without any predefined notions of what to look for, which can sometimes lead to the discovery of patterns that, while statistically apparent, are not meaningful or useful from a human perspective. For example, a clustering algorithm might group customers into segments based on purchasing behavior that, upon further inspection, do not correspond to any actionable marketing strategy. The inherent complexity of these models, especially when dealing with high-dimensional data, can further obscure understanding, as the relationships they uncover might be non-intuitive or too abstract to be readily explained. This difficulty not only hampers the ability to make informed decisions based on the model's output but also complicates efforts to debug or refine the model.

Finally, some argue that unsupervised learning needs to be subjected to greater philosophical scrutiny in the same way supervised and reinforcement learning algorithms have been.⁶ Yet despite these challenges and debates, unsupervised learning is largely regarded as the way forward for AI. 

Bank Account Fraud

Have you ever received a text message or a phone call from your bank’s fraud department advising you of suspicious activity on your account? The warning probably arrived before you even noticed there was something odd going on. 

Nowadays, banks use a type of unsupervised learning called ‘anomaly detection’ to identify unusual or fraudulent activities in financial transactions by identifying patterns and behaviors that don’t fit the norm.7 Unsupervised learning can actually analyze whole networks of applications to uncover hidden connections that may appear genuine when viewed in isolation.⁸

Let’s say, for example, that during the week your daily transactions are usually limited to a coffee at the café near your work, the odd meal out with friends in the evening, and the weekly car gas refill and supermarket shop. Suddenly, a transaction is attempted at a store hundreds of miles away from your work and home for an eye-watering amount of money. The transaction deviates wildly from your average weekly spend and given your earlier transactions in the day, it would be impossible for you to have been in two places at the same time. 

One of the ongoing challenges of applying machine learning to financial fraud is that the number of fraud samples is much smaller than the number of transaction samples.⁹ This imbalance has a number of implications for the accuracy of the model, including the bias towards the majority class (non-fraudulent transactions), difficulty accurately defining what constitutes an anomaly (fraudulent transaction), a lack of patterns or behaviors for the model to learn from, and difficulty evaluating the performance of the model. As banking systems move online and the threat from fraud and cybercrime increases, researchers are continually working to improve the AI systems that keep customers and their money safe. 

Google News Classifier

When the beta version of Google News was launched back in 2002, news articles were simply categorized so that readers would receive a range of sources on the same topic.10 Back then, this task was largely performed by humans with some help from machine learning techniques. Fast forward 15 years and Google News received a major update: AI. Google’s latest “news classifier” now uses unsupervised learning to do the categorizing, using simple labels such as “US” for results of a presidential election, for example.¹¹ The biggest difference between the old Google News and the new version is that the algorithms analyze the information in real-time as it hits the internet and organize it into storylines. 

Google News is an aggregator service that uses a data mining technique called ‘clustering’ to group unlabeled, raw data based on their similarities and differences. According to Google, applying the latest AI techniques to categorize the news enables them to synthesize information and put it together in a way that helps people make sense of what’s happening, and what the impact or reaction has been. Importantly, the machine learning algorithms are able to determine a user’s interests based on their previous engagement and use this information to adjust the app’s content.¹²

Algorithms for Simpler Decision-Making (1/2): The Case for Cognitive Prosthetics

Deep learning represents the outsourcing of human cognitive functions to computational algorithms. Interestingly, the more we delegate our information gathering and decision-making to these algorithms, we lose our cognitive autonomy and restrict our thinking to what the algorithm deems appropriate. This article takes a closer look at our relationship with algorithms and how they are shaping the way we think and act. 

Artificial General Intelligence

The hypothetical AI with human-like cognitive abilities, Artificial General Intelligence (AGI) is for some a fictional dream and for others, a rapidly approaching reality. The machine learning we have today still has a long way to go before it becomes more ‘intelligent’ than us, but AGI will apparently overtake our brain power. This article explores the fascinating world of AGI and explores the possible consequences of fully autonomous AGI systems. 

1.  Google Cloud. (n.d.). What is unsupervised learning? https://cloud.google.com/discover/what-is-unsupervised-learning
2. Graves, A., & Clancy, K. (2019, June 25). Unsupervised learning: The curious pupil. Google DeepMind.https://deepmind.google/discover/blog/unsupervised-learning-the-curious-pupil/
3. Wiggers, K. (2020, May 2). Yann LeCun and Yoshua Bengio: Self-supervised learning is the key to human-level intelligence. VentureBeat. https://venturebeat.com/ai/yann-lecun-and-yoshua-bengio-self-supervised-learning-is-the-key-to-human-level-intelligence/ 
4. Wolfewicz, A. (2023, February 15). Deep Learning vs. Machine Learning – What’s the Difference? Levity.https://levity.ai/blog/difference-machine-learning-deep-learning#:~:text=Deep%20Learning%20describes%20algorithms%20that,through%20supervised%20and%20unsupervised%20learning.
5. IBM. (2021, March 12). Supervised versus unsupervised learning: What’s the difference?’ IBM. https://www.ibm.com/think/topics/supervised-vs-unsupervised-learning
6. Watson, D. (2023). On the Philosophy of Unsupervised Learning. Philosophy & Technology, 36(28). https://doi.org/10.1007/s13347-023-00635-6 
7. Sapra, Y. (2023, November 14). How Machine Learning in Banking Helps in Fraud Detection. HashStudioz. https://hashstudioz.com/blog/how-machine-learning-in-banking-helps-in-fraud-detection/ 
8. Datavisor. (n.d.). Digital Fraud Wiki. Datavisor. https://www.datavisor.com/wiki/unsupervised-machine-learning/
9. Jiang, S., Dong, R., Wang, J., & Xia, M. (2023). Credit Card Fraud Detection Based on Unsupervised Attentional Anomaly Detection Network. Systems, 11(305). https://doi.org/10.3390/systems11060305
10. Upstill, T. (2018, May 8). The new Google News: AI meets human intelligence. Google. https://blog.google/products/news/new-google-news-ai-meets-human-intelligence/
11. IBM. (n.d.). What is unsupervised learning? IBM. https://www.ibm.com/topics/unsupervised-learning
12. Tung, L. (2017, July 19). Google is using machine learning to create a news feed from your searches. ZDNet.https://www.zdnet.com/article/google-is-using-machine-learning-to-create-a-news-feed-from-your-searches/