Active Learning

Although at first glance it may appear as though artificial intelligence (AI) automatically knows everything, this couldn’t be further from the truth. Just like humans, AI goes through a rigorous learning process known as machine learning. Yes, computers have to do homework too. 

There are three different types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. Active learning is a type of iterative supervised learning in which a learning algorithm can request a human user to label data with the desired outputs.

Unlike traditional supervised learning approaches, where the algorithm learns from a static dataset, active learning involves dynamically selecting instances (individual data points) from a larger pool of unlabeled data for annotation or labeling by an oracle (usually a human expert). Active learning is a type of human-in-the-loop model because it relies on human interaction and input. 

Ghassemi, F. (2023, June 26). Active Learning (ultimate guide). Medium. https://medium.com/@farnazgh73/ultimate-guide-for-active-learning-main-approaches-3cf53ce207f0

Let’s look at a simple example. Imagine you’re teaching a computer to recognize cats in pictures. Normally, you’d show the computer lots of labeled pictures of cats and non-cats so it can learn. But what if labeling those pictures is too time-consuming or expensive?

Active learning is like having a smart system that already knows which pictures would teach the computer the most. Instead of just showing random pictures, the system identifies ones it’s unsure about and asks a human to label only those. The computer then learns from those labeled examples and gets better at recognizing cats.

So, active learning helps the computer learn faster and with less effort by focusing on the most helpful examples for learning, similar to a tutor who knows exactly which questions to ask to help a student understand better.

The fundamental principle behind active learning is that the machine learning algorithm learns better and with less training when it gets to choose the data from which it wants to learn. In other words, by being active, curious, and exploratory.

It’s a bit like a child independently deciding which books or websites they explore in learning about the planets and outer space. Not all educational resources are useful or desirable to a child; an adult encyclopedia is useless to a pre-schooler who prefers to learn from an interactive app. The same idea can be applied to active learning; not all data points are equally important for training a model.  

In fact, active learning in AI follows the same principles as inquiry-based learning in human education. Based on American educator John Dewey’s philosophy that education begins with the curiosity of the learner,¹ inquiry-based learning aims to spark inquisitiveness in the learner. The inquiry-based approach encourages students to ask questions, explore new ideas, and conduct independent research, all with the objective of enhancing engagement and information retention. Just like a child might ask their teacher questions about a new topic, active learning queries a human oracle when it encounters unknown data. 

So, what’s the end goal of all this? The main objective of both human inquiry-based learning and AI active learning is that the student—be it a nine-year-old girl or a computer—eventually has the ability to learn on their own without outside help. In other words, to produce independent learners. 

Machine Learning: A subfield of AI that focuses on the development of algorithms and models that allow computers to learn from and make predictions or decisions based on data, without being explicitly programmed to do so. The term was first coined in 1959 by Arthur Samuel, an IBM employee and pioneer in the field of computer gaming and AI.   

Supervised Learning: A type of machine learning where the model is trained on a labeled dataset, meaning each input data point is paired with its corresponding output. The algorithm learns to map input data to the correct output by generalizing patterns from the labeled examples it has been provided. The goal of supervised learning is to enable the model to make accurate predictions or classifications on new, unseen data based on the patterns it has learned during training.

Human-in-the-loop model: A system or process where human intervention is incorporated into an automated process or algorithm. This setup is often used in various fields such as machine learning, artificial intelligence, and data analysis, where human judgment, intuition, or expertise is needed to handle complex or ambiguous situations that automated systems may struggle with.

Instances: In machine learning, instances refer to individual examples or data points used to train or test a model. Each instance typically represents a single observation or sample from the dataset.

Oracle: In machine learning, an oracle is a theoretical concept used to describe an idealized system or entity that provides the "correct" answers or labels for any given input. Usually, the oracle is a human. 

Unlabeled data: Pieces of data that have not been tagged with identifying characteristics, properties, or classifications. 

Edge Case: In AI, an edge case refers to a situation or scenario that lies at the extreme or boundary of what the system or model is designed to handle. These cases often deviate significantly from the norm or the typical data that the AI system has been trained on, posing challenges for accurate prediction or decision-making.

The concept of machine learning has been around for a surprisingly long time—since the 1950s, in fact. The first artificial neural network was created in the “Turing Test,” an imitation game built by British mathematician and computer scientist Alan Turing to test a machine’s ability to exhibit intelligent behavior similar to that of a human. It was around this time that the terms artificial intelligence and machine learning were first conceived. 

Since Turing’s breakthrough, researchers have been scratching their heads finding ways to teach machines to be more like humans. But to be more human, shouldn’t a computer start by learning like a human?  

In recent years, the world of AI has grown increasingly inspired by the flexible, iterative, and sometimes messy learning we see in children. Traditionally, artificial systems learn from carefully pre-curated data sets or gain large amounts of experience from simple, limited environments. However, recent efforts to unite developmental psychology with AI seek to replicate human learning processes in machine learning²—that is, learning through asking questions. 

Nick Haber, a professor at Stanford’s Graduate School of Education, argues that while recent successes in the world of AI during the last decade—such as deep learning and deep reinforcement learning—reflect many aspects of human learning, there are two critical limitations: reliance on supervision (labelling and dependence on rewards) and passive learning (computer simply learns a curated data set).³

If we take a look at inquiry-based learning in children again, we find that the emphasis is on action and minimal supervision—the teacher is only there when the child needs some guidance or reassurance. For AI to truly mimic human behavior and achieve the independence of thought that we see in humans, machine learning needs to embrace these important principles. 

John Dewey: American philosopher, psychologist, and educational reformer. He is best known for his work in pragmatism, emphasizing the importance of experience and experimentation in learning, and for his progressive views on education, advocating for a more student-centered and interactive approach to schooling.

Alan Turing: British mathematician, logician, cryptanalyst, and computer scientist who played a foundational role in the development of modern computing and artificial intelligence. He is best known for the concept of the Turing Test and his work during World War II at Bletchley Park where he led efforts to decrypt German codes.

Let’s start from a psychological perspective. It’s well known that psychological curiosity plays a significant role in human intelligence to enhance learning through exploration, experimentation, and information acquisition. Inspired by human cognitive development, artificial curiosity provides a natural intrinsic motivation for efficient machine learning. As researchers at Singapore’s Nanyang Technological University argue, incorporating curiosity mechanisms into machine learning is necessary to bridge the gap between human-based intelligence and machine-based intelligence in order to boost learning and efficiency in AI.⁴

Now on to practical matters. Most supervised machine learning systems need to be trained on many hundreds or thousands of labelled data instances in order for them to perform well. However, one of main problems in machine learning today is the vast quantity of unlabeled data available in our data-drenched society. As data continuously gets cheaper and easier to collect and store, data scientists are left grappling with more data than they are capable of analyzing and labelling. Getting a learning algorithm to help with the work improves efficiency and brings down costs:   

1. Efficient use of labeled data: Labeling data for machine learning models can be time-consuming and expensive. Active learning algorithms substantially reduce training steps by strategically selecting the most informative data points for annotation, maximizing the learning potential of each labeled example, and reducing the overall annotation effort required.⁵
2. Reduced data annotation costs: By prioritizing the labeling of the most relevant data points, active learning can significantly reduce the costs associated with data annotation while still maintaining accuracy. This is particularly useful in scenarios where labeling large datasets is impractical or prohibitively expensive.⁶

Active learning can also improve the accuracy of machine learning models by selecting the most informative data points for labelling. By focusing on the data points that are most challenging or uncertain for the model, active learning can effectively address model weaknesses and improve overall performance.

Chihuahua or Muffin?

Cast your mind back to 2016. Do you remember seeing a viral meme called ‘Chihuahua or Muffin’ or even ‘Sloth or Pain au Chocolat?’⁷ An image made up of 12 alternating pictures of chihuahuas and blueberry muffins was circulated on the internet to show how similar the two can look at first glance. While the meme was a bit of fun (and arguably quite cute), it actually demonstrated one of the major challenges of machine learning models—edge cases. 

In this example, the learning model struggled to differentiate between the two objects because the eyes and nose of a chihuahua, combined with the shape of its head and color of its fur, bore a striking resemblance to a muffin.⁸ So the learning model recognizes all of them as either chihuahuas or muffins rather than distinguishing between the two. These scenarios, where the model struggles to perform as it should due to situations that lie at the very limits of its experience, are known as edge cases and can limit the potential of AI. 

The number of data points dealing with edge cases in training data is usually low, resulting in the failure of machine learning models to train on the same data. Active learning, however, can be trained to identify these edge cases so that a human can label them correctly. When they are fed back into the data pool, the edge cases are adequately represented to avoid future misinterpretations. 

Active learning for self-driving cars 

Getting active learning to pick up edge cases might not be important when distinguishing between baked goods and puppies, but it is vital when training the software behind self-driving cars. Active learning has proven to have immense business value in the field of autonomous driving by helping to improve the accuracy of predictions demanded by the near-perfect expectations we have of self-driving systems. 

Think about this for a moment. If AI can misinterpret a chihuahua for a muffin, how many potential errors do you think a self-driving car might make on the open road? One potential edge case might be the reflection of cars and pedestrians in the rear end of a metallic gas tanker truck which the system reads as real obstacles. 

Vision-focused deep learning models (the systems that drive the car) require huge amounts of training data, but choosing the ‘right’ data that captures all possible scenarios, conditions, and edge cases in a driving environment is a huge challenge. NVIDIA, an AI computing company, used active learning to improve self-driving cars detection of obstacles during the night. Applying active learning to their deep neural system resulted in 3x more precision in pedestrian detection and 4.4x more precision in bicycle detection in comparison to data selected manually.⁹

So how does active learning label these potentially dangerous edge cases? Well, by asking humans for help. Companies such as Phantom Auto, Ottopia, and Cognicept Systems are developing ways to allow autonomous driving systems to request human assistance when they encounter edge cases. 

Artificial Intelligence Models

Machine learning is a type of artificial intelligence model in which computers are are taught to think by themselves and develop their own algorithms after processing vast amounts of data. Other artificial intelligence models function in different ways, such as experience or pre-programmed algorithms inputted by a human. Read this article to explore the world of artificial intelligence models and the everyday places you might find them. 

Getting Schooled by Artificial Intelligence: How to use AI to improve classroom learning

We apply principles of human learning to enhance the way AI works. But what happens when we apply AI to the way humans learn? Educational AI is a rapidly growing area which leverages the advances in technology to improve learning outcomes. In this article, we explore the benefits and current limitations of AI in educational contexts and what the future may hold for this partnership. 

1.     Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Boston: D.C. Health and Company. 

2.     Smith, L., & Slone, L. (2017). A Developmental Approach to Machine Learning? Frontiers in Psychology, 8  https://doi.org/10.3389/fpsyg.2017.02124

3.     Haber, N. (2023). Curiosity and Interactive Learning in Artificial Systems. In Niemi, H. et al. (Eds.), AI in Learning: Designing the Future, 37-54. 

4.     Sun, C., Qian, H., & Miao, C. (2022).From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Tasks. ArXiv, abs/2201.08300.

5.     Pathak, D., Gandhi, D., & Gupta, A. (2019). Self-Supervised Exploration via Disagreement. ArXiv:cs.LG/1906.04161.

6.     Wang, K., Zhang, D., Li, Y., Zhang, R., & Lin, L. (2017). Cost-Effective Active Learning for Deep Image Classification. IEEE Transactions on Circuits and Systems for Video Technology, 27(12), 2591-2600. 

7.     Elle.com. (2016, March 18). Do You Love Animals or Are You Just Hungry? Elle. https://www.elle.com/culture/news/a34939/animals-or-food/

8.     Hand, D. (2022, February 16). Of Muffins and Machine Learning Models. Cloudera Blog. https://blog.cloudera.com/of-muffins-and-machine-learning-models/

9.     NVIDIA. (2019, December 18). Scalable Active Learning for Autonomous Driving. Medium. https://medium.com/nvidia-ai/scalable-active-learning-for-autonomous-driving-a-practical-implementation-and-a-b-test-4d315ed04b5f

10.  Haber, N. (2023). Curiosity and Interactive Learning in Artificial Systems. In Niemi, H. et al. (Eds.), AI in Learning: Designing the Future, 37-54.