Recurrent Neural Networks

Artificial neural networks give computers the ability to solve complex problems and make intelligent decisions in a way that very loosely resembles how our human brains work. These networks are key to the advanced deep learning capabilities that are revolutionizing fields like language processing and data forecasting, but one type specifically excels in this area.

Recurrent neural networks (RNNs) are a type of artificial neural network designed to process sequential data by using an internal memory to recall previous information.

What does this mean, exactly? This type of neural network is ideal for processing data that occurs in a specific order, such as words, sentences, or information organized by time intervals like financial data or weather information.¹ A key characteristic of RRNs is their ability to remember previous information when processing current information, using a built-in memory to make future predictions.

RNNs achieve this through the use of a hidden state, which serves as a memory bank that retains information from previous data points, or time steps, in a sequence of data. At each time step, the RNN modifies its hidden state to blend the current input with previous information, then generates an output which is carried forward to the next time step, and so on.

This unique ability sets RNNs apart from traditional neural networks, called feedforward neural networks (FNNs). FNNs do not have a hidden state. They process data in only one direction — from input to output — without cycling back over previous information.² This makes them better for tasks where the order or context of the data is irrelevant. In handwriting recognition, for example, FNNs only need to identify the independent features of each character and not the sequence of strokes.

On the other hand, RNNs have a loop that allows information to be passed along as the system processes data.² In this way, the models are self-looping or recurrent. This is essential for generating words or sentences — where the order and context of the data do matter.

Imagine you’re telling your friend a story. For the end of the story to make sense, your friend has to remember important details from earlier parts of the story. Your friend may even be able to predict the end of your story based on what you’ve told them so far. An RNN works just like this, remembering the information it has received and using this information to understand and predict what’s coming next.

This makes RNNs well-suited for natural language processing (NLP), natural language generation (NLG), speech recognition, machine translation, autocomplete, and predictive text applications. For example, Google Translate uses RNNs to translate text and Apple’s Siri uses RNNs to recognize and generate speech.³

While RNNs do not fully mimic the complexity of our brains, they are great at identifying patterns, understanding context, and remembering sequences of events — just like us. This suggests that RNNs may be able to provide insight into our own cognitive processes. RNNs can even model human decision-making processes to help us learn about our behavior.

Beyond this, RNNs power many of the technologies you interact with daily and continue to find a place in emerging systems like self-driving cars, smart home devices, fraud detection, credit scoring, customer behavior analysis, healthcare research, and much more. We’ll discuss some of these interesting applications below and address the ever-present challenge of ensuring these tools are used ethically and responsibly.