Behind the Screens: Using AI to Dissect Social Media Persuasion

read time - icon

0 min read

Apr 08, 2024

AI's role in comprehending the complexities of human communication on social media presents unparalleled prospects for behavioral research. Our investigation focuses on how Large Language Models (LLMs) can transform our ability to recognize and analyze the persuasive strategies used by financial influencers across several platforms.

This approach to AI-driven insights demonstrates not only the integration of technology and human behavior but also the specific methodology we used to extract and classify massive volumes of content. By concentrating on the particular use of LLMs in social media listening, we highlight its crucial significance in providing a new viewpoint on how digital communication influences consumer perceptions.   

The Objective

We had the challenge of understanding the impact social media has on retail investors. A part of our research was aimed at analyzing the types of persuasive techniques influencers employ to encourage their audience to make a particular financial decision. The study focused on influencers across social media platforms including YouTube, X, TikTok, Instagram, and Reddit. This type of work usually follows two main steps:

  1. Extract the available social media content and clean the data to be analyzed.
  2. Use traditional natural language processing techniques and algorithms to classify the data and extract insights. 

Large Language Models (LLMs), such as ChatGPT and LLaMA, are algorithms that excel in classifying and analyzing text—which, in this case, are the millions of words we gathered from social media platforms. LLMs can aid in both of these steps: cleaning and classifying the data.

Behavioral Science, Democratized

We make 35,000 decisions each day, often in environments that aren’t conducive to making sound choices. 

At TDL, we work with organizations in the public and private sectors—from new startups, to governments, to established players like the Gates Foundation—to debias decision-making and create better outcomes for everyone.

More about our services

Cleaning the data

In the initial data cleaning phase, LLMs can scan through the whole dataset to pinpoint social media posts that pertain to financial advice. Traditional cleaning techniques for this type of work might revolve around matching the text to specific keywords referencing financial advice. If a post contained one or multiple words from a preselected group of keywords, we would classify such posts as containing financial advice content. One of the benefits of LLMs is they move away from preselection biases and classify the posts based on the prompt that is indicated. We opted not to utilize this specific approach in our study because we sourced our data from specific financial influencers selected for their popularity with the intent of assessing their broader communication patterns.

It is possible to feed a dataset of text to an LLM to indicate if the text contains what could be described as financial advice. Given the technical limitations of the current architecture of LLMs, we have to feed the AI with a social media post from the database and the classification prompt one by one to be classified. If done manually, this process would take a significant amount of time. Instead, we can automate this process by connecting to the LLM Application Programming Interface (API), an interface that enables us to interact directly with AI models. This connection allows for the automatic submission and labeling of social media posts, speeding up the study by processing large amounts of data quickly and effectively. 

This process allows for a better cleaning phase since context is crucial for text classification. Modern LLMs based on a specific neural network architecture called transformers are built to consider the context of the entire text.1 For example, if we use “invest” as a keyword, we could classify the following two statements as financial advice: 

  1. “Invest in crypto” 
  2. ”Musicians invest emotion” 

LLMs can take the context of the entire text and classify it correctly: “Invest in crypto” as financial advice and ”Musicians invest emotion” not as financial advice. 

This process is not exempt from errors since we have just introduced a possible data bias by relying on the AI’s training data to classify for us. Such training data could be disproportionately inclined toward certain classifications. An example of this bias could occur if the LLM was disproportionately trained on a tech-centric dataset. The LLM might be more adept at classifying texts related to technology investments as financial advice while also misclassifying statements about investments in other sectors, such as agriculture or education.

This is a known limitation and AI companies are constantly trying to minimize this risk.

Classifying the data

LLM application is also useful in the second part of natural language processing: classifying data and extracting insights. The behavioral application for this particular case is that we can prompt the LLM to identify and classify specific persuasive communication techniques within the data. Using AI in this manner, we can loop over the millions of words containing financial advice from social media and quantify the persuasive techniques such as authority, scarcity, and social proof used by influencers. 

We apply the same methodology to classify the construal of social media advice. Construal refers to the level of abstraction or specificity with which information is presented in a message. Low construal refers to thinking that focuses on concrete, specific details and immediate actions. Meanwhile, high construal involves abstract, general principles and long-term goals. Here are two hypothetical examples to help you get a better idea of construal in this context:

  1. Low-construal: "Start by setting aside $200 each month into a low-cost index fund. Use automatic transfers from your checking to your investment account on the first of each month to ensure consistency."
  2. High-construal: "Focus on building a diverse investment portfolio to secure your financial future. Prioritize investments that align with your long-term financial goals and risk tolerance."

Low-construal messages on social media are viewed as more credible, reducing the perceived riskiness of the information within, resulting in an increased potential for persuasion.2 The findings of our research are yet to be publicly available.

Final Considerations

Some limitations of this methodology are the training biases of LLMs as previously mentioned, as well as the necessity of connecting to an AI model available through an API. This means that you must share your data with a third party—and in some cases, when the text to be analyzed contains sensitive information, this approach is not viable. It is also important to mention that each call to the API will have a small cost depending on the length of the text and the model being used. The cost varies depending on the provider (in our case, the cost per post classified was a fraction of a cent) but can start to be significant with bigger datasets and large numbers of classifications.   

In summary, we found a way for AI to classify social media posts based on the persuasive techniques used by influencers and the level of construal in a more streamlined and precise manner than traditional methodologies. This allowed us to analyze behavior at a great scale within a digital environment. The specific findings from the mentioned study will be shared later this year with the complete report published by one of our clients. 

This is just one application of how AI can be used to solve challenges using behavioral science. If you are interested, reach out to discuss how we can tailor these strategies to achieve success in your field.

References

  1.  Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17).
  2. Balaji, M. S., Jiang, Y., & Jha, S. (2021). Nanoinfluencer marketing: How message features affect credibility and behavioral intentions. Journal of Business Research, 136, 293-304.

About the Author

Jerónimo Kanahuati

Jero is a Consultant at The Decision Lab with a passion for artificial intelligence and behavioral science. Prior to joining The Decision Lab he founded a startup in Mexico to develop apps for kids to encourage education, and developing web scraping bots. He also worked at Google as an account manager and technical specialist focused on ad placement across Google's products. Jero has a bachelor's degree in engineering and a postgraduate specialty degree in operations from Universidad Panamericana in Mexico City. 

Read Next

blue photo of romans
Insight

Taking a Hard Look at Democracy

Tom Spiegler, Co-Founder and Managing Director at The Decision Lab, joins Nathan Collett to talk about what behavioral science can tell us about the 2020 US election and the state of democracy more generally.

Insight

A New SPIN on Misinformation

We all stretch the truth from time to time. The real problem is when our lies spread to thousands of people, assisted by recent technological advancements such as social media or artificial intelligence. This has a real impact on the decisions people make—such as who to vote for or whether to get vaccinated.

Notes illustration

Eager to learn about how behavioral science can help your organization?