What to Expect When You're Diagnosing

read time - icon

0 min read

Jun 28, 2024

In part one of our series on behavioral diagnoses for student success initiatives, we explored the importance of leveraging the data that you, as an education administrator or instructor, already have at your university to improve academic outcomes. Viewing existing data through a behavioral science lens can provide you with insights you can use to help you tailor your student success strategies and design more successful interventions.

Although analyzing your initial data is a great starting point, it unfortunately doesn’t mean it’ll be sufficient to work with—in fact, it might just incite the need for more investigation. Think of the world of medicine: to fully understand how healthy you are, you may start by looking at whether you get enough exercise or sleep, but, at times, you may also need to run a series of blood tests or get an X-ray to have a more in-depth look at what is going on inside your body. In the case of education, you can also get a more in-depth X-ray of your student population by conducting a behavioral diagnosis from scratch. 

While conducting a behavioral diagnosis from the ground up is by no means a small feat, it can help you design student success initiatives with an even more profound impact. 

Here’s what you can expect when you’re diagnosing!

Executing a Behavioral Diagnosis: What, How, When, Who, and Why?

Although universities gather a lot of data on students, it is often not inherently collected using a behavioral science perspective. Looking at this kind of data can tell you that students engage in particular behaviors (such as procrastinating on assignments). Interpreting this data using a behavioral lens can help you plan targeted interventions (such as having students use precommitment strategies), often based on patterns in the data and existing research on the topic. However, since the data was not originally gathered using this lens, it often fails to provide insights into why students behave in a particular way.

In a behavioral diagnosis, data gathering is purposefully designed.

What does it mean to purposefully design data gathering? It means that you need to have a deliberate plan guiding your data collection initiatives. This entails careful consideration of a series of factors that help you guarantee that the data you collect is not only accurate but also relevant and useful when it comes to understanding and addressing the behaviors you are studying. 

When executing a behavioral diagnosis from the ground up, data must be purposefully collected. This means that 

  • what you ask
  • how you ask it, 
  • when you ask it, and
  • who you ask it to

all flow naturally from why you’re asking it from a behavioral science perspective. 

In practice, this means:

  • What you ask: Choose the questions you want to ask having a clear objective in mind. Your questions should be designed to gather the information that directly relates to the behaviors you are looking to understand and the underlying reasons for those behaviors.
  • How you ask it: Make sure you think through how you are asking your questions. How are you wording it (do you ask it as a question or a statement)? How are you formatting it (is it open-ended, a Likert scale, ranking, forced-choice, or something else entirely)? In what context are you asking these questions (is it an online survey or an in-person interview)? Determining how to ask a question is important to ensure that respondents can give you an honest and accurate response.
  • When you ask it: Timing is crucial! You should collect data at the moments that are most likely to provide you with insightful and relevant information. For example, asking at the beginning of the semester versus right in the middle of finals week might give you very different responses.
  • Who you ask: You can’t interact with every single person at your university, right? (You can try, but I am pretty sure you won’t succeed!) Will you talk to professors or students? If students, will you talk to all students or students from a particular educational level, major, or those with specific characteristics (such as those with a GPA under 3)? Selecting the right people to talk to is key in making sure you gather insights that are relevant to the issue at hand!
  • Why you ask it: Always ask yourself, “What is the objective?” Each question should have a clear purpose related to digging deeper into the behaviors you are trying to understand. This helps ensure that you don’t just collect random, “interesting” data, but data that is actually targeted and helps inform diagnosis and the interventions you may execute. (This is the same in the medical world! For instance, just because it is interesting to know what my cholesterol reading is, that might not help me understand what to do about a broken wrist!)

The exact details of how to run a behavioral diagnosis vary depending on many factors. Organizational needs, context, and resources, among others, all come into play. However, as a general, high-level outline, here are the steps you can anticipate when you are diagnosing…

Step One: Determining What You Want to Achieve

The first step is determining what you want to achieve. What challenges are you trying to tackle? What is the gap you are trying to fill? For example, you may have noticed high attrition rates. Or, perhaps, the teachers in your university report that students seem unmotivated and disengaged. Maybe you’ve recently noticed that more and more students are “falling through the cracks” in the system. A solid understanding of the “problem,” “challenge,” or “gap” helps you inform all other steps of your diagnosis.

When thinking about the “challenge,” it is often common to find that each stakeholder within the organization has a slightly different perspective.​​​​ For instance, while a teacher might feel that the school resources and materials available to students may need updating, administrators may be more focused on budget constraints and resource allocation. 

So, how do you go about determining a single challenge to tackle that everyone can get behind? Methods like interviews, brainstorming sessions, or even alignment sessions are common at this stage. Perhaps holding a brainstorming session with teachers to understand how and why they believe resources need to be updated may prove useful. Or it may be more helpful to interview school administrators from different departments, including those in charge of budget, student resources, and student well-being, among others to get a variety of perspectives on the topic. These methods can help you navigate through the many different opinions that internal stakeholders might have in order to find common ground. 

Step Two: Reviewing What is Already Known

Next, you may be tempted to run interviews and a survey with your students to gather information about them. Great idea… but stop! This is where things get tricky! Before jumping into things, don’t forget to incorporate behavioral science principles into the interview and survey design! Lest you run the risk of having one more dataset that you have to try to leverage but have a hard time doing since it is not designed through this perspective!

For a solid, evidence-based behavioral diagnosis, after you have identified the challenge, it is important to engage with scientific articles or reports, and perhaps even some international NGO websites, that already address the topic. Understanding what research in fields like psychology, decision-making, neuroscience, and education have to say about the issue can guide your decision on the core topics and areas the rest of your process will focus on. 

Looking at the literature from a behavioral perspective will help you sift through the gazillion results you may get when searching Google Scholar for articles that may be of interest to you (a quick, fairly specific Google Scholar search for student succes+attrition+universtiy limited to resources published after 2020 renders a whopping 17,000 results!). This “filter” will help you hone into areas to target in a feasible and manageable way through interventions. In fact, you may even use this stage to start developing some initial hypotheses that you can further dig into during the next stages!

Step Three: Digging Deeper with a Mixed Methods Approach

After delving into the literature, ideally, you would engage your students using a series of methods that will provide you with mixed (qualitative and quantitative) data about the problem. Interviews and focus groups, for example, may be a great way to obtain some deep qualitative insights about the topics you have decided to focus on. Meanwhile, for a broader look at student emotions, beliefs, and behaviors as they relate to the challenge you are tackling, you may decide to run a large-scale quantitative survey across your university. 

In both cases, the instruments you use should be designed using a behavioral perspective that tries to elicit responses not only about what the students’ behavior is, but also how and why it is occurring. The questions you would ask here, not only in interviews but also in surveys, should be planned through considering particular behavioral hypotheses. For example, you may want to ask about barriers that individuals face when trying to engage with an intervention that you already use. Alternatively, you may be looking to understand more about why students engage in particular behaviors or what they think about a specific aspect of their educational experience.

While you may start off by gathering qualitative data and use it to help inform and give shape to your quantitative instrument, or start with quantitative data gathering to then use the qualitative data to dig deeper into some of the patterns, the important thing about this stage is better understanding the students themselves: their perspectives, behaviors, motivations, barriers, drivers, and other behavioral insights. Unfortunately, just finding out more information about their attendance rates, gender, age, or other demographic data won’t cut it!

Step Four: Analyzing the Data

Finally, you should analyze all of your new data using a behavioral lens. In this step, you would consider the behavioral and cognitive patterns that may influence student well-being, academic performance, and achievement, among other things. You should usually seek to identify the connection between the what, the how, the when, the who, and, most importantly, the why of student behavior. Hypothetically, you might expect to see the following:

  • What: Are my students performing as expected? No, my students are underperforming academically.
  • How: How are students approaching challenges in the classroom? They are approaching them as obstacles, rather than growth opportunities. 
  • When: When are students challenged most in the classroom? During midterms and finals!
  • Who: Who is impacted the most by these challenges? Students who receive financial aid. 
  • Why: Why are students approaching challenges this way? They come from a sociocultural context that has taught them to fear failure and, consequently, they fear challenges that may lead to failure!

Step Five: Translating It!

“Ok, great, so now I have a lot of data, and I analyzed it using a behavioral lens. What now? I want to know what I can do with it!”

This, my friends, is the stage where the art and science of behavioral science translation comes into play.

Data analytics is all fine and dandy, but if you cannot translate this into actionable interventions, what is the use? Just as in medicine, we would hope our doctor does not give us a diagnosis and leaves us “hanging” (“You have strep throat. See ya!”), when it comes to a behavioral diagnosis, we want to know what we can do about it

In education, a behavioral diagnosis helps you understand the what, how, when, who, and why of students’ behaviors. Translation, on the other hand, helps you move from insights about students to effective interventions to improve student outcomes. 

How does this translation process work? Unfortunately, I'm going to leave you hanging on that one... until next time!

About the Author

Dr. Cynthia Borja

Dr. Cynthia Borja

Cynthia is an Associate Project Leader at The Decision Lab. She holds a doctorate in Psychology from Capella University, a Master’s in Psychology from Boston University, and a Bachelor’s in Neuroscience and Behavior from Vassar College. Her mission is to promote the application of the principles of brain, behavioral, and learning sciences to the real world.

About us

We are the leading applied research & innovation consultancy

Our insights are leveraged by the most ambitious organizations

Image

I was blown away with their application and translation of behavioral science into practice. They took a very complex ecosystem and created a series of interventions using an innovative mix of the latest research and creative client co-creation. I was so impressed at the final product they created, which was hugely comprehensive despite the large scope of the client being of the world's most far-reaching and best known consumer brands. I'm excited to see what we can create together in the future.

Heather McKee

BEHAVIORAL SCIENTIST

GLOBAL COFFEEHOUSE CHAIN PROJECT

OUR CLIENT SUCCESS

$0M

Annual Revenue Increase

By launching a behavioral science practice at the core of the organization, we helped one of the largest insurers in North America realize $30M increase in annual revenue.

0%

Increase in Monthly Users

By redesigning North America's first national digital platform for mental health, we achieved a 52% lift in monthly users and an 83% improvement on clinical assessment.

0%

Reduction In Design Time

By designing a new process and getting buy-in from the C-Suite team, we helped one of the largest smartphone manufacturers in the world reduce software design time by 75%.

0%

Reduction in Client Drop-Off

By implementing targeted nudges based on proactive interventions, we reduced drop-off rates for 450,000 clients belonging to USA's oldest debt consolidation organizations by 46%

Read Next

A group of people in a modern meeting room, some seated and working at a large table with laptops and cameras, while others stand and converse. The atmosphere is casual, with natural light filtering through large windows in the background.
Insight

Why Teams Make Bad Decisions

Sometimes, the best way to avoid group decision-making failures is not to make decisions as a group at all.

Notes illustration

Eager to learn about how behavioral science can help your organization?