The Role of Technology in Learning
Techknowlogy: Evidence as the missing ingredient to digital learning
“If the only tool you have is a hammer, it is tempting to treat everything as if it were a nail.”1 This is the Law of Instrument, often referred to as Maslow’s Hammer after the psychologist Abraham Maslow who wrote it back in 1966. It is a common cognitive bias in both the tech and education sectors but becomes particularly pronounced when the two fields intersect. Although technology can address educational challenges, we sometimes too quickly latch onto solutions that are not necessarily informed by evidence regarding the problem, the context, or even the solution itself. Technology has become the “hammer” or our one-trick solution for everything, which dilutes its potential impact.
Digital advancements like artificial intelligence and augmented or virtual reality have launched a revolution in virtual learning, reshaping traditional educational models and creating new possibilities for learners around the globe. But let’s be clear: technology has not always been good for learners. In fact, UNESCO’s new 2023 report on technology in education, subtitled “A tool on whose terms?,” highlights the numerous conditions necessary for technology to actually be beneficial (e.g. electrification, connectivity, impartial evidence on quality, being void of unnecessary and excessive use, and responsible approaches; or, to put it simply, a lot of stuff). However, when applied intentionally, the integration of digital technologies into education has proven to have the potential to bridge educational gaps, provide access to quality learning resources, and foster a more personalized learning experience. In fact, the technology doesn’t even have to be advanced to be effective – for instance, recorded lessons are helping to close the teacher quality gap in China.
A range of tools – including multimedia resources, adaptive and immersive technologies powered by artificial intelligence (AI), as well as online platforms and Learning Management Systems (LMS) – have, in many instances, democratized education by enabling learners to access a wealth of educational materials and services remotely. The flexibility afforded by virtual classrooms has been particularly crucial during global events that disrupted traditional learning environments, allowing education to persevere. Even nations lacking the infrastructure in place for virtual learning during the COVID-19 pandemic revisited their ICT policies to rethink ways they could leverage technology.
Beyond the Nudge: BeSci as a base
Scientifically and, more specifically, behaviorally informed digital technology presents a powerful opportunity to address both educational and skill gaps effectively. Adequately understanding an educational challenge (or any social challenge, at that) is intimately tied with adequately understanding the people involved. It is important that we ask ourselves: who are we serving and what are their realities?
Remember that behavioral science is not a stand-alone solution, but a path for inquiry – helping us ask different questions or reframe the problem altogether. This becomes especially important in digital learning. Take social learning platforms as an example. When these platforms are designed with social cognitive theory in mind, they can better facilitate collaborative learning and knowledge sharing, providing learners with valuable social reinforcement and support. In this case, behavioral science is not merely a tool leveraged on the platform but rather the foundation of the platform itself.
At The Decision Lab (TDL), we actively explore technological applications grounded in evidence and behavioral insights to elevate virtual learning. We showcased our ability to design assessment tools during our recent collaboration with CATALYST by Winchester College, a digital enrichment program that encourages students to deepen their education by taking electives beyond their standard curriculum. By gaining a thorough understanding of the drivers and barriers to student engagement and learning within CATALYST, we strategically implemented “light lift” assessments (requiring little student effort to complete) that provide a timely snapshot of student learning, benefiting both learners and facilitators in the process.
Behavioral Science, Democratized
We make 35,000 decisions each day, often in environments that aren’t conducive to making sound choices.
At TDL, we work with organizations in the public and private sectors—from new startups, to governments, to established players like the Gates Foundation—to debias decision-making and create better outcomes for everyone.
Innovative Virtual Teaching and Learning Strategies for Skills
The perception that online is inferior to in-person likely stems from poorly developed virtual pedagogy, antipathy born during COVID-19 lockdowns, or a general lack of familiarity with digital methods. The effectiveness of traditional schools depends on the strong social presence of teachers and students, dynamic interactions, physical activity, and natural engagement. Technology ostensibly lags in facilitating the same social presence. After all, a person in real life is more than just a video of their face. Physical appearance, gestures, authority, charisma, timbre, a brief look of derision, or a funny expression are all subtleties that could explain why students become captivated by a teacher. Recognizing this strength in physical teaching is important, but it cannot be the sole determinant as to whether online education is effective or not. Instead, the CATALYST program by Winchester College leverages three techniques that are fundamental for both in-person and online learning: rich peer interactions, well-designed activities, and engaging and applied tasks.
Good online teaching often shifts responsibility away from the classroom and instead onto the systems in place, addressing the common lament of poor learning experiences due to unengaging instructors. Rather than targeting teachers, addressing student interactions is an effective strategy for creating a successful online learning experience. Within skills-based learning, peer-to-peer interactions are the linchpin for developing communication, negotiation, and problem-solving abilities. In line with findings of how students’ arguments become more comparative and reasoned following organized conversations among peers, we observed at CATALYST that structured peer work (discussions, group tasks, challenges, academic puzzles) is a great training ground for building the above-mentioned skills and learning how to collaborate empathetically.
To be truly stimulating and engaging online, the chosen online learning platform must extend beyond basic video calling and breakout rooms – such as by incorporating elements like virtual movement and proximity chat for organic interactions. In online settings, student-centric learning is vital with a higher need for facilitating student learning rather than delivering knowledge. The online educator’s role, while still important for explaining key concepts, better serves learners when it emphasizes facilitating interaction. In such environments, student agency often deepens their inherent curiosity for knowledge..
Gamification is the application of game-like elements and principles in non-game contexts to engage and motivate people. It can be instrumental in creating educational moments for skill expression. The iterative nature of skill-building necessitates dynamic and stimulating environments to keep learners engaged (much like sports, learning an instrument, or immersive language courses abroad). More ludic and explorative learning activities are common in skills-based or online learning. Beyond the conventional use of PBL (points, badges, leaderboards), simulations of real-world scenarios, such as Model United Nations, mock trials, or role-playing, provide engaging learning experiences. These activities, coupled with competition and artificial conflict, are highly effective for skill development.
One of CATALYST’s favorites is the “unlock” mechanic, where content is gated behind task completion, ensuring equal learning opportunities and combating the passivity that can arise in online settings. This approach demands content that is stimulating and encourages active participation. In this way, well-executed online programs can be particularly effective for skills development, offering advantages like ease of meeting, resource access, and feedback.
Gamified learning should require minimal facilitator involvement, so that learning becomes student-centric and experiential, although success is dependent on student skill. Playful activities are central to online learning, fostering engagement and social presence.
The Pros of Project-based Learning
Intrinsic motivation in learning (learning because it is inherently satisfying and enjoyable, rather than being driven by external rewards or pressures) is often tied to a sense of relatedness, agency, or individual responsibility. While the causal relationship between intrinsic motivation and increased learning outcomes has been debated, it is widely understood to contribute to students' development of important exploratory skills that are crucial to the learning process. Project-based learning, particularly with real-world implications, can be highly effective. For instance, CATALYST’s Change Project encourages students to find solutions to tangible issues such as pollution, wastefulness, or poverty. Identifying a local issue offers practical problem-solving experience and more of a direct impact. Implementing solutions provides valuable hands-on experience.
Students have to learn how to evaluate the consequences of their plans, points of failure, as well as possible counterarguments to and feasibility of their solutions. It is therefore good to guide students in identifying a genuine problem in a local community, something relatable to their own experience. For instance, tackling global warming is too ambitious to gain practical experience in problem-solving, but applying a solution to reduce food waste in a local school provides a perspective for actionable ideas, access to stakeholders, and small-scale but observable impact. Putting a solution into action suddenly makes an idea real.
Virtual Skills Assessment
Traditional means of educational assessment have often prioritized evaluating knowledge while completely abandoning skill development.2 Therefore, the new emerging frameworks for assessing skills are founded on different constructions of learning by centering skills as the purpose of education. This has the significant benefit of ensuring that skills-based programs and their students’ learning are assessed more accurately.
There is a growing understanding of skills that youth should learn (e.g. collaboration, communication, and critical thinking); however, existing skill assessment tools often assume in-person instruction. This has important implications for how activities around these conceptualizations of skills are designed and the opportunities to assess students across the skill categories both within a classroom setting and a real-world context. Although in-person instruction provides opportunities for educators to have greater access to their students, advanced virtual platforms enhance our ability to collect data in unique and less interruptive ways.
TDL’s partnership with CATALYST by Winchester College aimed to rethink digital skill assessment tools. The CATALYST program offers both synchronous and asynchronous virtual learning to youth globally, over a 1-2 week period via Gather (a web-conferencing software). Gather provides a virtual room with kinaesthetic features that allow for movement via an avatar around multiple custom-made classrooms and hallways as well as proximity chat (hearing others more clearly when their avatar is nearby). The CATALYST program focuses on project-based learning and offers its students university-level topics taught in a way that aims to develop their skills across six competency areas: 1) negotiation and resolution, 2) narrative defense, 3) effective communication, 4) quantitative fluency, 5) empathetic collaboration, and 6) critical analysis and problem-solving.
The goal of the partnership was to account for the short and dense nature of the CATALYST curriculum and design a student assessment toolkit that is an easy lift, protecting the bandwidth of CATALYST facilitators and students alike. For this reason, it needed to comprise innovative approaches to the delivery, the data inputs that are assessed, and the individuals involved in the assessment process to arrive at a triangulated story of student progress that leverages the various online platforms that CATALYST uses.
Together, TDL and the CATALYST team developed an assessment tool that is based on the drivers and barriers of student engagement and learning as assessed via existing literature, CATALYST staff, and CATALYST alumni. The assessment process was designed to span the entire program timeline, incorporating a final check-in six months post-program to evaluate practical application of learnings.
Employing diffused delivery, students responded to brief questions throughout their day, promoting continuous engagement. The approach empowered students through self-assessment and reflection, fostering ownership of their learning. Timely feedback, facilitated by this method during the two-week program, provided opportunities for improvement while the facilitators were still present. The assessment relied on a multifaceted rubric, incorporating self-assessments, observations of project work, and submitted outputs of that work. Leveraging existing platforms, including AI for rubric translation, enhanced the efficiency and effectiveness of the assessment process.
These methods effectively capture the research reality and can inform future improvements in teaching methods. Removing barriers to engagement with assessments and framing them in a motivating manner resulted in nearly 90% completion rates for even optional self-assessments. The use of triangulated assessments not only enhanced student engagement but also instilled confidence in facilitators regarding student transcripts, underscoring the significance of behaviorally informed initiatives for the entire learning community.
Taking Your Skills-Based Program Online
So how can you ensure that your applications of technology in your virtual classroom are evidence-based? How can you leverage behavioral science to enhance your online delivery? As you navigate the ever-evolving landscape of digital education, we leave you with these few words to better harness the potential of skill-building online:
- Lead with peer-to-peer learning: take time to understand your target learners and their realities, then unlock collaborative insights in a way that is most relevant to them to deepen their subject understanding.
- Take time to craft evolved and well-researched project-based learning experiences that push your students to bring their physical experiences into the digital classroom. After all, hands-on experiences ignite creativity and the practical application of acquired skills.
- Advocate for triangulated formative assessments: providing a holistic view of learners' progress is always important and particularly so with skill-building, where the learning objective is very applied.
- Seize unique opportunities that technology creates: we are living in a time where technology affords experiences that in-person learning cannot (e.g. accessibility for differently abled learners) so be sure to take full advantage!
1. Maslow, Abraham Harold (1996). The Psychology of Science: A Reconnaissance. Harper & Row.
2. Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of Educational Objectives; The Classification of Educational Goals by a Committee of College and University Examiners. Longmans, Green.
About the Authors
Justin Pinnells, the Academic Director at CATALYST and former Head of Languages at Winchester College, is a formally trained languages teacher who grew up in Germany. His passions encompass educational technology, online learning, and empowering young minds to be changemakers. With a love for culture, music, and science fiction, Justin keenly follows the evolving AI landscape, anticipating a revolution in personalized learning and innovative educational technologies. His 10-year teaching career reflects his commitment to transforming education. Justin aims to break the mould of traditional learning by fostering environments where students develop empathy, conflict resolution skills, and the courage for radical candour.
Maraki is an Education Consultant at The Decision Lab. Her research focuses on social and spatial equity in education globally, and has been featured in peer-reviewed journals, edited volumes, and international conferences. Maraki has worked with several international organizations to craft pathways to empower underserved school-aged children and youth in Africa, including UNESCO, the World Bank, the Institute of International Education, and Geneva Global Inc.