On O-rings and Online Learning

Key Points

  • There are many different components that must come together to create high-quality online learning.
  • Overall quality can be modeled as the product of the quality of each of these components.
  • Huge gains in overall quality can be achieved through incremental improvements to the individual components.
  • Major breakthroughs in online learning could happen through this compounding incremental improvement process, without a need for any new inventions or proprietary technology.

An Introduction to “O-ring Theory”

The tragic explosion of the space shuttle Challenger was a lesson in the ways in which complex human and technological systems break down. Famously, an investigation revealed that the failure of one tiny, inexpensive part – an O-ring – led to the explosion. If anything good came of the tragedy, it was the advances it spurred in a variety of fields. It even influenced education, albeit obliquely: the Challenger case study is now taught in business schools around the world. 

Challenger prompted Michael Kremer to develop O-ring theory: a model that helps to explain, among other things, why it is so challenging for low income countries to jump start economic development. Some of the basic precepts of this theory are that:

  • Complex products are created in multiple distinct steps,
  • A deterioration of quality in any one step affects the quality of the whole product, and
  • Quality deteriorations are multiplicative: the more steps there are in a process, the more the overall quality degrades.

To see how this works, we can represent the quality of each step in a production process on a scale from zero (unusable) to one (perfect) and multiply the qualities of each step to determine overall product quality. A quality of 90% for each step in a 30-step process yields an overall quality of (0.9)30, or 0.04, which means the end-product is essentially useless. 

Kremer built these assumptions into a larger model that helps to explain all kinds of economic phenomena, but one conclusion is fairly obvious: to create complex products that are high quality, the quality of each individual component must be extremely high

Online Learning Experiences are Complex Products

Consider all of the different steps, and expertises required, to create an online course. There are a lot of different components that go into the experience. Here are a few. 

  1. Strategy: the “why,” which clearly and compellingly aligns the overall learning experience with the needs of the target audience,
  2. Platform: the interface that learners use to interact with learning resources,
  3. Learning design: the specific activities that learners work through in order to learn,
  4. Content: the material to be learned, which interacts heavily with learning design,
  5. Behavioral science: commitment mechanisms, reminders, social interactions, and rewards that keep a learner engaged and coming back, and
  6. Quality control and learner support: everything done to make sure that the whole experience works as expected and that learners have the support they need in order to be successful.

Though such ratings of “quality” are obviously arbitrary, it seems reasonable to assume that most online learning experiences could be improved in each of these domains. Let’s say that each of these six components has a quality of 0.7, so the overall quality is just 0.12 (0.76). Even a small reduction in the quality of each component of an online learning experience equates to a huge degradation in its overall quality. 

Explaining Skepticism

The O-ring model helps to explain pervasive skepticism about online learning. Even when the quality of individual components is reasonable, the outputs (such as course completion rates and learning outcomes) are not very impressive. The quality of each component needs to become very high for the overall experience to achieve outcomes that are acceptable. This means that organizations need to evaluate the impact of every step in the learning process, and implement improvements across the spectrum of the learning experiences they offer, to substantially increase overall quality.

Predicting the “Next Big Thing”

If O-ring theory applies to online learning, it may tell us something about future innovations in EdTech. There’s a lot of talk about revolutionary, game-changing technologies like AI, blockchain, or virtual reality. Those technologies may well bring about major innovations in online learning. However, the compounding nature of quality suggests that we could make much better learning products by just thoughtfully implementing a portfolio of best practices from diverse fields (including education research, technology product development, behavioral science, and business). Such “compounding incrementalism” has a lot in common with lean innovation, but it’s distinct from other mental models of innovation that rely on new inventions and proprietary technology. EdTech providers that master best practices and strive for operational excellence might realize “10X” improvements over existing online learning without a need for any of the latest fads in technology.

Barriers to the “Next Big Thing”

Compounding incrementalism doesn’t sell. In the for-profit space, investors are looking for proprietary technologies that give businesses a monopoly advantage. It’s often easier for nonprofits to get funding for new initiatives and more difficult to secure support for incremental improvements to existing programs. This could create a sort of trap in which innovations that are well within reach don’t attract the funds or attention that they need to thrive.