This is a measure of our confidence in any given model or technique, including whether it was developed from cognitive science research, through practical iteration, or a mix of both. We currently use five confidence categories:
These are our most hypothetical courses, comprising newly generated theories, experimental pedagogy, and untested or anecdotal conjectures. While we wouldn’t include them if they weren’t promising, they are likely to change as we evaluate and iterate.
Like the courses in the preliminary/tentative category, these include some elements that lack strong research-based underpinnings, but they also rely heavily on established literature. In general, we try to highlight which parts are more speculative.
This category overlaps with mixed in terms of confidence level. Courses that are labeled anecdotally strong generally have little or no formal research behind them, but have been refined through iteration and feedback from participants.
These are courses that are directly derived from or supported by research, and are generally straightforward applications of well-known concepts in cognitive psychology, sociology, economics, game theory, or decision theory.
Established and confirmed.
These are the courses with the strongest possible confidence, based on “laws” rather than “theories”— concepts that have been repeatedly confirmed and interventions that reliably have large and significant effects.