FOR PARENTS & TEACHERS: Every mechanic in Order Up! is tied to published evidence, and the evidence is sorted honestly. TIER 1 lists causal evidence: randomized controlled trials and strong quasi-experiments that justify the game's instructional choices. TIER 2 lists descriptive research: studies that document the misconceptions the game targets (no causal claim needed to know an error is common). Where a source is descriptive, it is labeled as such.
TIER 1: CAUSAL EVIDENCE (randomized and quasi-experimental trials)
🎮 The wrong-chef kitchen: Lefty, Chef Rigatoni, and Basil serve botched dishes, and players find the bad line
Randomized in-vivo classroom experiments: studying INCORRECT worked examples, and explaining what is wrong and why, improved students' conceptual understanding of algebra beyond correct examples alone (no differences were found on procedural measures). Every error-analysis plate in this game is an incorrect worked example targeting a documented order-of-operations error.
Booth, Lange, Koedinger & Newton (2013): differentiating correct and incorrect examples.
🎮 Two Cooks side by side (rank-skipping vs the real order), and smart-regrouping choices ("which pair makes 100?")
Randomized classroom experiment: students who compared solution methods side by side gained more procedural knowledge and flexibility, with comparable conceptual gains, than students who studied the same methods one at a time. The two-cooks lessons and the pick-the-easy-pair regrouping steppers are direct comparisons of strategies on the same problem, and each lesson teaches one method before comparing, since comparison helps most once students already know one strategy.
Rittle-Johnson & Star (2007): comparing solution methods.
🎮 The NEXT BITE stepper: read the expression's structure and pick WHICH piece computes next, before touching any arithmetic
The IES/WWC algebra practice guide is an evidence synthesis that grades each of its recommendations minimal, moderate, or strong; where trial evidence is thin, recommendations also rest partly on panel expert judgment. Recommendation 2, teach students to notice and use structure before manipulating, is a minimal-evidence recommendation reflecting expert consensus and emerging research. The stepper forces a structure decision on every single step, and wrong picks get a targeted explanation, not just a buzzer.
Star et al. (2015), Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students (NCEE 2015-4010), Rec 2.
🎮 Jokes that ENCODE the math: cookie-swapping for commutativity, trays that feed every item for distribution
The design basis is Ziv's two semester-long randomized experiments with college students (same instructor, same syllabus, only the content-related humor varied): humor groups scored significantly higher on the same final exams. Separately, randomized experiments with young children found well-PACED humorous inserts, unrelated to the lesson content, increased attention and information acquisition from educational TV; that supports attention and pacing effects, not content-linked humor itself. Honest caveat: experimental evidence on classroom humor in K-8 is thin, and one recent college experiment found integrated humor REDUCED learning. That is exactly why this game's humor follows strict content-linked guardrails: load-bearing jokes restate the concept, a joke option is never the correct answer, at most one per question, and never a defensible answer.
Ziv (1988): two randomized semester experiments, college students, content-related humor. Zillmann et al. (1980): randomized experiments, pacing of humorous inserts in children's educational TV. Bolkan, Griffin & Goodboy (2018): college experiments where integrated humor lowered test performance.
TIER 2: WHAT WE TARGET (descriptive research documenting the misconceptions)
🎮 The "detached minus sign" trap (8 − 2 + 3 cooked as 8 − 5 = 3), the rigid-PEMDAS chef (12 ÷ 4 × 3 served as 1), and the ingredient-cards lessons that teach the sign to stay glued to its number
Descriptive studies of students' structure sense in numerical and algebraic expressions document the central misconception this game targets: the detached minus sign. Students split an expression at the operation signs, detach the − from the number it belongs to (computing 2 + 3 first in 8 − 2 + 3), and lose the value. The same literature documents treating the PEMDAS letters as six strict ranks instead of four. Descriptive: it tells us WHAT goes wrong, not how to fix it. The game's response is to teach the underlying structure honestly: a − b + c is a sum of signed terms, so same-rank terms may be reordered as long as each sign stays attached to its number, and walking left to right is taught as the always-safe default rather than as a mathematical law. Players then catch both wrong chefs (the rank-skipper and the sign thief) in the act.
Linchevski & Livneh (1999): structure sense in numerical contexts, including the detached minus sign. Kieran (1979): children's operational thinking with bracketing and the order of operations.
🎮 Basil's tray mistake: 3(x + 4) = 3x + 4, feeding only the first item
The same structure-sense literature documents students applying an operation to only the first term inside a grouping. The game teaches the tray model and the sliced-rectangle area model first, then serves the classic error as a plate to inspect. Descriptive, labeled as such.
Linchevski & Livneh (1999); Kieran (1979), cited via the later literature (Linchevski & Livneh is the load-bearing source).
📋 An honest note on the evidence base for this topic
We could find no randomized trials of order-of-operations training in grades K-8. The misconceptions are well documented (the descriptive studies above), and a recent preregistered experiment with 130 adults found the same misconceptions persist into adulthood and brief reminders are not enough to fix them. Direct training trials are lacking, so this game's design extrapolates from adjacent strong evidence on worked examples, error analysis, and structure, and we say so rather than implying trials that do not exist.
Eaves, Attridge & Gilmore (2025): preregistered experiment with adults, Learning and Instruction.
SIGNED NUMBERS: THE BELOW ZERO LESSON
🎮 "Cards Below Zero" and the Below Zero drill: negatives taught as a walk below zero on the number line, with contrasting signed cases side by side (3 − (−4) next to 3 + (−4)), placed early in Station 1
How to TEACH signed numbers has only a small causal base: no meta-analysis and no federal practice guide of its own, so we tread carefully. What experimental work exists points in two directions, and the game follows both. On model choice, a cluster-randomized comparison of an integer number-line ("moving along a path") model against a two-color chips ("collecting objects") model found the motion model at least as effective, and argued that the collecting-objects metaphor can itself become an obstacle for negatives; small number-line teaching experiments with young children moved them toward correct order-and-value understanding. The game uses the number-line "below zero" model rather than chips or a money and debt context, and it extends the number line kids already own. On sequencing, short interventions in which students analyze contrasting signed problems side by side produced significant gains, and younger students gained MORE than older ones, because over-generalized whole-number rules harden with age; that is why the lesson lands early. Labeled honestly: these are small studies and one conference cluster-randomized comparison, not a settled result, and the subtract-a-negative case specifically is the thinnest, least-tested part of the base.
Nurnberger-Haag (2015): cluster-randomized number-line vs chips comparison, PME-NA 37. Bofferding (2014): number-line teaching experiment with first graders. Aqazade, Bofferding & Farmer (2016): contrasting integer cases. Stephan & Akyuz (2012): the money and net-worth alternative, a descriptive design study.
🎮 "Two minuses in a row flip to a PLUS" and "taking away a minus is adding" (3 − (−4) = 7), with the classic error 3 − 4 served as the wrong plate
Descriptive research documents exactly what to preempt. The minus sign carries two meanings children conflate: the binary "subtract" and the unary "negative" or "opposite"; students who read it only as "subtract" mishandle adjacent signs like 3 − (−4) (Vlassis). And across every grade tested, the hardest integer problems are the counterintuitive ones where a subtraction makes the value GROW, which is exactly the subtract-a-negative case, because children carry over the whole-number rule that "subtraction always makes smaller" (Bishop and colleagues; Lamb and colleagues). The lesson names both meanings of the minus sign, teaches that removing a below-zero card sends the count up, and shows 3 − 4 as the wrong dish so the double negative is met head on. Descriptive: it documents the error, not a proven cure.
Vlassis (2004, 2008): the two meanings of the minus sign. Bishop, Lamb, Philipp, Whitacre, Schappelle & Lewis (2014); Lamb, Bishop, Philipp, Whitacre & Schappelle (2018): the counterintuitive subtract-a-negative problem is hardest at every grade.
🎮 The Cookbook: "−4 sits four steps left of 0, so −4 is smaller than −1", and the whole below-zero model living on a number line
A signature early error is ordering integers by the size of their digits: judging −5 as greater than −3 because 5 is bigger than 3 (Bofferding). The Cookbook states the order on the number line directly. And the number line is a well-supported picture of magnitude: children's mental number lines grow more linear with development, and that linearity predicts math achievement; while estimates of negative numbers lag behind positives, students do form linear representations of them, helped by anchoring on zero, which is precisely the below-zero framing the game uses (Booth & Siegler; Young & Booth). Descriptive and correlational.
Bofferding (2014): first graders' order and value mental models. Booth & Siegler (2006): the developing linear number line. Young & Booth (2015): magnitude of negative numbers on the number line.
FULL REFERENCES
Aqazade, M., Bofferding, L., & Farmer, S. (2016). Benefits of analyzing contrasting integer problems: The case of four second graders. In Proceedings of the 38th Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (PME-NA). [ERIC ED583641]
Bishop, J. P., Lamb, L. L., Philipp, R. A., Whitacre, I., Schappelle, B. P., & Lewis, M. L. (2014). Obstacles and affordances for integer reasoning: An analysis of children's thinking and the history of mathematics. Journal for Research in Mathematics Education, 45(1), 19-61.
Bofferding, L. (2014). Negative integer understanding: Characterizing first graders' mental models. Journal for Research in Mathematics Education, 45(2), 194-245.
Bolkan, S., Griffin, D. J., & Goodboy, A. K. (2018). Humor in the classroom: The effects of integrated humor on student learning. Communication Education, 67(2), 144-164.
Booth, J. L., Lange, K. E., Koedinger, K. R., & Newton, K. J. (2013). Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples. Learning and Instruction, 25, 24-34.
Booth, J. L., & Siegler, R. S. (2006). Developmental and individual differences in pure numerical estimation. Developmental Psychology, 42(1), 189-201.
Eaves, J., Attridge, N., & Gilmore, C. (2025). Misconceptions of the order of operations and associativity use. Learning and Instruction, 97, 102074.
Kieran, C. (1979). Children's operational thinking within the context of bracketing and the order of operations. In D. Tall (Ed.), Proceedings of the Third International Conference for the Psychology of Mathematics Education (pp. 128-133). Coventry: Mathematics Education Research Centre, Warwick University.
Lamb, L. L., Bishop, J. P., Philipp, R. A., Whitacre, I., & Schappelle, B. P. (2018). A cross-sectional investigation of students' reasoning about integer addition and subtraction: Ways of reasoning, problem types, and flexibility. Journal for Research in Mathematics Education, 49(5), 575-613.
Linchevski, L., & Livneh, D. (1999). Structure sense: The relationship between algebraic and numerical contexts. Educational Studies in Mathematics, 40(2), 173-196.
Nurnberger-Haag, J. (2015). How students' integer arithmetic learning depends on whether they walk a path or collect objects. In Proceedings of the 37th Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (PME-NA) (pp. 165-172).
Rittle-Johnson, B., & Star, J. R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, 99(3), 561-574.
Star, J. R., Caronongan, P., Foegen, A., Furgeson, J., Keating, B., Larson, M. R., Lyskawa, J., McCallum, W. G., Porath, J., & Zbiek, R. M. (2015). Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students (NCEE 2015-4010). Washington, DC: National Center for Education Evaluation and Regional Assistance (NCEE), Institute of Education Sciences, U.S. Department of Education.
Stephan, M., & Akyuz, D. (2012). A proposed instructional theory for integer addition and subtraction. Journal for Research in Mathematics Education, 43(4), 428-464.
Vlassis, J. (2004). Making sense of the minus sign or becoming flexible in 'negativity.' Learning and Instruction, 14(5), 469-484.
Vlassis, J. (2008). The role of mathematical symbols in the development of number conceptualization: The case of the minus sign. Philosophical Psychology, 21(4), 555-570.
Young, L. K., & Booth, J. L. (2015). Student magnitude knowledge of negative numbers. Journal of Numerical Cognition, 1(1), 38-55.
Zillmann, D., Williams, B. R., Bryant, J., Boynton, K. R., & Wolf, M. A. (1980). Acquisition of information from educational television programs as a function of differently paced humorous inserts. Journal of Educational Psychology, 72(2), 170-180.
Ziv, A. (1988). Teaching and learning with humor: Experiment and replication. Journal of Experimental Education, 57(1), 5-15.
All WWC practice guides: ies.ed.gov/ncee/wwc/PracticeGuides