Skip to content

Runtime API

Public symbols exported from mograder.runtime, used inside notebooks for autograding checks, per-question marks, and hints.

from mograder.runtime import check, Grader, hint

API Reference

Runtime helpers for mograder notebooks.

Notebooks import either check (holistic grading) or Grader (per-question marks with reactive tracking) from this module.

Holistic usage::

from mograder.runtime import check

Per-question marks usage::

from mograder.runtime import Grader
grader = Grader({"Q1": 10, "Q2": 15, "Analysis": 60})
check = grader.check

Grader

Per-question marks with reactive score tracking.

Usage in a marimo notebook::

grader = Grader({"Q1": 10, "Q2": 15, "Analysis": 60})
check = grader.check

Then use check(label, checks) exactly like the standalone version. Call grader.scores() to display a reactive score table.

Source code in src/mograder/runtime.py 
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
class Grader:
    """Per-question marks with reactive score tracking.

    Usage in a marimo notebook::

        grader = Grader({"Q1": 10, "Q2": 15, "Analysis": 60})
        check = grader.check

    Then use ``check(label, checks)`` exactly like the standalone version.
    Call ``grader.scores()`` to display a reactive score table.
    """

    def __init__(self, marks: dict[str, int | float]):
        self.marks = marks
        self._state, self._set = mo.state({})
        # Detect hidden tests flag from PEP 723 metadata in __file__
        self._has_hidden = False
        try:
            import __main__

            if hasattr(__main__, "__file__") and __main__.__file__:
                with open(__main__.__file__) as f:
                    head = f.read(4096)
                self._has_hidden = bool(_HIDDEN_TESTS_RE.search(head))
        except Exception:
            pass

    def check(
        self,
        label: str,
        checks: list[tuple[bool, str] | tuple[bool, str, int | float]],
    ) -> mo.Html:
        """Check with auto marks badge and state tracking.

        Looks up marks from ``self.marks`` using the question key
        (text before the first colon in label).  Each check can
        optionally carry a weight as a third element; the default
        weight is 1.  Earned marks are proportional to the weight of
        passing checks.
        """
        key = label.split(":")[0].strip()
        avail = self.marks.get(key)

        if not checks:
            _write_sidecar(label, "warn", [])
            if avail is not None and not self._has_hidden:
                badge = (
                    f'<span style="float:right"><code>[0/{avail} marks]</code></span>'
                )
            else:
                badge = ""
            return mo.callout(
                mo.md(f"{badge}**{label}** — waiting for your code"), kind="warn"
            )

        parsed = _parse_checks(checks)
        failures = [msg for ok, msg, _w in parsed if not ok]
        earned_w = sum(w for ok, _, w in parsed if ok)
        total_w = sum(w for _, _, w in parsed)

        # Update reactive state with (earned_weight, total_weight) tuple
        self._set(lambda prev, k=key, ew=earned_w, tw=total_w: {**prev, k: (ew, tw)})

        # Build marks badge (suppressed when hidden tests exist)
        if avail is not None and not self._has_hidden:
            earned = round(avail * earned_w / total_w, 1) if total_w > 0 else 0
            # Display as int if whole number
            earned_str = str(int(earned)) if earned == int(earned) else str(earned)
            badge = f'<span style="float:right"><code>[{earned_str}/{avail} marks]</code></span>'
        else:
            badge = ""

        if failures:
            if earned_w > 0:
                kind = "info"  # blue — partial credit
                sidecar_status = "partial"
            else:
                kind = "danger"
                sidecar_status = "danger"
            _write_sidecar(label, sidecar_status, failures, earned_w, total_w)
            items = "\n".join(f"- {f}" for f in failures)
            return mo.callout(
                mo.md(f"{badge}**{label}** — some checks failed:\n\n{items}"),
                kind=kind,
            )
        _write_sidecar(label, "success", [], earned_w, total_w)
        return mo.callout(
            mo.md(f"{badge}**{label}** — all checks passed"), kind="success"
        )

    def scores(self):
        # MOGRADER_SCORES_CELL — removed during feedback export
        """Display a reactive score table callout."""
        if self._has_hidden:
            return mo.callout(
                mo.md("Scores will be available in your feedback after grading."),
                kind="neutral",
            )
        results = self._state()
        total = sum(self.marks.values())
        auto = 0.0
        rows = ""
        for q, pts in self.marks.items():
            val = results.get(q)
            if isinstance(val, tuple):
                # (earned_weight, total_weight) from partial credit
                ew, tw = val
                got = round(pts * ew / tw, 1) if tw > 0 else 0
                if ew == tw:
                    icon = "PASS"
                elif ew == 0:
                    icon = "FAIL"
                else:
                    icon = "PARTIAL"
            elif isinstance(val, bool):
                # Backward compat: old bool state
                got = pts if val else 0
                icon = "PASS" if val else "FAIL"
            elif val is None or q not in results:
                got = 0
                icon = "\u2014"
            else:
                got = 0
                icon = "\u2014"
            auto += got
            got_str = str(int(got)) if got == int(got) else str(got)
            rows += f"| {q} | {icon} | {got_str}/{pts} |\n"
        auto_str = str(int(auto)) if auto == int(auto) else str(auto)
        rows += f"| **Total** | | **{auto_str}/{total}** |\n"
        return mo.callout(
            mo.md(
                f"## Your Score\n\n"
                f"| Question | Status | Marks |\n|----------|--------|-------|\n{rows}"
            ),
            kind="success" if auto == total else "neutral",
        )

check(label, checks)

Check with auto marks badge and state tracking.

Looks up marks from self.marks using the question key (text before the first colon in label). Each check can optionally carry a weight as a third element; the default weight is 1. Earned marks are proportional to the weight of passing checks.

Source code in src/mograder/runtime.py 
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
def check(
    self,
    label: str,
    checks: list[tuple[bool, str] | tuple[bool, str, int | float]],
) -> mo.Html:
    """Check with auto marks badge and state tracking.

    Looks up marks from ``self.marks`` using the question key
    (text before the first colon in label).  Each check can
    optionally carry a weight as a third element; the default
    weight is 1.  Earned marks are proportional to the weight of
    passing checks.
    """
    key = label.split(":")[0].strip()
    avail = self.marks.get(key)

    if not checks:
        _write_sidecar(label, "warn", [])
        if avail is not None and not self._has_hidden:
            badge = (
                f'<span style="float:right"><code>[0/{avail} marks]</code></span>'
            )
        else:
            badge = ""
        return mo.callout(
            mo.md(f"{badge}**{label}** — waiting for your code"), kind="warn"
        )

    parsed = _parse_checks(checks)
    failures = [msg for ok, msg, _w in parsed if not ok]
    earned_w = sum(w for ok, _, w in parsed if ok)
    total_w = sum(w for _, _, w in parsed)

    # Update reactive state with (earned_weight, total_weight) tuple
    self._set(lambda prev, k=key, ew=earned_w, tw=total_w: {**prev, k: (ew, tw)})

    # Build marks badge (suppressed when hidden tests exist)
    if avail is not None and not self._has_hidden:
        earned = round(avail * earned_w / total_w, 1) if total_w > 0 else 0
        # Display as int if whole number
        earned_str = str(int(earned)) if earned == int(earned) else str(earned)
        badge = f'<span style="float:right"><code>[{earned_str}/{avail} marks]</code></span>'
    else:
        badge = ""

    if failures:
        if earned_w > 0:
            kind = "info"  # blue — partial credit
            sidecar_status = "partial"
        else:
            kind = "danger"
            sidecar_status = "danger"
        _write_sidecar(label, sidecar_status, failures, earned_w, total_w)
        items = "\n".join(f"- {f}" for f in failures)
        return mo.callout(
            mo.md(f"{badge}**{label}** — some checks failed:\n\n{items}"),
            kind=kind,
        )
    _write_sidecar(label, "success", [], earned_w, total_w)
    return mo.callout(
        mo.md(f"{badge}**{label}** — all checks passed"), kind="success"
    )

scores()

Display a reactive score table callout.

Source code in src/mograder/runtime.py 
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
def scores(self):
    # MOGRADER_SCORES_CELL — removed during feedback export
    """Display a reactive score table callout."""
    if self._has_hidden:
        return mo.callout(
            mo.md("Scores will be available in your feedback after grading."),
            kind="neutral",
        )
    results = self._state()
    total = sum(self.marks.values())
    auto = 0.0
    rows = ""
    for q, pts in self.marks.items():
        val = results.get(q)
        if isinstance(val, tuple):
            # (earned_weight, total_weight) from partial credit
            ew, tw = val
            got = round(pts * ew / tw, 1) if tw > 0 else 0
            if ew == tw:
                icon = "PASS"
            elif ew == 0:
                icon = "FAIL"
            else:
                icon = "PARTIAL"
        elif isinstance(val, bool):
            # Backward compat: old bool state
            got = pts if val else 0
            icon = "PASS" if val else "FAIL"
        elif val is None or q not in results:
            got = 0
            icon = "\u2014"
        else:
            got = 0
            icon = "\u2014"
        auto += got
        got_str = str(int(got)) if got == int(got) else str(got)
        rows += f"| {q} | {icon} | {got_str}/{pts} |\n"
    auto_str = str(int(auto)) if auto == int(auto) else str(auto)
    rows += f"| **Total** | | **{auto_str}/{total}** |\n"
    return mo.callout(
        mo.md(
            f"## Your Score\n\n"
            f"| Question | Status | Marks |\n|----------|--------|-------|\n{rows}"
        ),
        kind="success" if auto == total else "neutral",
    )

check(label, checks)

Run a list of checks and display coloured feedback.

Parameters:

Name Type Description Default
label str

Name of the test (e.g. "Q2: Model evaluation")

 required
checks list[tuple[bool, str] | tuple[bool, str, int | float]]

List of (bool_expr, fail_message) or (bool_expr, fail_message, weight) tuples.

 required

Returns a coloured callout: green (PASS), red (FAIL), or amber (WAIT).

Source code in src/mograder/runtime.py 
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
def check(
    label: str, checks: list[tuple[bool, str] | tuple[bool, str, int | float]]
) -> mo.Html:
    """Run a list of checks and display coloured feedback.

    Args:
        label: Name of the test (e.g. "Q2: Model evaluation")
        checks: List of ``(bool_expr, fail_message)`` or
                ``(bool_expr, fail_message, weight)`` tuples.

    Returns a coloured callout: green (PASS), red (FAIL), or amber (WAIT).
    """
    if not checks:
        _write_sidecar(label, "warn", [])
        return mo.callout(mo.md(f"**{label}** — waiting for your code"), kind="warn")
    parsed = _parse_checks(checks)
    failures = [msg for ok, msg, _w in parsed if not ok]
    earned_w = sum(w for ok, _, w in parsed if ok)
    total_w = sum(w for _, _, w in parsed)
    if failures:
        if earned_w > 0:
            sidecar_status = "partial"
        else:
            sidecar_status = "danger"
        _write_sidecar(label, sidecar_status, failures, earned_w, total_w)
        items = "\n".join(f"- {f}" for f in failures)
        return mo.callout(
            mo.md(f"**{label}** — some checks failed:\n\n{items}"),
            kind="danger",
        )
    _write_sidecar(label, "success", [], earned_w, total_w)
    return mo.callout(mo.md(f"**{label}** — all checks passed"), kind="success")

hint(*hints)

Display progressive hints in a collapsed accordion.

Source code in src/mograder/runtime.py 
68
69
70
71
72
73
74
def hint(*hints: str) -> mo.Html:
    """Display progressive hints in a collapsed accordion."""
    if len(hints) == 1:
        items = {"Hint": mo.md(hints[0])}
    else:
        items = {f"Hint {i}": mo.md(h) for i, h in enumerate(hints, 1)}
    return mo.accordion(items)

Known Constraints

  • Reactive ordering: In marimo, cells execute in dependency order, not top-to-bottom. Ensure check() cells depend on the variables they test.
  • Empty checks = WAIT: An empty checks list always produces an amber "waiting" callout and does not write to the sidecar. This is by design for mo.stop() guards.
  • Question key extraction: The key is label.split(":")[0].strip(). If your label has no colon, the entire label is the key. Ensure keys match between check() labels and the _marks dictionary.
  • Sidecar is append-only: If a cell re-executes (e.g. due to reactive updates), multiple entries for the same label may appear. The runner uses the last entry per label.
  • Sidecar mechanism: During mograder autograde, the environment variable MOGRADER_SIDECAR_PATH is set to a temp JSONL file. Each check() call appends a JSON record {"label", "status", "details", "earned_weight", "total_weight"} to this file. The runner polls it for live progress. Empty-check calls (the mo.stop() guard) do not write to the sidecar to avoid false results.