Educational Technology · Course Instructors
Check Student Quiz Answers for Concept Understanding with Noul
You need to grade open-ended quiz answers at scale without reading every response manually. A Noul question evaluates whether a student's free-text explanation demonstrates understanding of the instructor's concept summary, helping you decide whether to mark the answer correct and advance the lesson or prompt revision.
Updated · Examples verified with jev-1.13.0
Interactive demo
Concept Mastery Gauge
A minimal quiz grader. Type a question, concept, and student answer; the gauge sweeps to show how well the answer demonstrates understanding.
Three text fields: quiz question, instructor concept summary, student's written answer. Each run sends one live request to the public API; results are model judgments and can vary.
Verified sample results (3 inputs, jev-1.13.0)
Clear, complete answer covering all key components of photosynthesis.
{ "state": { "quiz_question": "What is photosynthesis?", "student_answer": "Photosynthesis is when plants use sunlight to turn carbon dioxide and water into glucose sugar for energy, and they release oxygen into the air.", "concept_summary": "The process by which plants convert light energy into chemical energy stored in glucose, using carbon dioxide and water as inputs and releasing oxygen as a byproduct." }, "answers": { "demonstrates_understanding": { "noul": 0.98, "type": "noul" } } }Incomplete answer missing the motion aspect and the concept of inertia.
{ "state": { "quiz_question": "Explain Newton's First Law of Motion.", "student_answer": "Things don't move unless you push them.", "concept_summary": "An object at rest stays at rest and an object in motion stays in motion with constant velocity unless acted upon by an external force (the law of inertia)." }, "answers": { "demonstrates_understanding": { "noul": 0.26, "type": "noul" } } }Strong answer demonstrating understanding of axial tilt and its effect on sunlight.
{ "state": { "quiz_question": "What causes the seasons on Earth?", "student_answer": "The tilt of Earth's axis means that when the Northern Hemisphere is tilted toward the sun it gets more direct sunlight and has summer, while the Southern Hemisphere has winter.", "concept_summary": "Earth's axial tilt of 23.5 degrees causes different hemispheres to receive varying amounts of direct sunlight throughout the year as Earth orbits the sun." }, "answers": { "demonstrates_understanding": { "noul": 0.97, "type": "noul" } } }
The problem
The input contains the quiz question, the instructor's concept summary, and the student's typed answer. The answer may articulate the concept using different wording, include irrelevant details, or miss key points entirely. The application must judge whether the student demonstrates sufficient understanding so the platform can unlock the next lesson or display a hint requesting clarification. The Noul probability reflects how well the answer aligns with the concept; it does not verify factual correctness against external sources or guarantee that two equivalent answers will receive identical probabilities. Application code sets the threshold for marking an answer correct and decides when to escalate ambiguous cases for instructor review.
How TypeSafe helps
The input contains the quiz question, the instructor's concept summary, and the student's typed answer. The Noul question asks whether the student answer demonstrates understanding of the core concept, and the API returns a probability between 0 and 1. In the first example the student articulates the key inputs, output, and chlorophyll's role using informal wording, returning 0.96; the incomplete explanation that confuses nutrient absorption with photosynthesis returns 0.19; the placeholder "idk" returns 0.02. Application code can set a threshold—such as 0.7—to mark answers correct and unlock the next lesson, display a hint and request revision for lower probabilities, or flag mid-range probabilities for instructor review.
- Input
- A JSON object with the quiz question, instructor concept summary, and student free-text answer.
- Decision
- Noul: Does the student answer demonstrate understanding of the core concept described in the instructor summary?
- Next action
- Learning platform marks the answer correct and unlocks the next lesson or displays a hint and requests revision.
Three verified examples
These responses were returned during a previous API verification. New probabilities can differ.
Student correctly explains photosynthesis concept (primary)
Expected behavior: Student articulates the key inputs (sunlight, water, carbon dioxide) and output (sugar/glucose) and references the role of chlorophyll using accessible language, demonstrating solid understanding despite informal wording.
Input
{
"question": "How do plants produce their own food?",
"student_answer": "Plants make food by taking in sunlight through their leaves. They also use water from the soil and carbon dioxide from the air. The green stuff in leaves helps turn these into sugar that the plant uses for energy.",
"concept_summary": "Plants use sunlight, water, and carbon dioxide to produce glucose through photosynthesis. Chlorophyll in the leaves captures light energy to drive this chemical reaction."
}Previous verification response
jev-1.13.0 ·
{
"model": "jev-1.13.0",
"usage": {
"input_tokens": 388,
"output_tokens": 22
},
"answers": {
"understanding": {
"noul": 0.96,
"type": "noul"
}
}
}Student provides incomplete explanation missing key components (alternative)
Expected behavior: Student mentions sunlight but confuses nutrient absorption with photosynthesis, omits carbon dioxide and glucose production, and does not demonstrate understanding of the chemical process described in the concept summary.
Input
{
"question": "How do plants produce their own food?",
"student_answer": "Plants need sunlight to grow. The sun gives them energy and they use their roots to get nutrients from the dirt.",
"concept_summary": "Plants use sunlight, water, and carbon dioxide to produce glucose through photosynthesis. Chlorophyll in the leaves captures light energy to drive this chemical reaction."
}Previous verification response
jev-1.13.0 ·
{
"model": "jev-1.13.0",
"usage": {
"input_tokens": 369,
"output_tokens": 22
},
"answers": {
"understanding": {
"noul": 0.19,
"type": "noul"
}
}
}Student submits vague placeholder response (edge case)
Expected behavior: Student provides no substantive explanation and does not demonstrate any understanding of the photosynthesis concept.
Input
{
"question": "How do plants produce their own food?",
"student_answer": "idk",
"concept_summary": "Plants use sunlight, water, and carbon dioxide to produce glucose through photosynthesis. Chlorophyll in the leaves captures light energy to drive this chemical reaction."
}Previous verification response
jev-1.13.0 ·
{
"model": "jev-1.13.0",
"usage": {
"input_tokens": 348,
"output_tokens": 22
},
"answers": {
"understanding": {
"noul": 0.02,
"type": "noul"
}
}
}Try online
Start with a verified example, edit the request, then ask Jev. A live request is sent only when you press Try online.
Expected behavior: Student articulates the key inputs (sunlight, water, carbon dioxide) and output (sugar/glucose) and references the role of chlorophyll using accessible language, demonstrating solid understanding despite informal wording.
Use non-sensitive test data. Live input goes to typesafe.pro and TypeSafe. Anonymous requests use the gateway’s free rate limit.
Probability that the answer is yes
Near 1 is a strong yes. Near 0 is a strong no. Around 0.5 means uncertainty. Noul has no separate confidence value.
import jsonfrom urllib.error import HTTPError, URLErrorfrom urllib.request import Request, urlopenBASE_URL = "https://api.typesafe.pro"payload = { "model": "jev-latest", "state": { "question": "How do plants produce their own food?", "student_answer": "Plants make food by taking in sunlight through their leaves. They also use water from the soil and carbon dioxide from the air. The green stuff in leaves helps turn these into sugar that the plant uses for energy.", "concept_summary": "Plants use sunlight, water, and carbon dioxide to produce glucose through photosynthesis. Chlorophyll in the leaves captures light energy to drive this chemical reaction.", }, "questions": { "understanding": { "type": "noul", "instructions": "Does the student answer demonstrate understanding of the core concept described in the instructor summary?", }, },}request = Request( f"{BASE_URL}/v1/systemone", data=json.dumps(payload).encode("utf-8"), headers={"Content-Type": "application/json"}, method="POST",)try: with urlopen(request, timeout=20) as response: result = json.load(response)except HTTPError as error: raise SystemExit(f"API returned HTTP {error.code}") from errorexcept URLError as error: raise SystemExit(f"Connection failed: {error.reason}") from errorprint(json.dumps(result["answers"], indent=2))Limitations and review
- The Noul probability reflects how well the student answer aligns with the instructor's concept summary using the supplied wording; it does not verify factual correctness against external sources, and equivalent answers may receive different probabilities across evaluations.
- Application code chooses the threshold for marking an answer correct and decides whether to offer hints, request revision, or escalate ambiguous cases for instructor review; the API itself does not execute grading or advancement actions.
- The examples demonstrate single-concept questions with clear instructor summaries; answers that address multi-part concepts or questions with vague summaries may require you to adjust thresholds or add instructor review for mid-range probabilities.
Set thresholds against your own examples before relying on an automated decision.