IELTS EDULYTICS

Building an AI Feedback Pipeline

Turning an LLM response into structured, validated feedback that the product could actually use.

The Problem

A model can generate feedback, but a production learning platform needs more than generated text. The output has to be structured, validated, connected to scoring and student data, and reliable enough to become part of the actual learning workflow.

The Constraint

Assessment

The output needed to map back to the student’s submitted response.

Feedback

Feedback had to remain useful, clear, and reusable inside the product.

Scoring

Generated output still needed to work with predictable scoring logic.

Student Profiles

Evaluation data needed to connect back to each learner’s history and performance.

Personalisation

The system needed structured data that could support future personalised learning.

  • AssessmentThe output needed to map back to the student’s submitted response.
  • FeedbackFeedback had to remain useful, clear, and reusable inside the product.
  • ScoringGenerated output still needed to work with predictable scoring logic.
  • Student ProfilesEvaluation data needed to connect back to each learner’s history and performance.
  • PersonalisationThe system needed structured data that could support future personalised learning.

The Options

Raw Model Output

Not Chosen

Structured Prompt Output

Not Chosen

AI Feedback Pipeline

Chosen

Why We Didn't Choose It

Raw model responses were quick to display, but difficult for the application to interpret or reuse consistently.

Benefit

Fastest path from model response to user.

Drawback

Unpredictable structure and little control over downstream use.

Decision Rationale

Why We Chose The Approach

From model response to reusable learner evaluationModel responses are converted into structured output, validated before use, processed by deterministic domain rules, then persisted as useful evaluation data connected to the learner. The saved data can inform student profiles and future personalisation. Select a stage or its explanation to highlight that decision.Model responseStructured outputValidationDomain rulesSaved evaluationPROFILES · PERSONALISATION

The Result

AI feedback became structured, validated, and reusable across the learning workflow.

In Retrospect

If I were building this again, I’d evaluate the pipeline around these questions:

  1. What happens when the model is wrong?
  2. What should remain deterministic, and where should domain rules override AI?
  3. How do we measure consistency across evaluations?