AI Grading
AI Grading
Skillnote uses two grading methods depending on the question type. Simple selection and input-based questions are graded instantly, while open-ended questions are evaluated semantically by the AI.
Two Grading Methods
Automatic Grading (Local)
Applies to: True/False (true_false), Multiple Choice (multiple_choice), Fill in the Blank (fill_blank)
Results are returned immediately on the client side by comparing your answer to the correct answer — no server or AI call is needed.
| Type | Comparison Method |
|---|---|
| True/False | Your “True” / “False” selection is compared exactly to the correct answer. |
| Multiple Choice | Your selected option (A/B/C/D) is compared exactly to the correct answer. |
| Fill in the Blank | Each blank’s input is compared to the correct answer using case-insensitive matching. react and React are treated as the same. |
Automatic grading results are confirmed instantly, with no processing delay.
AI Grading (Lambda)
Applies to: Short Answer (short_answer), Essay (long_answer)
When you complete or end a review session, all open-ended answers are sent in a batch to a Lambda function, where the AI evaluates them semantically. This is the process shown on the “Grading…” screen.
AI grading results include the following fields.
| Field | Description |
|---|---|
is_correct | Whether the answer is correct or incorrect (boolean) |
score | A score of 0–100, applied to Essay (long_answer) questions only. Short Answer questions return correct/incorrect only. |
feedback | Detailed AI feedback on the answer. Displayed on the results page as “SkillNote Feedback: {feedback}”. |
is_gradable | Whether the AI can determine correctness (boolean). If false, the answer is treated as a creative response. |
Synonym Recognition
AI grading evaluates answers based on semantic similarity, not exact string matching. If you express the correct concept using different words, your answer can still be accepted as correct.
Synonym recognition examples
| Submitted Answer | Correct Answer | Result |
|---|---|---|
JS | JavaScript | Marked correct |
React | React | Marked correct |
DB | database | Marked correct |
async | asynchronous | Marked correct |
When an answer is accepted through synonym recognition, a “Accepted as synonym” badge is shown on that item in the results page.
Creative Response Handling
When the AI cannot clearly determine whether an answer is correct or incorrect, the answer is treated as a creative response.
- Internally, the
is_gradablefield is set tofalse. - A “Creative Response” badge is shown on that item in the results page.
- Creative responses are not counted as incorrect and do not negatively affect your accuracy score.
- They offer a metacognitive learning opportunity — read the content yourself and assess your own understanding.
Common situations where a creative response is assigned:
- The question is open-ended with multiple valid answers
- The answer includes personal experiences or opinions
- The question is a high-level essay question without a clearly defined correct answer
Fallback Mechanism (When AI Grading Fails)
If Lambda grading fails due to a network error or AI service outage, the system automatically falls back to local grading.
- A notification is shown on screen indicating that fallback grading has been applied.
- Fallback grading uses simple text comparison and has the following limitations:
- Synonyms and similar expressions may not be recognized.
- Semantically correct answers may be marked as incorrect.
- All answer data is saved to the database regardless of grading failure, so your learning history is preserved.
Automatic vs. AI Grading: Comparison
| Item | Automatic Grading | AI Grading |
|---|---|---|
| Applies to | True/False, Multiple Choice, Fill in the Blank | Short Answer, Essay |
| When graded | Immediately (during the session) | After session completion, in a batch |
| Where graded | Client (local) | Lambda function (server) |
| Synonym recognition | None (Fill in the Blank ignores case only) | Yes |
| Score format | Correct / Incorrect only | Correct/Incorrect + 0–100 score for Essay |
| Feedback | None | ”SkillNote Feedback” provided |
| Creative response handling | None | Applied when is_gradable === false |
| Fallback on failure | N/A | Falls back to local text comparison |
Answer Storage
All answers submitted during a review session are saved to the database along with their grading results.
- Stored fields: correct/incorrect status, AI score (
score), AI feedback (feedback), creative response flag (is_gradable) - Stored data is used by the spaced repetition algorithm to calculate future review intervals.
- The results page can be revisited at any time after a session to review past results.