Owner Nikunj

Custom Test

1. What is a Custom Test?

A Custom Test is a multi-question MCQ session launched from the Custom Test builder. A fixed set of questions is batch-loaded at start; all answers submit together at the end and the student is taken to a Result Screen.

2. Creation

The Custom Test builder is the only entry point. The student configures the test once and taps Create to launch the session.

Builder inputs:

  • Scope — subjects, topics, and/or years the questions are drawn from
  • Question count
  • Duration — required for Exam mode; not used in Study
  • Mode — STUDY or EXAM
  • Explanation mode (Study only) — SHORT or FULL

3. MCQ selection algorithm

On Create, the server picks the question set and freezes it on the test definition. The current version (V3) is fresh-first: serve questions the user has never seen, and only repeat old ones if the fresh pool is too small.

Inputs from the builder:

  • taxonomy_ids — subjects / topics
  • tag_ids
  • years
  • limit — number_of_mcqs (5–50)
  • already_served_mcq_ids — the user’s queue of MCQs seen in previous custom tests, persisted at user_data_bucket.mcq_ids_used_in_test_creation

Two phases:

Phase 1 — unattempted first. Query the bank for PUBLISHED MCQs matching the filters and not in already_served_mcq_ids. If this returns ≥ limit, take the first limit and stop.

Phase 2 — top up with repeats. If Phase 1 came up short (e.g. user asked for 30, only 18 fresh exist):

  1. Walk already_served_mcq_ids in append order (oldest first).
  2. Re-apply the same filters in-memory (taxonomy / tag / year intersect).
  3. Dedupe against Phase 1.
  4. Take just enough to reach limit.

Final list = [fresh MCQs] + [oldest repeats filtered], saved as the test’s ordered mcq_ids. The set is fixed from here.

4. Modes

Two mutually exclusive runtime modes, chosen at creation:

  • Study
    • Immediate per-question feedback (correct/incorrect + explanation)
    • No timer
    • Forward-only navigation
  • Exam
    • No feedback until results
    • Countdown timer
    • Full Prev / Next / Skip navigation
    • Question palette and Mark for Review

5. Explanation modes (Study only)

Set at test creation. Applies to all questions in the test.

  • SHORT — compact, fixed explanation panel; content blocks omitted
  • FULL — complete explanation inline below the question, including content blocks

Both include topic, tags, a thumbs-up / down rating (persisted, editable), and the MCQ ID.

6. Per-question actions

  • Answer — Study: locked after submit. Exam: editable until final submit.
  • Mark as Guessed — both modes; editable until submit.
  • Mark for Review — Exam only.
  • Bookmark / Share / Report — both modes.

7. Progress & navigation

Both modes share a common header: title, exit (opens the quit sheet), and — Exam only — the timer countdown.

  • Study — linear progress bar showing answered count out of total. Forward-only; no jumping back to prior questions.
  • Exam — question palette accessible from a footer button. Each cell is color-coded by state (answered, unanswered, marked for review, current). Tapping a cell jumps to that question. Footer has Prev / Next / Skip.

8. Submission

All paths (Done / Submit / Finish & Submit / time’s up) → single batch call → success screen → Result Screen.

Confirmation rules:

  • Study — any unanswered question prompts a confirm sheet; otherwise submit directly.
  • Exam — if all questions are answered and none are marked for review, submit directly; otherwise confirm.

Error handling:

  • Already submitted on another device — a “View Results” / “Close” sheet is shown.
  • Any other error — toast; user retries.

9. Timer (Exam only)

Persistent countdown. On expiry, a non-dismissable “time’s up” sheet replaces any open overlay and forces submission. On resume, if elapsed time already exceeds duration, the time’s up sheet appears immediately.

10. Resume

Progress is persisted locally on every interaction (position, answers, guessed flags, marked-for-review). After a force close or crash, the session reopens at the exact question with all state intact.

The quit sheet (from the header exit) offers:

  • Resume Later — preserves state without submitting.
  • Submit Now — routes through the standard submission flow.

11. Data & sync

  • Question load — full batch upfront at test open.
  • Local persistence — position, answers (incl. unanswered as -1), guessed, marked-for-review.
  • Submission model — single batch at end.
  • Offline — not supported; submission failure shows a toast and the user retries.

Submission payload includes:

  • answers — map of mcq_id → selected option, or -1 for unanswered
  • started_at, ended_at — session timestamps
  • guessed_mcq_ids — both modes
  • marked_for_review_mcq_ids — Exam mode only

The submission response includes total / correct counts, marks, duration, and per-taxonomy scores → passed through to the Result Screen.

12. Shared components

  • Bookmark — collection-based; questions can be saved to a collection from any question in the session.
  • Share — share a single question via link.
  • Report — issue type + comments.
  • Result Screen — terminal screen for every submission.

13. DB collections

Two Mongo collections back the feature. Audit fields (created_at, updated_at, is_deleted, created_by, updated_by) come from BaseBeanieDocumentModel on both and are omitted below.

13.1 custom_tests — test definition

One doc per test created. Stores the question set, owner, and the params used to build it.

  • _id — primary key
  • short_uid — short human-readable ID for share / deep-link; unique
  • mcq_ids — ordered MCQs in the test (≥1 enforced)
  • l1_taxonomy_ids — precomputed root taxonomy IDs of the MCQs; stored to avoid recompute on sync
  • course_id — course this test belongs to (CourseEnum)
  • user_id — creator
  • sort_order — display position per (user_id, course_id); assigned from an atomic counter at create time
  • creation_params — optional embed of builder inputs captured at create:
    • mcq_selection_filters — taxonomy / year / tag filters and question-type distribution used by selection algo
    • number_of_mcqs — requested count
    • duration_in_mins — Exam-mode timer; null / ignored in Study
    • test_mode — STUDY | EXAM
    • explanation_mode — ALL | WRONG_ONLY | NONE
    • explanation_detail_level — SHORT | FULL

13.2 custom_test_data — per-user attempt

One doc per (user × course × custom_test). Tracks lifecycle (LIVE → SUBMITTED | DISCARDED), the submission payload, and the computed result.

  • _id — primary key
  • unique_key — composite <user_id>_<course_id>_<custom_test_id>; auto-built on insert; used for idempotent upserts
  • custom_test_id — FK to the parent custom_tests doc
  • custom_test_short_uid — denormalized for fast lookup
  • course_id — scope (CourseEnum)
  • user_id — owner
  • status — LIVE (in progress) | SUBMITTED (done) | DISCARDED (quit without submission); default LIVE
  • submission — optional embed of what the client sent at submit:
    • answers — mcq_id → "option_1"…"option_4", or -1 for unanswered
    • started_at, ended_at — epoch ms
    • silly_mistake_mcq_ids — flagged after results (Study)
    • guessed_mcq_ids — both modes
    • marked_for_review_mcq_ids — Exam only
    • stars_earned — Study-mode stars from client streak machine; clamped server-side, capped at 500 (10 × max 50 MCQs)
  • result — optional embed computed at submission, returned to the Result Screen:
    • total_mcq_count, total_correct_count
    • marks — +2 correct, −0.66 wrong, 0 unattempted
    • stars_earned — mirrors submission’s clamped value; preserved on result for history endpoints
    • duration_in_seconds
    • percentile_distribution — list of {range, count} score buckets
    • taxonomy_wise_scores_client — list of {taxonomy_id, total_mcq_count, total_correct_count} for the Result Screen