Owner Burhan

Docket Module Cleanup — PRD

Context

The docket module already shipsdockets collection with full admin CRUD (av1), a mobile sync feed (v1), and web read/list/backlinks (v2). Content has already migrated from the embedded body blocks to a first-class blocks collection referenced by docket.block_ids (per the shipped Block Sync PRD); block_idsblocks is now the canonical source of truth for docket content.

This PRD is a cleanup / slim-down delta, not a feature. It does three things and defers a fourth:

  1. Remove accumulated dead code in DocketService / DocketRepository (zero-caller and transitively-dead methods, an orphaned search-helper cluster) plus two deregistered task-route stubs.
  2. Retire the legacy body / bode storage fields now that blocks have replaced them — storage-only, with no change to any response shape.
  3. Remove one unused admin endpoint (PATCH /dockets/{docket_id}/reorder) and its now-orphaned service/repo functions.
  4. (Deferred) Consolidate the docket CRUD surface to one GET / one POST / one PATCH — handled in a separate backend spec + plan.

It does not re-spec the docket feature, the blocks model or sync protocol, the sort_order mechanism, or the docket-family lookalike routers (docket_link, docket_review, mcq_docket_link, bookmark buckets) — those are separate modules with their own models/services.

Scope guardrail (decided). Every change here is internal: no client-facing request or response contract changes in M1–M3. Removing body from response contracts, and any endpoint consolidation, are explicitly out of scope for this PRD (the latter is M4, deferred).

Scope

MVP — a four-phase docket-module slim-down, sequenced safest-first:

  1. M1 — Dead-code removal. Delete zero-caller / transitively-dead service & repo functions, the orphaned full-text-search helper cluster, and two deregistered task-route stubs.
  2. M2 — Retire body / bode (storage-only). Drop the two DB fields and the bode at-rest encryption, stop writing them on create/edit, and rewire the two image paths that still read/write body over to the blocks collection. Responses keep hydrating body from blocks — no client change.
  3. M3 — Remove unused PATCH /reorder. Delete the endpoint and its orphaned reorder_by_id / reorder_docket chain.
  4. M4 — API consolidation (deferred). Target shape only; design happens in a separate backend spec/plan.

M1 — Dead-code removal

DocketService.get_by_id is NOT dead — do not remove (decided). It is reached via the unified-service-container alias services.docket_service.get_by_id(...) (3 live callers: av1 block routes, v2 blocks-backlinks, bookmark create). A full sweep of services.docket_service.<method> aliases found only this method (and the already-live extract_inline_mcq_short_ids_from_body) reached indirectly — so no other dead-flagged symbol is secretly alive.

Migrations kept as-is (decided). The three stale body→block migrations (jul_22_25/migrate_dockets_from_html.py, jul_25_25/migrate_docket_blocks_to_blocks_table.py, docket_content_cleanup.py) reference the long-removed new_body field and would error if run; none is registered with a runner. They are retained as immutable history — M1 does not touch them.

Deregistered task-route stubs removed (decided). The two av1 /tasks endpoints below never dispatch work (one literally returns "Task is deregistered."). Remove both handlers + their route registration. The sibling task routes that do dispatch work are untouched.

Removal manifest

LayerSymbols to removeLocation
Service fns (10)increment_mcq_stats, update_all_docket_mcq_counts, get_by_block_id, get_max_sort_order, list_by_updated_at_with_blocks, and the PR #283 linked-MCQ cluster: get_by_id_with_linked_mcqs, get_by_id_or_short_uid_with_linked_mcqs, list_with_linked_mcqs, list_by_updated_at_with_linked_mcqs, + transitively-dead populate_linked_mcqssrc/services/docket_services.py
Repo fns (7)list_by_id, get_max_sort_order, get_by_block_id, increment_mcq_stats, update_all_docket_mcq_counts, update_all_docket_linked_mcq_ids, update_all_docket_linked_docket_idssrc/repository/docket_repository.py
Repo private helpers (4 — orphaned search cluster)_determine_match_info, _determine_search_matches, _extract_context, _calculate_relevance_scoresrc/repository/docket_repository.py
Task-route stubs (2)av1_tasks_post_update_all_docket_mcq_counts (:62), av1_tasks_post_update_all_docket_linked_mcq_ids (:298) + any request/response model used only by themsrc/api/av1/tasks/routes.py
  • Dead schemas: 0 — dead methods reused live/shared response models; no docket schema class becomes orphaned.
  • Test impact: 0 — no test across the suite references any removed symbol (reorder/get_max_sort_order test hits belong to the exam/mcq modules).
  • The search-helper cluster is the remnant of a search_dockets/search_suggestions feature that no longer exists in the repo — removed as a unit.

M2 — Retire body / bode (storage-only)

Blocks are the source of truth (decided). block_ids → the blocks collection is canonical. body is a legacy mirror, and bode is a dead at-rest cipher of body read by no backend code. Evidence: reads already hydrate body from blocks at runtime (docket_services.py:851-860, :1360-1376); block ops (attach/detach/reorder) maintain block_ids + inline_mcq_short_ids and never touch body/bode; blocks carry their own at-rest cipher conte.

Storage-only removal (decided). Drop the DB body + bode fields and stop writing them; responses keep hydrating body from blocks at runtime, so no response shape changes. Removing body from the av1/v2 detail response contract is out of scope (it would be a breaking client change; tracked separately, not here). DB-field removal and response-contract removal are decoupled precisely because the response is rebuilt from blocks.

Migration precondition (verified). The body→block migration (apr_15_26/1_docket_body_to_block.py) has run to completion in all environments and every docket is represented by block_ids. ⚠️ This is an assertion about production data, not enforced in code — see Risks.

Create authors no content (decided). Blocks are added only during edit, never at creation. The create request drops body; a newly created docket starts with no content and gains blocks via the edit/block endpoints. (Behaviour change: dockets can now be created “empty”.) The update/edit request also drops body; edits manage content via block_ids + block endpoints (inline-MCQ extraction already has a blocks path: inline_mcq_short_ids_for_block_ids).

Existing Mongo data left in place (decided). Drop the Pydantic fields and stop writing; existing body/bode values in stored documents are left untouched (Mongo is schemaless — ignored on read). An optional $unset cleanup migration may follow later; no hard delete now.

Removal buckets

(a) Drop outright — no successor needed

ItemLocation
DB bode field defmodels/docket_models/db_model.py:236-242
DB body field def (last, after writes stop)models/docket_models/db_model.py:231-235
build_docket_bode_column()core/encryption/field_encryption_utils.py:104-116
bode compute on createrepository/docket_repository.py:195-198
bode refresh on updaterepository/docket_repository.py:634-644
DocketProjection.bode / DocketListProjection.bodemodels/docket_models/projection_model.py:59-62, :293-296
av1 response bode field (always None — no route sets encrypted=True)api/av1/docket/schemas.py:209-212

(b) Rewire to blocks first — the real work

ItemLocationRewire
Image-content lookuprepository/docket_repository.py:3150-3184 (list_ids_by_image_short_uid_in_body)Replace the {body:{$elemMatch:{content:{$regex}}}} query with a regex over blocks.content + distinct(docket_id).
Image-metadata sync writerservices/docket_services.py:673-755 (sync_plate_image_metadata_in_dockets_for_short_uid)Patch + bulk-update the blocks collection instead of reading/writing docket body.
body on createrequest schema + routes.py:79-93 + create serviceDrop body from the create request/service; store no content.
body on updaterequest schema (schemas.py:779-901) + routes.py:314-331 + update service (docket_services.py:189-327)Drop body from the update request; manage content via block_ids.

(c) Already safe — no action

  • v1 sync, av1 list, v2 list already exclude body.
  • Inline-MCQ extraction already has a blocks-native path (inline_mcq_short_ids_for_block_ids), so create/update switch to it cleanly. The body-based extract_inline_mcq_short_ids_from_body loses its create/update callers but is still referenced by the dec_25_25 migration, so it is retained (consistent with “migrations kept as-is”), not removed in this PRD.
  • The body-content image walker _extract_image_short_uids_from_content and the get_..._with_images_and_mcqs hydration path stay — they already read from blocks / are format-agnostic.

Data Model (requirements-level)

DocketDBModel loses two fields; everything else is unchanged.

FieldChangeNotes
bodyRemovedLegacy embedded blocks. Content is now block_idsblocks. Responses hydrate a body-shaped payload from blocks at runtime (unchanged).
bodeRemovedAt-rest cipher of body; read by nothing. Blocks carry their own at-rest cipher conte.
block_idsunchangedCanonical ordered content references.
sort_orderunchangedRetained — used by create, delete-cascade, and list ordering (see M3).

M3 — Remove unused PATCH /dockets/{docket_id}/reorder

Endpoint unused by clients (decided). The admin frontend defines no client method for it and has no /reorder call-site — the docket editor reorders blocks locally and persists via update with block_ids. The docket-level reposition endpoint is dead from the client side.

Index and sort_order retained (decided). Removing reorder_docket does not free the dockt_taxonomy_sort_order_idx (taxonomy_ids, sort_order) index — the delete_by_id sibling-cascade (docket_repository.py:750-758) filters on exactly {taxonomy_ids, sort_order:$gt}. The code comment “used in reorder_docket” is incomplete; keep the index (and optionally fix the comment). sort_order and its counter migrations are unaffected.

Removal cascade (self-contained)

ItemLocation
Route handler reorder_by_id + the summary/description/operation_id blockapi/av1/docket/routes.py:345-391
DocketReorderResponse import alias (it is just BaseMessageResponse, no dedicated schema)api/av1/docket/routes.py:26
DocketService.reorder_by_id (only caller is the route)services/docket_services.py:453-~507
docket_repo.reorder_docket (only caller is reorder_by_id)repository/docket_repository.py:902-~1091
  • Request body schema: none (uses query/path params: course_id, docket_id, new_position). Dedicated response schema: none. Tests: none. Index changes: none.
  • Do not touch the separate PATCH /dockets/{docket_id}/reorder-blocks in the block module — that is block reordering, out of scope here.

M4 — API consolidation (deferred)

Deferred to a separate backend spec + plan (decided). Target shape: docket CRUD should be exactly one GET (variations via query params), one POST, and one PATCH (variations via request data). The exact consolidation design and the reconciliation of the “4 PATCH” expectation against the actual surface are not specced here — they will be decided during a dedicated backend spec + plan after M1–M3 land.

Current docket surface (for reference when M4 is picked up):

SurfaceRoutes
av1POST /dockets, GET /dockets, GET /dockets/{id}, GET /dockets/list-updated-at-bidirectional, PATCH /dockets/{id}, DELETE /dockets/{id} (and PATCH /dockets/{id}/reorder until M3 removes it)
v1GET /dockets/sync
v2GET /dockets/{id}, GET /dockets, POST /dockets/blocks/backlinks

Delivery Milestones

#MilestoneOutcomeStatusPlan
1Dead-code removalDocketService/DocketRepository shed 10 service fns + 11 repo symbols; 2 deregistered task endpoints removed; no behaviour change.✅ complete (PR #773)…/plans/2026-06-17-docket-m1-deadcode.md
2Retire body/bode (storage-only)DB body+bode and bode encryption gone; create/edit stop writing them; image paths read/write blocks; responses unchanged (still hydrated from blocks).✅ complete (PR #773)…/plans/2026-06-17-docket-m2-body-bode.md
3Remove PATCH /reorderUnused docket-reorder endpoint + reorder_by_id + reorder_docket removed; index/sort_order retained.✅ complete (PR #773)…/plans/2026-06-17-docket-m3-reorder.md
4API consolidation1 GET / 1 POST / 1 PATCH for docket CRUD.deferred — separate backend spec + plan

Plan key: BE = backend plan tasks in keystone (keystone/docs/superpowers/specs/ + …/plans/), authored per milestone from this PRD via /plan. M4 gets its own spec + plan when picked up.

Sequencing. M1 → M2 → M3, safest-first. M1 is mechanical (dead code). M2 depends on the verified body→block migration precondition. M3 is independent of M1/M2 and may land in parallel. M4 is gated on M1–M3.

Open Questions

  • M4 design — the consolidated GET/POST/PATCH contract and the “4 PATCH” reconciliation are deferred to a separate backend spec + plan (decided), not resolved here.
  • Everything in M1–M3 is resolved during PRD review: migrations kept as history; task stubs removed; body/bode storage-only with create/edit dropping body; image lookup → blocks.content regex; reorder endpoint + chain removed; index/sort_order retained; existing Mongo body/bode data left in place.

Risks

RiskLikelihoodImpactMitigation
Body→block migration not 100% complete. A docket with body content but empty/unset block_ids would render empty once body is dropped (migration filtered {'body.0':{$exists:true}}, skipping empty-body and post-migration dockets).LowHigh — silent content lossPrecondition verified by owner across all envs. Recommended belt-and-suspenders before dropping the field: a one-off scan for body-non-empty + block_ids-empty dockets returning zero; keep the optional $unset migration separate so body data survives until coverage is confirmed.
Stale image lookup after rewire. The current image-content query scans possibly-stale body; the blocks-native replacement must target live block content.MedMed — sync patches wrong/no docketsMove list_ids_by_image_short_uid_in_body and sync_plate_image_metadata... to the blocks collection together, before body writes stop, so the lookup reads the canonical source.
Hidden caller via dynamic dispatch. A removed method is reached through a path static grep missed (e.g. another services.docket_service.* alias).LowHigh — runtime AttributeErrorAlias sweep already run (only get_by_id + extract_inline_mcq_short_ids_from_body reached indirectly — both retained). Re-grep at implementation time; rely on the test suite + a smoke pass.
Accidentally dropping a shared index. Treating dockt_taxonomy_sort_order_idx as reorder-only would slow the delete sibling-cascade.LowMed — slower deletesDocumented: the delete cascade uses it; index is retained.
Touching the wrong reorder. Confusing docket /reorder (remove) with block /reorder-blocks (keep).LowMed — breaks block reorderingExplicitly scoped: only api/av1/docket/routes.py reorder is removed.

Technical Details (as built)

Status: M1–M3 built — PR #773dev (open). Shipped in the keystone repo on branch fixture/docket-cleanup as 15 code commits (5 / 7 / 3 across M1 / M2 / M3) plus a docs commit. Full keystone test suite green (235 passed via pre-push hook); net code diff ≈ +253 / −1,985 across 12 files. Per-milestone specs at keystone/docs/superpowers/specs/2026-06-17-docket-m{1,2,3}-*-design.md and plans at …/plans/2026-06-17-docket-m{1,2,3}-*.md. M4 (API consolidation) remains deferred to its own spec + plan.

Naming: PRD vs code

PRD termCode
Docket entityDocketDBModel (collection dockets) / DocketProjection, DocketListProjection
Docket service / repoDocketService (src/services/docket_services.py), docket_repository / docket_repo alias (src/repository/docket_repository.py)
Legacy content fieldsbody (List[DocketBlockWithMcqStats]), bode (at-rest CryptoV2.1 ciphertext)
Canonical contentblock_idsblocks collection; per-block at-rest cipher conte
bode builderbuild_docket_bode_column (src/core/encryption/field_encryption_utils.py)
Image-content lookup (to rewire)DocketRepository.list_ids_by_image_short_uid_in_body
Image-metadata sync (to rewire)DocketService.sync_plate_image_metadata_in_dockets_for_short_uid
Unused reorder endpointav1_docket_patch_reorder_by_idDocketService.reorder_by_iddocket_repo.reorder_docket
Retained reorder indexdockt_taxonomy_sort_order_idx (taxonomy_ids, sort_order) (also used by delete_by_id cascade)
Deregistered task stubsav1_tasks_post_update_all_docket_mcq_counts, av1_tasks_post_update_all_docket_linked_mcq_ids

Decisions & deviations from the PRD (as built)

M1 — Dead-code removal. Removed as specced: 10 service fns (incl. the linked-MCQ cluster *_with_linked_mcqs + populate_linked_mcqs), 7 repo methods, the 4-method search-helper cluster, and both task stubs. Kept DocketService.get_by_id and extract_inline_mcq_short_ids_from_body (latter still used by the dec_25_25 migration).

  • Deviation: the PRD called both /tasks/udmc and /tasks/udli “deregistered”. In fact only udli returns "Task is deregistered."; udmc was a no-op that returned a misleading "…has been triggered." message (dispatched nothing, called no service). Both removed; no functional impact.

M2 — Retire body/bode. DB body+bode, the bode builder, and the bode field on both projections and the av1 detail response removed; create/edit stopped writing them (dockets can now be created empty). Decisions/deviations (resolved with owner):

  • Image lookup implemented as a content regex over blocks.content, added to BlockRepository as list_ids_by_image_short_uid_in_contentnot linked_image_ids (denormalized, stale-prone), and not block.docket_id/distinct(docket_id) (origin-only; unreliable for shared blocks).
  • The image-metadata sync writes blocks only — it does not reverse-map to dockets or bump docket last_updated_flag; the blocks sync feed propagates the change (the PRD left the docket signal implicit).
  • DocketService.sync_plate_image_metadata_in_dockets_for_short_uid kept its name/home, calling block_repo directly (not block_service, to avoid its linked-image / inline-MCQ side-effects); the image-bank bg_task caller is unchanged.
  • DocketProjection.body / DocketListProjection.body retained as the runtime hydration target; only bode dropped from projections. No response-shape change.

M3 — Remove PATCH /reorder. Endpoint + reorder_by_id + reorder_docket removed; dockt_taxonomy_sort_order_idx (delete sibling-cascade) and the separate block /reorder-blocks endpoint retained. No deviations.

Verification. Full keystone suite green (235 passed). New tests: src/tests/block/test_block_image_lookup.py (content-regex lookup) and src/tests/v2/test_docket_image_sync.py (block patch on change / skip on no-change).

Migration

  • M1: code-only, no data migration. Three stale body→block migrations retained as-is (not run).
  • M2: no data migration at field-drop time — existing body/bode left in stored docs (schemaless, ignored). Optional $unset cleanup migration may follow once block_ids coverage is re-confirmed.
  • M3: code-only, no data migration; sort_order and its counter migrations unaffected.