Docket Module Cleanup — PRD
Context
The docket module already ships — dockets collection with full admin CRUD (av1), a mobile sync feed (v1), and web read/list/backlinks (v2). Content has already migrated from the embedded body blocks to a first-class blocks collection referenced by docket.block_ids (per the shipped Block Sync PRD); block_ids → blocks is now the canonical source of truth for docket content.
This PRD is a cleanup / slim-down delta, not a feature. It does three things and defers a fourth:
- Remove accumulated dead code in
DocketService/DocketRepository(zero-caller and transitively-dead methods, an orphaned search-helper cluster) plus two deregistered task-route stubs. - Retire the legacy
body/bodestorage fields now that blocks have replaced them — storage-only, with no change to any response shape. - Remove one unused admin endpoint (
PATCH /dockets/{docket_id}/reorder) and its now-orphaned service/repo functions. - (Deferred) Consolidate the docket CRUD surface to one GET / one POST / one PATCH — handled in a separate backend spec + plan.
It does not re-spec the docket feature, the blocks model or sync protocol, the sort_order mechanism, or the docket-family lookalike routers (docket_link, docket_review, mcq_docket_link, bookmark buckets) — those are separate modules with their own models/services.
Scope guardrail (decided). Every change here is internal: no client-facing request or response contract changes in M1–M3. Removing
bodyfrom response contracts, and any endpoint consolidation, are explicitly out of scope for this PRD (the latter is M4, deferred).
Scope
MVP — a four-phase docket-module slim-down, sequenced safest-first:
- M1 — Dead-code removal. Delete zero-caller / transitively-dead service & repo functions, the orphaned full-text-search helper cluster, and two deregistered task-route stubs.
- M2 — Retire
body/bode(storage-only). Drop the two DB fields and thebodeat-rest encryption, stop writing them on create/edit, and rewire the two image paths that still read/writebodyover to theblockscollection. Responses keep hydratingbodyfrom blocks — no client change. - M3 — Remove unused
PATCH /reorder. Delete the endpoint and its orphanedreorder_by_id/reorder_docketchain. - M4 — API consolidation (deferred). Target shape only; design happens in a separate backend spec/plan.
M1 — Dead-code removal
DocketService.get_by_idis NOT dead — do not remove (decided). It is reached via the unified-service-container aliasservices.docket_service.get_by_id(...)(3 live callers: av1 block routes, v2 blocks-backlinks, bookmark create). A full sweep ofservices.docket_service.<method>aliases found only this method (and the already-liveextract_inline_mcq_short_ids_from_body) reached indirectly — so no other dead-flagged symbol is secretly alive.
Migrations kept as-is (decided). The three stale body→block migrations (
jul_22_25/migrate_dockets_from_html.py,jul_25_25/migrate_docket_blocks_to_blocks_table.py,docket_content_cleanup.py) reference the long-removednew_bodyfield and would error if run; none is registered with a runner. They are retained as immutable history — M1 does not touch them.
Deregistered task-route stubs removed (decided). The two av1
/tasksendpoints below never dispatch work (one literally returns"Task is deregistered."). Remove both handlers + their route registration. The sibling task routes that do dispatch work are untouched.
Removal manifest
| Layer | Symbols to remove | Location |
|---|---|---|
| Service fns (10) | increment_mcq_stats, update_all_docket_mcq_counts, get_by_block_id, get_max_sort_order, list_by_updated_at_with_blocks, and the PR #283 linked-MCQ cluster: get_by_id_with_linked_mcqs, get_by_id_or_short_uid_with_linked_mcqs, list_with_linked_mcqs, list_by_updated_at_with_linked_mcqs, + transitively-dead populate_linked_mcqs | src/services/docket_services.py |
| Repo fns (7) | list_by_id, get_max_sort_order, get_by_block_id, increment_mcq_stats, update_all_docket_mcq_counts, update_all_docket_linked_mcq_ids, update_all_docket_linked_docket_ids | src/repository/docket_repository.py |
| Repo private helpers (4 — orphaned search cluster) | _determine_match_info, _determine_search_matches, _extract_context, _calculate_relevance_score | src/repository/docket_repository.py |
| Task-route stubs (2) | av1_tasks_post_update_all_docket_mcq_counts (:62), av1_tasks_post_update_all_docket_linked_mcq_ids (:298) + any request/response model used only by them | src/api/av1/tasks/routes.py |
- Dead schemas: 0 — dead methods reused live/shared response models; no docket schema class becomes orphaned.
- Test impact: 0 — no test across the suite references any removed symbol (
reorder/get_max_sort_ordertest hits belong to the exam/mcq modules). - The search-helper cluster is the remnant of a
search_dockets/search_suggestionsfeature that no longer exists in the repo — removed as a unit.
M2 — Retire body / bode (storage-only)
Blocks are the source of truth (decided).
block_ids→ theblockscollection is canonical.bodyis a legacy mirror, andbodeis a dead at-rest cipher ofbodyread by no backend code. Evidence: reads already hydratebodyfrom blocks at runtime (docket_services.py:851-860,:1360-1376); block ops (attach/detach/reorder) maintainblock_ids+inline_mcq_short_idsand never touchbody/bode; blocks carry their own at-rest cipherconte.
Storage-only removal (decided). Drop the DB
body+bodefields and stop writing them; responses keep hydratingbodyfrom blocks at runtime, so no response shape changes. Removingbodyfrom the av1/v2 detail response contract is out of scope (it would be a breaking client change; tracked separately, not here). DB-field removal and response-contract removal are decoupled precisely because the response is rebuilt from blocks.
Migration precondition (verified). The body→block migration (
apr_15_26/1_docket_body_to_block.py) has run to completion in all environments and every docket is represented byblock_ids. ⚠️ This is an assertion about production data, not enforced in code — see Risks.
Create authors no content (decided). Blocks are added only during edit, never at creation. The create request drops
body; a newly created docket starts with no content and gains blocks via the edit/block endpoints. (Behaviour change: dockets can now be created “empty”.) The update/edit request also dropsbody; edits manage content viablock_ids+ block endpoints (inline-MCQ extraction already has a blocks path:inline_mcq_short_ids_for_block_ids).
Existing Mongo data left in place (decided). Drop the Pydantic fields and stop writing; existing
body/bodevalues in stored documents are left untouched (Mongo is schemaless — ignored on read). An optional$unsetcleanup migration may follow later; no hard delete now.
Removal buckets
(a) Drop outright — no successor needed
| Item | Location |
|---|---|
DB bode field def | models/docket_models/db_model.py:236-242 |
DB body field def (last, after writes stop) | models/docket_models/db_model.py:231-235 |
build_docket_bode_column() | core/encryption/field_encryption_utils.py:104-116 |
bode compute on create | repository/docket_repository.py:195-198 |
bode refresh on update | repository/docket_repository.py:634-644 |
DocketProjection.bode / DocketListProjection.bode | models/docket_models/projection_model.py:59-62, :293-296 |
av1 response bode field (always None — no route sets encrypted=True) | api/av1/docket/schemas.py:209-212 |
(b) Rewire to blocks first — the real work
| Item | Location | Rewire |
|---|---|---|
| Image-content lookup | repository/docket_repository.py:3150-3184 (list_ids_by_image_short_uid_in_body) | Replace the {body:{$elemMatch:{content:{$regex}}}} query with a regex over blocks.content + distinct(docket_id). |
| Image-metadata sync writer | services/docket_services.py:673-755 (sync_plate_image_metadata_in_dockets_for_short_uid) | Patch + bulk-update the blocks collection instead of reading/writing docket body. |
body on create | request schema + routes.py:79-93 + create service | Drop body from the create request/service; store no content. |
body on update | request schema (schemas.py:779-901) + routes.py:314-331 + update service (docket_services.py:189-327) | Drop body from the update request; manage content via block_ids. |
(c) Already safe — no action
- v1 sync, av1 list, v2 list already exclude
body. - Inline-MCQ extraction already has a blocks-native path (
inline_mcq_short_ids_for_block_ids), so create/update switch to it cleanly. The body-basedextract_inline_mcq_short_ids_from_bodyloses its create/update callers but is still referenced by thedec_25_25migration, so it is retained (consistent with “migrations kept as-is”), not removed in this PRD. - The body-content image walker
_extract_image_short_uids_from_contentand theget_..._with_images_and_mcqshydration path stay — they already read from blocks / are format-agnostic.
Data Model (requirements-level)
DocketDBModel loses two fields; everything else is unchanged.
| Field | Change | Notes |
|---|---|---|
body | Removed | Legacy embedded blocks. Content is now block_ids → blocks. Responses hydrate a body-shaped payload from blocks at runtime (unchanged). |
bode | Removed | At-rest cipher of body; read by nothing. Blocks carry their own at-rest cipher conte. |
block_ids | unchanged | Canonical ordered content references. |
sort_order | unchanged | Retained — used by create, delete-cascade, and list ordering (see M3). |
M3 — Remove unused PATCH /dockets/{docket_id}/reorder
Endpoint unused by clients (decided). The admin frontend defines no client method for it and has no
/reordercall-site — the docket editor reorders blocks locally and persists viaupdatewithblock_ids. The docket-level reposition endpoint is dead from the client side.
Index and
sort_orderretained (decided). Removingreorder_docketdoes not free thedockt_taxonomy_sort_order_idx (taxonomy_ids, sort_order)index — thedelete_by_idsibling-cascade (docket_repository.py:750-758) filters on exactly{taxonomy_ids, sort_order:$gt}. The code comment “used in reorder_docket” is incomplete; keep the index (and optionally fix the comment).sort_orderand its counter migrations are unaffected.
Removal cascade (self-contained)
| Item | Location |
|---|---|
Route handler reorder_by_id + the summary/description/operation_id block | api/av1/docket/routes.py:345-391 |
DocketReorderResponse import alias (it is just BaseMessageResponse, no dedicated schema) | api/av1/docket/routes.py:26 |
DocketService.reorder_by_id (only caller is the route) | services/docket_services.py:453-~507 |
docket_repo.reorder_docket (only caller is reorder_by_id) | repository/docket_repository.py:902-~1091 |
- Request body schema: none (uses query/path params:
course_id,docket_id,new_position). Dedicated response schema: none. Tests: none. Index changes: none. - Do not touch the separate
PATCH /dockets/{docket_id}/reorder-blocksin the block module — that is block reordering, out of scope here.
M4 — API consolidation (deferred)
Deferred to a separate backend spec + plan (decided). Target shape: docket CRUD should be exactly one GET (variations via query params), one POST, and one PATCH (variations via request data). The exact consolidation design and the reconciliation of the “4 PATCH” expectation against the actual surface are not specced here — they will be decided during a dedicated backend spec + plan after M1–M3 land.
Current docket surface (for reference when M4 is picked up):
| Surface | Routes |
|---|---|
| av1 | POST /dockets, GET /dockets, GET /dockets/{id}, GET /dockets/list-updated-at-bidirectional, PATCH /dockets/{id}, DELETE /dockets/{id} (and PATCH /dockets/{id}/reorder until M3 removes it) |
| v1 | GET /dockets/sync |
| v2 | GET /dockets/{id}, GET /dockets, POST /dockets/blocks/backlinks |
Delivery Milestones
| # | Milestone | Outcome | Status | Plan |
|---|---|---|---|---|
| 1 | Dead-code removal | DocketService/DocketRepository shed 10 service fns + 11 repo symbols; 2 deregistered task endpoints removed; no behaviour change. | ✅ complete (PR #773) | …/plans/2026-06-17-docket-m1-deadcode.md |
| 2 | Retire body/bode (storage-only) | DB body+bode and bode encryption gone; create/edit stop writing them; image paths read/write blocks; responses unchanged (still hydrated from blocks). | ✅ complete (PR #773) | …/plans/2026-06-17-docket-m2-body-bode.md |
| 3 | Remove PATCH /reorder | Unused docket-reorder endpoint + reorder_by_id + reorder_docket removed; index/sort_order retained. | ✅ complete (PR #773) | …/plans/2026-06-17-docket-m3-reorder.md |
| 4 | API consolidation | 1 GET / 1 POST / 1 PATCH for docket CRUD. | deferred — separate backend spec + plan | — |
Plan key:
BE= backend plan tasks in keystone (keystone/docs/superpowers/specs/+…/plans/), authored per milestone from this PRD via/plan. M4 gets its own spec + plan when picked up.
Sequencing. M1 → M2 → M3, safest-first. M1 is mechanical (dead code). M2 depends on the verified body→block migration precondition. M3 is independent of M1/M2 and may land in parallel. M4 is gated on M1–M3.
Open Questions
- M4 design — the consolidated GET/POST/PATCH contract and the “4 PATCH” reconciliation are deferred to a separate backend spec + plan (decided), not resolved here.
- Everything in M1–M3 is resolved during PRD review: migrations kept as history; task stubs removed;
body/bodestorage-only with create/edit droppingbody; image lookup →blocks.contentregex; reorder endpoint + chain removed; index/sort_orderretained; existing Mongobody/bodedata left in place.
Risks
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
Body→block migration not 100% complete. A docket with body content but empty/unset block_ids would render empty once body is dropped (migration filtered {'body.0':{$exists:true}}, skipping empty-body and post-migration dockets). | Low | High — silent content loss | Precondition verified by owner across all envs. Recommended belt-and-suspenders before dropping the field: a one-off scan for body-non-empty + block_ids-empty dockets returning zero; keep the optional $unset migration separate so body data survives until coverage is confirmed. |
Stale image lookup after rewire. The current image-content query scans possibly-stale body; the blocks-native replacement must target live block content. | Med | Med — sync patches wrong/no dockets | Move list_ids_by_image_short_uid_in_body and sync_plate_image_metadata... to the blocks collection together, before body writes stop, so the lookup reads the canonical source. |
Hidden caller via dynamic dispatch. A removed method is reached through a path static grep missed (e.g. another services.docket_service.* alias). | Low | High — runtime AttributeError | Alias sweep already run (only get_by_id + extract_inline_mcq_short_ids_from_body reached indirectly — both retained). Re-grep at implementation time; rely on the test suite + a smoke pass. |
Accidentally dropping a shared index. Treating dockt_taxonomy_sort_order_idx as reorder-only would slow the delete sibling-cascade. | Low | Med — slower deletes | Documented: the delete cascade uses it; index is retained. |
Touching the wrong reorder. Confusing docket /reorder (remove) with block /reorder-blocks (keep). | Low | Med — breaks block reordering | Explicitly scoped: only api/av1/docket/routes.py reorder is removed. |
Technical Details (as built)
Status: M1–M3 built — PR #773 → dev (open). Shipped in the keystone repo on branch fixture/docket-cleanup as 15 code commits (5 / 7 / 3 across M1 / M2 / M3) plus a docs commit. Full keystone test suite green (235 passed via pre-push hook); net code diff ≈ +253 / −1,985 across 12 files. Per-milestone specs at keystone/docs/superpowers/specs/2026-06-17-docket-m{1,2,3}-*-design.md and plans at …/plans/2026-06-17-docket-m{1,2,3}-*.md. M4 (API consolidation) remains deferred to its own spec + plan.
Naming: PRD vs code
| PRD term | Code |
|---|---|
| Docket entity | DocketDBModel (collection dockets) / DocketProjection, DocketListProjection |
| Docket service / repo | DocketService (src/services/docket_services.py), docket_repository / docket_repo alias (src/repository/docket_repository.py) |
| Legacy content fields | body (List[DocketBlockWithMcqStats]), bode (at-rest CryptoV2.1 ciphertext) |
| Canonical content | block_ids → blocks collection; per-block at-rest cipher conte |
bode builder | build_docket_bode_column (src/core/encryption/field_encryption_utils.py) |
| Image-content lookup (to rewire) | DocketRepository.list_ids_by_image_short_uid_in_body |
| Image-metadata sync (to rewire) | DocketService.sync_plate_image_metadata_in_dockets_for_short_uid |
| Unused reorder endpoint | av1_docket_patch_reorder_by_id → DocketService.reorder_by_id → docket_repo.reorder_docket |
| Retained reorder index | dockt_taxonomy_sort_order_idx (taxonomy_ids, sort_order) (also used by delete_by_id cascade) |
| Deregistered task stubs | av1_tasks_post_update_all_docket_mcq_counts, av1_tasks_post_update_all_docket_linked_mcq_ids |
Decisions & deviations from the PRD (as built)
M1 — Dead-code removal. Removed as specced: 10 service fns (incl. the linked-MCQ cluster *_with_linked_mcqs + populate_linked_mcqs), 7 repo methods, the 4-method search-helper cluster, and both task stubs. Kept DocketService.get_by_id and extract_inline_mcq_short_ids_from_body (latter still used by the dec_25_25 migration).
- Deviation: the PRD called both
/tasks/udmcand/tasks/udli“deregistered”. In fact onlyudlireturns"Task is deregistered.";udmcwas a no-op that returned a misleading"…has been triggered."message (dispatched nothing, called no service). Both removed; no functional impact.
M2 — Retire body/bode. DB body+bode, the bode builder, and the bode field on both projections and the av1 detail response removed; create/edit stopped writing them (dockets can now be created empty). Decisions/deviations (resolved with owner):
- Image lookup implemented as a content regex over
blocks.content, added toBlockRepositoryaslist_ids_by_image_short_uid_in_content— notlinked_image_ids(denormalized, stale-prone), and notblock.docket_id/distinct(docket_id)(origin-only; unreliable for shared blocks). - The image-metadata sync writes blocks only — it does not reverse-map to dockets or bump docket
last_updated_flag; theblockssync feed propagates the change (the PRD left the docket signal implicit). DocketService.sync_plate_image_metadata_in_dockets_for_short_uidkept its name/home, callingblock_repodirectly (notblock_service, to avoid its linked-image / inline-MCQ side-effects); the image-bank bg_task caller is unchanged.DocketProjection.body/DocketListProjection.bodyretained as the runtime hydration target; onlybodedropped from projections. No response-shape change.
M3 — Remove PATCH /reorder. Endpoint + reorder_by_id + reorder_docket removed; dockt_taxonomy_sort_order_idx (delete sibling-cascade) and the separate block /reorder-blocks endpoint retained. No deviations.
Verification. Full keystone suite green (235 passed). New tests: src/tests/block/test_block_image_lookup.py (content-regex lookup) and src/tests/v2/test_docket_image_sync.py (block patch on change / skip on no-change).
Migration
- M1: code-only, no data migration. Three stale body→block migrations retained as-is (not run).
- M2: no data migration at field-drop time — existing
body/bodeleft in stored docs (schemaless, ignored). Optional$unsetcleanup migration may follow onceblock_idscoverage is re-confirmed. - M3: code-only, no data migration;
sort_orderand its counter migrations unaffected.