Owner Burhan

Image Bank — Linked Entities & Taxonomy Filter — PRD

Context

The Image Bank already ships — editors create images (with license), list and search them, and attach them across MCQs and Blocks. This PRD is a delta on that shipped feature: it adds the same linked_entities relation that the Flowchart and Table Repository already have, so an image knows everywhere it is used, can be filtered by taxonomy of usage, and is protected from being archived out from under live content.

It does not re-spec image creation, the license list, upload/CDN, or the existing list/search UI — those are unchanged.

Scope

MVP — purely additive to the existing Image Bank:

  1. linked_entities (“where used”) — an image surfaces every Block/MCQ it is attached to. One image is referenceable by many entities.
  2. usage_context — per linkage, record where inside the entity the image sits (e.g. an MCQ’s question vs solution), so an editor knows what they are about to unlink.
  3. Denormalized taxonomy usageused_in_l1_ids / used_in_l2_ids / used_in_l3_ids on the image, derived from each linked entity’s immutable chain (Option 1; same as Flowchart/Table).
  4. Inverse pointerlinked_image_ids on both Block and MCQ (the inverse side of Image.linked_entities; both stored to save read cost).
  5. Filter the list by L1 / L2 / L3 usage — reads the denormalized fields directly; an image used under multiple taxonomies appears under each.
  6. Archive (same as Flowchart/Table) — add a status enum and an archive operation that sets status = ARCHIVED (reversible, non-destructive — keeps the S3 assets), blocked whenever the image still has linked entities; the editor is shown the full “where used” list and must remove all linkages first. This replaces the current destructive delete as the archive action.
  7. One-time backfill (both sides) — populate linked_image_ids on existing Blocks/MCQs by parsing their content, then invert into each image’s linked_entities and derive used_in_l*_ids.

Data Model (requirements-level)

Consistency rule (decided): all denormalized fields are maintained synchronously through a single write path — one operation updates the entity’s image reference, linked_image_ids, Image.linked_entities, and used_in_l*_ids together. No async / background recompute. The only trigger is attach / detach of an image to a Block/MCQ. This rests on a product assumption (the same one Flowchart/Table already make): a linked entity’s taxonomy chain is immutable — Block→Docket moves and Docket re-parenting don’t happen, so used_in_l*_ids never needs re-deriving after attach. ⚠️ Verified against code: this is not enforced today — a Docket’s taxonomy_ids and a Block’s docket_id are writable. If the assumption is ever violated, used_in_l*_ids silently goes stale (see Risks).

ID convention (decided, matches the Table model): every id used in a relation is the short_uid, never the Mongo _id. This mirrors TableLinkedEntity.content_short_uid: str exactly. Note the existing repos disagreeFlowchartLinkedEntity.content_id stores the Mongo ObjectId, while Table stores the short_uid; we deliberately follow Table. linked_image_ids mirrors the existing linked_table_ids / flowchart_ids, which are both List[str] of short_uids. (used_in_l*_ids are the exception — they are taxonomy-node ObjectIds, List[PydanticObjectId], same as Flowchart/Table.)

Image — fields added by this PRD (on ImageBankDBModel)

FieldTypeNotes
linked_entitiesarray[{ content_short_uid, content_type, usage_context? }] — where the image is attached (denormalized “where used”). Mirrors TableLinkedEntity: content_short_uid is the Block’s / MCQ’s short_uid (string); content_type is an int enum (MCQ = 1, BLOCK = 2).
usage_contextenum (optional, per linkage)Where inside the entity the image sits: question (IMAGE_MCQ thumbnails or a rich-title image node) | solution for an MCQ, or content for a Block. (MCQ options do not hold images.) Optional — absence means “referenced, location unspecified”.
used_in_l1_ids / used_in_l2_ids / used_in_l3_idsarrayDenormalized taxonomy usage — List[ObjectId] of taxonomy-node ids (same type/role as Flowchart/Table). Derived per linked entity: an MCQ exposes taxonomy_ids ([L1, L2, L3]) directly; a Block resolves via docket_id → docket.taxonomy_ids. Powers the L1/L2/L3 filter.
statusenumAdded by this PRDImageStatusEnum (PUBLISHED = 1 | ARCHIVED = 2), a two-state lifecycle (no DRAFT) mirroring FlowchartStatusEnum / TableStatusEnum. Default PUBLISHED.

Why a new status field: ImageBankDBModel has no status enum today — its only lifecycle op is a destructive delete (is_deleted + S3 wipe; see Archive). To make image archive behave exactly like Flowchart and Table (a reversible, non-destructive published/archived toggle, guarded by linkage), this PRD adds a matching ImageStatusEnum. Existing fields (short_uid — prefix IMG, course_id, license, original / variants, is_public, is_deleted, audit) are unchanged.

Block & MCQ changes

  • linked_image_ids: List[str] — on both Block and MCQ — denormalized list of image short_uids referenced by that entity (the inverse side of Image.linked_entities; both stored to save read cost). Mirrors the existing flowchart_ids / linked_table_ids (both List[str]). Introduced by this PRD — added to the Block/MCQ schema and seeded for existing data by the backfill (Phase A).
    • For an MCQ, linked_image_ids is the union of its image surfaces: thumbnails (the IMAGE_MCQ question images — these already exist on the model) plus image nodes in the rich title / solution. (Options hold no images.) linked_image_ids is a superset of thumbnails, not a replacement for it.

Single Write Path & Consistency

Mirror the existing FlowchartService.sync_linked_entities_for_content / TableService.sync_linked_entities_for_content (called on Block/MCQ save: diff the entity’s old vs new image short_uid list, attach to newly-referenced images, detach from de-referenced ones, recompute taxonomy each time). One write path updates, together:

  1. the entity’s image references — PlateJS img / inline_image nodes (keyed by the image short_uid) in Block content and MCQ rich title / solution; the IMAGE_MCQ thumbnails field; and a Block whose type is an image (bare short_uid in content),
  2. the entity’s linked_image_ids,
  3. each affected image’s linked_entities (with usage_context),
  4. each affected image’s used_in_l*_ids, re-derived from its (now-updated) set of linked entities.

There is no second trigger and no background job. The existing plate_image_bank_sync.py util already maps PlateJS image nodes ↔ image-bank rows and is the natural place to source the referenced short_uids.

”Where Used”

From an image, the editor can see every Block/MCQ it is attached to, each with its usage_context (e.g. “MCQ MCQ7K2… — solution”). This view is the unlink surface the archive guard depends on.

L1 / L2 / L3 Usage Filter

The image list can be filtered by L1 / L2 / L3, matching the Table Repository and Flowchart listings. The filter reads the denormalized used_in_l1_ids / used_in_l2_ids / used_in_l3_ids on the image directly — no join through linked entities. An image used under multiple taxonomies appears under each: it matches when used under any of the selected L1/L2/L3 values.

Archive (Lifecycle & Cascade)

Image archive is brought in line with Flowchart and Table: a reversible, non-destructive status toggle, guarded by linkage. Mirror FlowchartService.archive / TableService.archive.

  • archive operation. A new admin endpoint + ImageBankService.archive that sets status = ARCHIVED and clears used_in_l*_ids defensively. It keeps the S3 assets (reversible) — unlike the current delete.
  • Archive guard. Block archive whenever linked_entities is non-empty — raise a new IMAGE_ARCHIVE_BLOCKED error (mirroring FLOWCHART_ARCHIVE_BLOCKED / TABLE_ARCHIVE_BLOCKED) that returns the full “where used” list. The editor must remove all linkages first, then archive. All three repos now agree — Flowchart, Table, and Image block archive on any linkage (enforced in code; flowchart doc corrected to match).
  • Cascade on linked-entity archive. If a linked Block or MCQ is archived, remove that linkage from the image and re-derive used_in_l*_ids — through the same single write path that maintains linked_entities.
  • Re-publish (unarchive). No dedicated endpoint — mirror the siblings, which re-publish through the generic update path with an explicit status override (the Flowchart/Table update signatures take status: Optional[...] = None precisely “to re-publish an archived” entity). So image unarchive = update with status = PUBLISHED; assets are intact because archive never deleted them.
  • Listing. Mirror the siblings’ published_only flag: the client path serves PUBLISHED only; the admin (av1) path leaves it off so editors can still preview/manage archived images.

The destructive delete stays as-is (decided). Today the admin UI’s Archive button (imageBankApi.archiveImage) calls DELETE /image_bank/{id}ImageBankService.delete_by_id, which sets is_deleted = True and wipes the S3 assets (“This action cannot be undone”). That endpoint and service are left unchanged — retained as a separate permanent-delete. Only the Archive UI action repoints to the new non-destructive archive endpoint; the backend delete is untouched.

Backfill (One-Time Migration)

Existing images, Blocks, and MCQs predate this relation. Neither side exists on legacy datalinked_image_ids is not yet populated on old Blocks/MCQs, and images have no linked_entities. The migration therefore populates both sides, sourced from the content itself:

  1. Phase A — populate linked_image_ids on every existing Block & MCQ. Collect the image short_uids actually referenced by parsing: PlateJS img / inline_image nodes (Block content, MCQ rich title / solution), the MCQ thumbnails field (IMAGE_MCQ), and Block-as-image content. Reuse plate_image_bank_sync.py’s node walk. Write the union to linked_image_ids.
  2. Phase B — invert into Image.linked_entities. From the now-populated linked_image_ids, build each image’s linked_entities ({ content_short_uid, content_type }, where content_short_uid is the Block’s / MCQ’s short_uid; usage_context populated where derivable from the surface in Phase A, omitted otherwise).
  3. Phase C — derive taxonomy. Compute used_in_l1_ids / used_in_l2_ids / used_in_l3_ids per linked entity (MCQ → taxonomy_ids; Block → docket_id → docket.taxonomy_ids).

Existing images are set to status = PUBLISHED (the enum default) by the same migration. Going forward, the Single Write Path keeps both sides + taxonomy maintained; the backfill runs once.

Delivery Milestones

#MilestoneOutcomeStatusPlan
1linked_entities + inverse linked_image_idsOn attach/detach, an image’s “where used” and the entity’s linked_image_ids stay in sync via the single write pathcompleteBE T2,4,7,8 · FE T2–5
2usage_contextEach linkage records where inside the entity the image sitscompleteBE T1,2,4,8 · FE T1,4
3”Where used” viewAn image shows every Block/MCQ it is attached to, with contextcompleteBE T5 · FE T8
4Denormalized taxonomy + L1/L2/L3 filterEditors filter the image list by taxonomy of usage (denormalized)completeBE T4,5 · FE T1,3,7
5status + non-destructive archive + cascadeAdd ImageStatusEnum; images archive reversibly (assets kept) only when unlinked; UI Archive repointed off the destructive delete; archiving a linked entity detaches itcompleteBE T1,2,4,5,6,8 · FE T1,3,6,8
6Backfill (both sides)Existing Blocks/MCQs get linked_image_ids populated, then existing images get linked_entities + used_in_l*_idscompleteBE T9,10

Plan key: BE = backend plan tasks (keystone/docs/superpowers/plans/2026-06-08-image-bank-relation-backend.md); FE = admin plan tasks (keystone-web/plans/image-bank-relation-admin-frontend.md). Full design in keystone/docs/superpowers/specs/2026-06-08-image-bank-relation-design.md. See Technical Details (as built) below.

Open Questions

  • None remaining — all resolved during PRD review. (The earlier Flowchart-doc archive-rule mismatch has been fixed: flowchart-prd.md now blocks archive on any linkage, matching Table, Image, and the shipped code.)

Risks

RiskLikelihoodImpactMitigation
Ref-node ↔ denormalization drift. Image nodes in a Block/MCQ’s content and the denormalized lists (linked_image_ids, linked_entities, used_in_l*_ids) can diverge if updated separately.MediumMedium — wrong “where used” / filter resultsSingle write path: one atomic operation updates the image ref + all denormalized lists together; validate on save.
Backfill parser misses a reference shape. Phase A derives linked_image_ids by parsing content; any image-reference shape the parser doesn’t handle (an unusual PlateJS node, the IMAGE_MCQ thumbnails field, a Block-as-image) is silently dropped, so that image looks unlinked (and wrongly archivable).MediumMedium — silent under-linkingCover the full surface set listed in Single Write Path / Backfill; spot-check backfill output against known multi-use images; re-runnable migration so a missed shape can be re-swept.
Stale used_in_l*_ids if the immutability assumption breaks. Taxonomy is denormalized only on attach/detach. A Docket’s taxonomy_ids and a Block’s docket_id are writable in code (no guard found), so a taxonomy edit or block move would leave used_in_l*_ids stale — the same latent gap Flowchart/Table already have.LowMedium — wrong filter resultsConfirm the “no moves / no re-parenting” rule is real (product) and ideally enforce it, or add a re-derive trigger on Docket-taxonomy / Block-docket change (would benefit all three repos).
Archive friction. A heavily-reused image can only be archived after every reference is unlinked by hand.MediumLow — editor effortThe “where used” view lists every linkage so the editor can unlink quickly before archiving.

Technical Details (as built)

Implemented on branch feature/image-bank-relation — keystone 2885dc0e (linked-entities, taxonomy filter, non-destructive archive + tests + backfill) and f75589a9 (usage-context coercion + legacy visibility); keystone-web 49e2c7d (linked_image_ids on save + gallery taxonomy filter & archive). Design spec: keystone/docs/superpowers/specs/2026-06-08-image-bank-relation-design.md; backend plan: keystone/docs/superpowers/plans/2026-06-08-image-bank-relation-backend.md; admin plan: keystone-web/plans/image-bank-relation-admin-frontend.md. Built by applying the shipped Table linkage / taxonomy / archive machinery onto the existing ImageBankDBModel (additive only) — image payload, license list, upload/CDN, and the existing list/search/upload/edit UI are unchanged. Covers keystone backend + keystone-web admin; no new PlateJS node — images already attach via the inline-image node (inline_image/img) and the IMAGE_MCQ thumbnails field. Backend tests in src/tests/image_bank/ (schemas, linkage, repository, backfill, paged-scan).

Naming: PRD vs code

PRD termCode
Image entity (existing)ImageBankDBModel (collection image_bank) / ImageBankProjection, short_uid prefix IMG
linked_entities[]ImageBankLinkedEntity { content_short_uid, content_type, usage_context? }
content_typeImageBankLinkedContentTypeEnum (MCQ=1, BLOCK=2)
usage_contextImageUsageContextEnum (QUESTION=1, SOLUTION=2, CONTENT=3), optional per linkage
status (PRD ImageStatusEnum)ImageBankStatusEnum (PUBLISHED=1, ARCHIVED=2), default PUBLISHED
Block/MCQ inverse linklinked_image_ids: List[str] (image short_uids)
image-reference surfacesinline inline_image / img Plate nodes + IMAGE_MCQ thumbnails + Block-as-image content

Decisions & deviations from the PRD (as built)

  • linked_entities stores content_short_uid, not content_id. Follows Table (the PRD’s locked “ID convention”), not Flowchart’s Mongo _id — symmetric with the forward linked_image_ids and the admin’s by-short_uid lookups, so the “where used” modal needs no id→short_uid resolution. (ImageBankLinkedEntity docstring documents the divergence from FlowchartLinkedEntity.)
  • Route prefix is /image-bank (hyphen), not /image_bank. New endpoint: POST /image-bank/{id}/archive.
  • status enum is ImageBankStatusEnum (PRD’s informal ImageStatusEnum), mirroring the ImageBankDBModel name. ImageBankLinkedContentTypeEnum (MCQ=1,BLOCK=2); ImageUsageContextEnum (QUESTION=1,SOLUTION=2,CONTENT=3).
  • IMAGE_ARCHIVE_BLOCKED = 6106 — contiguous with the existing image-bank error block (6100–6105), not a new 9500 block. Frontend mirrors it as IMAGE_ARCHIVE_BLOCKED_CODE = 6106 (src/constants/image-bank.ts).
  • Live single-write-path is frontend-derived (mirror Flowchart/Table; confirmed 2026-06-08): the admin’s collectImageIds walker sends linked_image_ids on MCQ/Block save and ImageBankService.sync_linked_entities_for_content diffs old-vs-new. The PRD’s “source short_uids from plate_image_bank_sync.py” applies to the one-time backfill — that util only patches known nodes; the collector used by the backfill is DocketService._extract_image_short_uids_from_content.
  • usage_context is an additive side-channel: the admin sends an optional image_usage_contexts: { short_uid → context } alongside the flat linked_image_ids (keeping the id diff identical to siblings); absence ⇒ usage_context = None (“unspecified”). As-built hardening (f75589a9): invalid/out-of-range context values coerce to None rather than failing the Block/MCQ save they ride along with.
  • Legacy client visibility (f75589a9). The published_only client guard filters status != ARCHIVED (negative-space match), not status == PUBLISHED — so pre-feature images (no status field ⇒ Mongo null) stay visible to clients without depending on the backfill having run.
  • Destructive delete retained. DELETE /image-bank/{id}ImageBankService.delete_by_id (wipes S3) is unchanged; only the admin Archive button repoints to the new endpoint, and the destructive path stays as a separate action (imageBankApi.archiveImage kept alongside the new imageBankApi.archive).

Entity (src/models/image_bank_models/ — additive)

Existing ImageBankDBModel(BaseBeanieDocumentModel), collection image_bank, short_uid prefix IMG. Adds: linked_entities: List[ImageBankLinkedEntity], used_in_l1_ids / used_in_l2_ids / used_in_l3_ids (List[PydanticObjectId], denormalized taxonomy), status: ImageBankStatusEnum (default PUBLISHED) — plus the same fields (Optional) on ImageBankProjection. Existing fields (original/variants, license, is_public, is_deleted, audit) unchanged.

Adds 5 indexes (imgbnk_ prefix) to the existing 6: (course_id, status); linked_entities.content_short_uid (reverse-cascade lookup); (course_id, used_in_l{1,2,3}_ids) (L1/L2/L3 filter).

Cross-entity linkage (Block & MCQ)

  • Block and MCQ each gain linked_image_ids: List[str] (image short_uids, default_factory=list) beside the existing flowchart_ids / linked_table_ids. For an MCQ it is the union of its image surfaces (thumbnails + inline images in question/solution_admin); linked_image_ids is a superset of thumbnails, not a replacement.
  • Frontend-derived linking. keystone-web’s new collectImageIds walker extracts short_uids from inline_image/img nodes on save (union with thumbnails) and sends linked_image_ids in the av1 Block/MCQ create/update payload — the backend never parses Plate JSON on the live path.
  • Single synchronous write path. ImageBankService.sync_linked_entities_for_content(course_id, content_short_uid, content_type, old_image_short_uids, new_image_short_uids, usage_contexts?) — called by Block/MCQ create+update via the unified service container (already registers image_bank_service), slotted beside the existing flowchart/table sync at the same 6 call sites — diffs added/removed short_uids, adds/removes the {content_short_uid, content_type, usage_context} reverse entry, recomputes used_in_* in one update_by_id per affected image. No Celery.
  • Taxonomy derivation (_derive_taxonomy_usage, batched like Table): MCQ short_uids → mcq.taxonomy_ids; Block short_uids → distinct docket_ids → one batched docket_repo.get_by_idsdocket.taxonomy_ids ([L1, L2, L3], deduped). Orphan blocks (no docket_id) silently skipped. Re-derived only when linked_entities changes (taxonomy chains assumed immutable — see Risks; not enforced in code today).

Archive & cascade

  • Archive blocked whenever ≥1 link exists. ImageBankService.archive raises IMAGE_ARCHIVE_BLOCKED while linked_entities is non-empty; only 0 links archive (defensively zeroing used_in_*, keeping S3 assets). Editor unlinks all references first via the “where used” list. Re-publish via update with status = PUBLISHED.
  • Reverse cascade. Block delete and MCQ archive/delete call unlink_content_from_images(content_short_uid), removing that content from every linking image and recomputing used_in_*. MCQ archive also clears its own linked_image_ids (alongside the existing flowchart_ids/linked_table_ids) in the same status write.

Surface & semantics

SurfaceRoutesNotes
av1 (admin, write)new POST /image-bank/{id}/archive; existing POST /upload, GET /, GET /list-updated-at-bidirectional (+ new status / used_in_l{1,2,3}_ids filters), GET /filters (+ statuses), GET /{id}, PATCH /{id} (un-archive via status); DELETE /{id} retained (destructive permanent-delete)JWT + editor/admin role; admin GET/list can preview archived (published_only off).
v2 (web, read)GET /v2/image-bank/{id}published_only=True — archived images hidden from clients.
v1 (mobile)none today (empty module)out of scope unless product asks.
  • av1 ImageBankDetailsResponse gains linked_entities, used_in_l*_ids, status; ImageBankFilterParams gains status + used_in_l{1,2,3}_ids; ImageBankFilterResponse gains statuses. used_in_l*_ids back the admin filter only.
  • Listing/search remains MongoDB-filter based, no Typesense: existing bidirectional keyset cursor on (updated_at, _id) (ImageBankTimestampCursor).
  • New error code: IMAGE_ARCHIVE_BLOCKED = 6106 (in the existing 6100–6299 Image Bank block).

Migration

One-time, idempotent backfill src/core/migrations/jun_08_26/backfill_image_bank_linkage.py (--dry-run), mirroring jun_03_26/backfill_flowchart_taxonomy_usage.py. 3 phases: (A) paged scan of blocks + mcqs → populate linked_image_ids from content (reusing DocketService._extract_image_short_uids_from_content) ∪ thumbnails ∪ Block-as-image content; (B) invert into each image’s linked_entities; (C) derive used_in_l*_ids (reuse _derive_taxonomy_usage). Sets legacy images status = PUBLISHED. Re-runnable so a missed reference shape can be re-swept (Risk: backfill parser gap). Block/MCQ repos gained a list_batch_by_course_after_id paged-scan helper (mirrors the one on image_bank_repository). Covered by src/tests/image_bank/test_image_bank_backfill.py + test_paged_scan.py. (The legacy-visibility guard above means clients see pre-backfill images correctly even before this runs.)