CAP-2031 · same-bucket migration

CAP-2031 Change Set

Everything the branch changed, and what it did to the database when it ran. The plan is on Same-Bucket Running Order; the audit that checked it against the data is on Same-Bucket Path Audit; this page is the result.

Repositories
4API, web app, migrations, thumbnails
Commits
12Plus one new service
Migrations
8All applied
Rows rewritten
966,501Across five tables
Still open
4Named at the foot of this page

The database, after

every migration applied
MigrationWhat it didRowsLeft
101441Archived equipment_models.images to images_old and cleared the values that were not file names — scraped vendor JSON, and the string {}1,6070 left
101442Widened the path columns; five became varchar[], each element trimmed and the empties dropped—applied
101443Widened warranties.file to TEXT, dropping and restoring the three triggers that depend on the column—text
101444Equipment model and category pictures4,1350 bare
101445The remaining eleven columns, plus docusign_requests.attachments by request type14 + 60 bare
101446Ten tables holding a file name inside JSON—applied
101447Inventory stickers, trigger suspended, then camp_inventory_items.images rebuilt once709,8010 bare
101448The warranty search key takes the file's name rather than its address—applied

The sticker migration also rebuilt camp_inventory_items.images for 252,545 inventories. Twenty were left holding bare names: every one has no live sticker row behind it, so the trigger would not have rebuilt them either.

What changed, by repository

12 commits

capExpertAPI · 5 commits

backendApi

110 files, +2,860 −1,111

  • Uploads staged in temp/ and filed on save, through Redis rather than a table
  • Records store the complete path, not a bare name
  • Thirteen generated-document uploads now name their module
  • make-file-public authorises from the upload grant
a35bc53 · 5efec37 · 2535f46 · 88aa4d3 · fba886e

capExpertApp · 3 commits

Web front end

100 files, +1,042 −1,016

  • Upload sites send the key the presign returned
  • One path rule resolves every stored file
  • Thumbnails asked for only where one is written
  • The cropper opens on the file just picked
bfc4dd3 · ce4077b · 309e62a

dbMigrations · 4 commits

Database

14 files, +1,383

  • The backfill split into four migrations, ordered around the schema changes
  • Statement builders shared from one helper
  • The warranty trigger and its functions checked in
498b977 · 8c40e24 · d6a2180 · 3f0169d

New service

thumbnailService

Node 22, GCS finalize trigger, 53 tests

  • Thumbnails for the whole img/ tree, not one folder
  • A staged thumbnail is moved, never regenerated
  • An original under 50 KB is copied as its own thumbnail
  • Public and thumbnail are separate decisions
not yet deployed

The seven root causes

what the audit found
  1. The complete path was discarded.Fourteen sites sent a bare file name where the upload had already returned an address. Every one of them was a live 404 once the object moved.
  2. Paths were prefixed twice.Five sites re-prefixed a value already in the new tree, moving objects back out of it.
  3. Four modules never committed at all.DeNovo documents, account formularies, divestiture checks and the shipping-quote logo stayed in temp/ forever.
  4. The commit path could throw.The record is durable by the time it runs, so a Redis fault now leaves the files staged — which still read — instead of reaching the caller.
  5. An empty array is truthy.warrantyDocument is a varchar[], so a guard written for an empty string passed on every row.
  6. Any caller could expose any object.make-file-public took a bare path, which cannot say who asked. It authorises from the grant left when the upload URL was issued.
  7. A CER document was addressed as a PA one.The document is uploaded to the folder its type names, but the column was written with the PA folder either way, so six rows pointed at an object that was not there.

Two bugs the tests caught before deploy

not found by reading
Thumbnails written to a top-level prefix
The path builder dropped the img/ segment, so every thumbnail would have landed at CAP-1234/assets/thumb/a.jpg — a prefix nothing reads. It would have looked like it worked.
A backfill that prefixed 6,438 values
The JSON walker addressed changes[].new by position, so a status, a manufacturer and a zip code were prefixed as though they were files. Restricted to the sibling key, the same 2,456 rows give 104 rewrites — 15 quotes and 89 attachments.

Still open

4 items
  1. The thumbnail service is not deployed.Written and tested, and it needs a check that capexpert-local is not on uniform bucket-level access — makePublic fails with 400 there while the run still looks green.
  2. Nothing backfills existing thumbnails.The service is trigger-only, so every image already in the bucket stays without one until it is re-uploaded.
  3. Two upload sites keep the legacy folder.The primary docusign helper, whose callers have no account in scope, and one marketplace invoice path. The QR code is deliberate: it is printed onto physical labels and must never move.
  4. The copier has not been built.Step 5 of the plan. Records now hold complete paths, but nothing yet moves the objects that are still in the legacy folders.

The shape flag and the legacy-read logging from steps 4 and 6 are also unbuilt — new construction rather than repair, and deliberately left until the copier exists.