← All reports

2026-11-12

SailScan: model accuracy worklist for Skippy

Date: 12 Nov 2026. From Alex, prepared by Claude after reading the whole Boats thread (3 Apr to 12 Nov, 6,582 messages), the Burst 001 report, the 13 Oct accuracy write-up, the 20 Aug guardrails report, and the Ultralytics docs and source.

Scope: things you can check, build or retrain to make the stripe model more accurate. No user taps, no burst capture. Everything is automated.

Rule: this is all sandbox work. Nothing ships to users before METS (17 to 19 Nov) without Alex's OK.

How to read it: Section 1 is what the history shows. Sections 2 to 6 are the work, in the order I would do it. Section 7 lists shelved ideas whose verdicts do not hold. Section 8 turns the older never-run ideas into numbered tasks (the lane-detection model is S1). Section 10 is the checklist to report against: every item in this file has an ID there. Appendix B is the full test ledger so nothing gets re-run by accident.

Revised 12 Nov, afternoon: Section 8 items now have task IDs and specs, the two Section 6 steps are M1 and M2, and Section 10 was added.

Second revision 12 Nov: audited against everything found in the search. Added Section 1.7, checks C9 to C13, retrain T12, pipeline jobs P7 and P8, tasks S11 to S15, two extra fair-test items in Section 6, and Section 11 (things recorded but not asked of you). Companion files in the project, same folder: SailScan-METS-Accuracy-Plan-2026-11-11.md and SailScan-Accuracy-Deep-Dive-2026-11-09.md.

Where I state something about the Ultralytics library I have checked it against their docs or source and say so. Where I state something about our own training config I could not see it, so it is written as a check for you to run.

1. What the full history shows

1.1 The normal score cannot see shape

The 13 Oct accuracy page defines a perfect photo as: every stripe found, no extra lines, and every line "inside a tight band around the person's line (3 percent of the photo's diagonal)". The 20 Aug guardrails page says the same: "closest line within 3% of the photo diagonal".

At the model's working size (960 x 720) that band is 36 px. You measured the tape at about 5 px (27 Oct, scale not stated; if that was at upload size the band is wider still in tape widths). So the band is about seven tape widths each side. A top stripe is about 100 px long with about 10 px of depth, so a dead straight line through it passes.

Almost every promote and reject decision since late August has used this score (a strict score was added on 20 Oct, but the headline number is still this one). It measures stripe count and rough position. It does not measure camber, draft, entry, exit or where the ends are, which is what the customer pays for.

Evidence that it matters: - 18 Oct: scored against the key, SAM 3 off looked better (draft error 8.3 to 4.5, exit 7.6 to 4.8). Alex's blind vote on the 42 most different photos: SAM 3 on 27, off 8. - 20 Oct: Alex marked 20 of 29 right by eye; the scorer passed 29 and 30. - 12 Aug (your own finding): snapping ends to the sail edge did not improve the score "because we grade against human labels, and the human stripes ALSO stop short of the edge. The grader punishes the model for being right."

If the normal scorer has changed since 13 Oct, say so in your report.

1.2 The training loss is just as loose (verified in the library)

Ultralytics pose docs, dataset page: "kpt_oks_sigmas sets custom per-keypoint OKS sigmas used during training and validation... When omitted, the COCO 17-keypoint sigmas are used for kpt_shape: [17, 3] and a uniform 1/N otherwise."

For our 8 points that default is 0.125. The COCO body sigmas run from 0.025 (eyes) to 0.107 (hips), so our stripe points train with a tolerance looser than a human hip.

The loss code (KeypointLoss.forward): e = d / ((2 * sigmas).pow(2) * (area + 1e-9) * 2), loss = 1 - exp(-e), where d is squared pixel distance and area is the target box width times height. In YOLO26 the same sigmas also drive the pose label assigner, and the same sigmas are used for the validation pose mAP.

What that means in pixels at 960 input (my calculation, typical box sizes):

So with the default, a bottom-stripe point can sit 19 px off (about four tape widths) and still earn 95%, and a 5 px miss on the bottom stripe costs 37 times less than the same miss on the top stripe. The validation mAP saturates at the same place, which is one reason it tells us nothing (1.3).

This fits the oldest complaint in the thread. 16 Jul: the bottom line "rides a few pixels above the real stripe... cuts the corner where the stripe sags hardest". 18 Oct: "in the deepest part of the stripe it sits above where it should, on 82% of bottom stripes". 24 Oct: bottom too flat on 77%.

Hypothesis for why loose means flat: inside the tolerance the loss is a plain squared error, so the model learns the average of where the labels put each point. Labels disagree about where point k sits along the stripe, and the average of positions spread along a curve falls inside the curve, toward the chord. A tight sigma saturates on far-off labels, so the model follows the common position instead of the average. Test T1 settles it.

I found no message in the thread where sigmas were set or discussed.

1.3 Burst 001 already answered two questions that were not read off it

I put the 30 finished runs from burst-001.sailscanapp.pages.dev through a regression (all recipe settings at once, the failed roll 12 at 166 left out).

• Flip is the only setting that clearly mattered. Mean score by flip rate: 0.3 = 230.9 (9 runs), 0.2 = 213.3 (9 runs, 219 without the failed run), 0 = 211.8 (12 runs). Permutation test, 0.3 against 0: p = 0.0007. The top five runs all used 0.3, the highest rate tried. The report's own summary did not pick flip out; it concluded that 640 px did better than 960 px and that no recipe mix stood out.
• Left-over scatter after the settings are accounted for: 9.5 photos (1 SD). That is an upper bound on single-run noise for that model generation. Two single runs then differ by about 13 photos by luck. You later called a couple of photos "run-to-run noise" (22 Oct). The true figure is somewhere between and has never been measured for v39. Test T0 measures it.
• Validation mAP against exam score: correlation +0.02 over 30 runs, -0.40 without the failed run. Flip rate against validation mAP: -0.59. So the setting that helped the real exam made the validation score worse. Ultralytics picks best.pt on its own fitness, which is mAP based (default weights 0.1 on mAP50, 0.9 on mAP50-95, no config option to change it). If we ship best.pt, the checkpoint is being chosen by a number that points the wrong way.

Caveat: Burst 001 was the August medium model on the 3% score. Treat these as strong leads, not as settled for v39.

1.4 The 332 is now a practice set

20 Oct: "I traced each miss on the test photos to the step causing it, tried a fix in the cloud, and only shipped what beat the live app on all 327 photos." Good engineering, but it means the clean-up rules have been fitted to those photos one miss at a time. 317 of 332 no longer predicts what a new customer photo does.

Evidence: on 29 Oct Alex marked both versions wrong on 6 of 30 recent customer sails. RTMO raw beat v39 raw (291 v 276) and RF-DETR raw beat ours raw (272 v 266), and both lost after clean-up, which you described as "all tuned on v39's output" (28 Oct).

1.5 Point order: a possible cause of lines stopping short

What the thread says: - 4 Oct: "The labels are stored left to right." - 20 Oct: some hand-corrected stripes were found to have been drawn leech-first (paraphrase). - 16 Oct: 117 archive training labels and 10 test answers were mirror images. Fixed for archive photos. App labels were not checked. - 26 Aug: the QC page inserted a clicked dot before the last one, scrambling order. Saved labels were not audited. - 26 Oct: the main model trains with "heavy rotations and upside-down flips". RF-DETR trained with left-right flips only; when given the main model's augmentation set it came out much worse.

What the library does (verified in v8_transforms and RandomFlip): - If flip_idx is missing from the dataset file, both flips are switched off with the warning "No 'flip_idx' array defined in data.yaml, disabling 'fliplr' and 'flipud' augmentations." - Both the vertical and the horizontal flip are built with the same flip_idx. - Rotation never reorders keypoints.

Why that matters for points ordered left to right: - A left-right flip reverses the order, so it needs flip_idx: [7,6,5,4,3,2,1,0]. - An up-down flip does not change left-to-right order, so it must not reorder. If the installed version applies that same flip_idx on up-down flips, every upside-down-flipped sample is stored right to left. - A rotation past 90 degrees reverses the order with no reindexing.

One flip_idx cannot be right for both flips under a left-to-right rule.

What reversed labels do under an averaging loss: point k is pulled toward where point 7-k sits. Each 1% of samples with reversed order pulls both ends of every line in by 1% of its length.

The model can partly work out orientation from the picture, so the real effect is smaller than the table. But the symptom matches Alex's most repeated note (29 Oct: "top and bottom stopping short of luff and leech"; 19 Oct: lengthening the top line won 37 to 0). This is unproven. Check C2 settles it in an hour.

1.6 The model already says which points are off the photo, and we ignore it

Verified in RandomPerspective.apply_keypoints: visible[out_mask] = 0 for any keypoint that lands outside the frame after augmentation. The loss then skips those points. So although every point in our labels is marked visible, the YOLO model has been trained on augmented samples where cut stripes had their off-frame points unsupervised and flagged invisible. How many depends on the mosaic, scale and translate settings (C1).

27 Aug, your config check: the per-point visibility output is ignored at inference.

So when a stripe leaves the photo, the model's coordinates for the off-frame points are untrained, its visibility output probably says so, and the pipeline draws them anyway. That is the shape of the bug Alex hand-corrected "over 500 times" (2 Oct), and of the three rule patches since (16 Oct corner-to-corner bottom stripe, 19 to 20 Oct bent tail, 27 Oct fold-back).

There is also a mismatch in convention. A human label for a stripe that leaves the photo is 8 points spread along the visible part, ending at the border. An augmented crop keeps the original 8 positions and hides some. The model sees both.

Check C3 and test T4 cover this.

1.7 What customer photos and customer corrections show (my 11 Nov measurements)

From the 134 saved customer corrections: - Stripe size at 960 input: top stripe about 100 px long, bottom about 648 px. Uploads are 1,500 to 2,000 px on the long side. - draft and draftStable differ by a median of 2.4 points and by 6.3 at the 90th percentile. - On 117 stripes the customer nudged by less than about 2 px, draft moved by a median of 0.7 points and 4.2 at the 90th percentile when read from the curve through the points, against 0.3 and 2.0 from a smooth fit. Camber moved the same either way. So a 2 px nudge can move the draft we report by 4 points. - Where customers did move a stripe, a quarter of the time an end moved by more than 14 to 21% of the chord. Ends are where customers correct us most. - 52 of the 134 carry the camber-over-limit flag.

From v39's finder stage on 25 customer photos sampled in the QC app (judged by eye): - 16 were proper look-up shots and the lines sat on the stripes on all 16. Two blemishes: a fifth line along the foot on one, and a clipped piece of the bottom stripe measured as a whole stripe on another. - 4 of those 16 read camber of 14 to 20% with entry angles of 34 to 42 degrees. - 9 were not measurable (1 far away, 2 edge-on, 3 sideways, 2 small-boat sails where the lines went on battens, 1 sail on the ground). Only the far-away one returned nothing. One sideways photo returned 25% camber and a 150 degree exit.

Also on 11 Nov: the QC Combined Review page was seeding its lines from a cached prediction (recipe finder_v27_flow_v26_refine_1, computed between 1 Oct and 4 Nov), not from the live v39 pipeline. See C11.

Simulation, beyond the table in Section 5: at 250 px with 2 px of point noise, camber scatter is 0.9 and draft scatter 7.6 (curve through points) against 3.5 to 4 (smooth fit). A line that stops 10% short at one end reads camber 0.7 to 1.4 points low and turns the chord 2 to 2.5 degrees.

2. Zero-cost checks (no training, do these first)

C4 needs no labels and gives a repeatability number for every customer photo we hold. It is also the cheapest health check of the clean-up stack: if a 2 degree turn changes the stripe count, a rule is on a knife edge.

One note for C1: Burst 001 varied an "erase" setting and it showed no effect (-2.7 photos, std error 4.8). The Ultralytics erasing setting is described in their docs as applying to classification training. I did not verify this in source. Confirm whether it does anything in a pose run before spending a recipe slot on it again.

3. Fix the ruler before any retrain

R1. Measurement scorecard. Keep the count check. Add, per matched stripe: - distance from the answer line in pixels at stored-upload size and as % of chord (mean and worst point) - luff-end error and leech-end error, along the stripe, as % of chord - camber, draft (stable method), entry, exit, twist error - all of it split by top, middle, bottom; customer or archive; main or jib; portrait or landscape; stripe leaves the frame or not - report median and 90th percentile, never just a pass count

R2. Practice set and sealed set. Take the customer candidates (custset, 180 waiting on Alex's keep or drop) and split by boat and day, so one sail never sits on both sides. Half is the practice set: open every miss, tune every rule on it. Half is sealed: only totals are ever looked at, nobody opens a photo, and no rule is tuned on it. If it is ever opened, it becomes practice and a new sealed half is drawn from newer scans (the keep-unsaved-scans change in 1.0.13 gives a steady supply). The 332 stays as a regression check only.

R3. Gold answers. For the practice and sealed sets, two independent traces per photo, drawn in a zoomed view with no model line showing, bottom edge of the tape, luff to leech (the label guide rule). The reviewer role from 3 Nov can do this. The gap between the two traces is the human noise floor, which was proposed on 20 Aug and never measured. No model can be shown to beat it, so it tells us when to stop.

One labelling rule is still open and must be settled in the label guide before gold tracing starts, because it decides whether the model or the key is wrong on a class of misses: "very short stripes right at the head: always label, or never?" (27 Oct, never answered). Alex decides; the guide and the scorer then follow it.

R4. Repeats and tests. Any retrain is run 3 times with different seeds. Compare to the T0 mean, not to the shipped v39 (v39 was picked from several tuned runs, so its own score is flattered). For photo-level pass or fail use a paired test (McNemar). For measurements use a paired bootstrap. Example of why: the 16 Oct retrain on the mirror-fixed labels "fixed 6, broke 15" and was dropped; exact McNemar on 6 v 15 is p = 0.078.

R5. Win rule, fixed now. A challenger replaces the baseline only if, on the sealed set: 1. median camber error is lower and the 90th percentile is no worse (paired bootstrap 95% interval excludes zero) 2. end error is lower 3. draft error is no worse 4. exact stripe count is no worse (paired test) 5. jitter spread (C4) is no worse 6. Alex's blind vote on 40 photos where the two differ goes its way

R6. Results cannot be lost. Every run writes its config, seed, per-photo results and scorecard to storage before the GPU shuts down. On 24 Oct two results could not be found and had to be treated as unproven.

4. Retrains

One change at a time against T0, three seeds each, scored with R1 on practice then sealed. Burst 001 runs cost about $2 each on the medium model; the large model costs more.

Order: T0 first. T1, T3 and T5 next because they aim straight at flat lines, short ends and the mid-tape offset. T2 once C2b is clean. T12 if T1 helps but does not close the gap.

5. Pipeline work that needs no retraining

| P7 | Twist that does not depend on a bad boom | If C13 confirms the boom is unreliable: show twist only when the boom passes a confidence check, and offer twist relative to the lowest stripe's chord as a fallback that needs no boom. How twist is defined with no boom is Alex's call | | P8 | Use the whole sail to check the top stripe | Camber, draft and twist change smoothly up a sail. The top stripe is the shortest and the noisiest (about 100 px). Fit the trend through the lower stripes and flag a top stripe that sits outside its own error bar from that trend. Flag only, do not overwrite: a top stripe that really is odd is exactly what the sailor wants to see. Needs error bars first (C5 or M2) |

Simulation behind P6 (10% camber, 45% draft, 8 points, 1 px of point noise):

A top stripe is about 100 px at 960. At full phone resolution it is 400 to 800 px.

6. The challenger: a measuring stage that works on a straightened strip

This is the new model to train and sandbox against the current one. The finder stays. What changes is how each line is placed and measured.

The idea has been on our own lists three times and never run: guardrails playbook 20 Aug ("Full-resolution line refinement", Steger sub-pixel ridge snap with a max-move budget; "Bottom-stripe crop pass at high res"), Alex's 22 Oct note (held), and 3 Nov ("next, not started").

Why the earlier "snapping made it worse" results do not rule it out: - 16 Jul: implementation failure ("snapping everything further away"), no numbers. - 14 Aug colour snap and 20 Aug ridge polish: graded against labels that sit mid-tape or nowhere fixed, stop short of the edges, and that training had smoothed (3.05 draft points, guardrails page). A step that moves a line onto the real tape scores worse against an answer line that is off the tape. The 20 Aug loss was on draft alone (4.0 to 4.6), the number you showed the same day to be unstable. - SAM 3, once pointed at the bottom edge of its mask, won Alex's blind vote 19 to 1 (20 Oct). So putting the line on the tape is what the eye wants. SAM 3 does it at about 1,000 px internally with a mask; this does it at full size with sub-pixel output.

Build, in two steps.

Step A (ID M1), no training (a few days): 1. For each found stripe, cut a band from the largest stored image along the finder's line, wide enough to hold the tape plus a margin either side. 2. Straighten it so the finder's line is the horizontal axis. 3. Normalise so the tape is always dark on light. 4. In every column find the bottom edge of the tape to sub-pixel precision (Steger-style ridge and edge response). 5. Link the columns with a dynamic-programming path that cannot jump, with a cap on how far it may move from the finder's line. 6. Fit one smooth section curve to all column positions, weighted by edge strength. Measure camber, draft and angles on that curve, not on a Catmull-Rom curve through 8 points. Use a spline with a smoothness penalty, not a fixed low-order shape: in my simulation a simple polynomial section read a true 35% draft as 39 to 41%. Check the fit for bias on synthetic strips with known shape before trusting it. 7. Columns with no clear tape (glare, logos, numbers) get low weight and are bridged by the fit, not guessed.

Step B (ID M2), a small trained model (if M1 beats the baseline on R1): - Input: the straightened strip. Output per column: tape bottom-edge offset (sub-pixel), visible or not, and an uncertainty. Plus two end outputs: where the tape meets the luff and where it meets the leech. - Training data: (1) stripes a human drew or moved, bottom edge only (the label's seedCorrection.perStripe.dist says which stripes were moved); (2) synthetic tape drawn onto real sailcloth strips with exact known shape, covering white on white, glare, seams crossing, numbers and blur; (3) SAM 3 bottom-edge masks as low-weight labels; (4) unlabelled customer strips with a consistency loss between two jittered views. - Per-column uncertainty gives an error bar on every number we show.

Why a second stage at all: the finder places all 8 points of a stripe from a single grid cell. On a 650 px bottom stripe the end points are hundreds of pixels from that cell, so fine placement at the ends is the hardest thing for that design. A stage that looks only at the strip, at full size, does not have that limit.

Fair test. Baseline is today's pipeline. Score both on: - the sealed set with gold answers (R3) and the R5 win rule - the synthetic strips, where truth is exact - the C4 jitter spread - repeat pairs: two photos of the same sail taken moments apart should give the same numbers. Mine these from customers who scanned the same sail twice in one session; no labels needed - any sail we hold both a SailScan photo and a reading from another sail-vision system for. A cross-check only, since the other system is not truth either. I found no published accuracy figure for any competing system.

7. Shelved ideas whose verdicts do not hold

Each was dropped on a gap inside single-run noise, on the 3% score, with no repeat. Do not re-run them all. Re-judge any that come up again with R1 and R4.

The clean-up rule changes are different: they are deterministic, so their gains on the 332 are real on the 332. The open question for them is whether they hold on the sealed set.

8. Suggested before and never run: now numbered tasks

These were each proposed at least once in the thread and never run. They are tasks, not reminders. Report on each by ID.

S1. Lane-detection model as a challenger finder

Proposed 5 Sep ("lane detection is the closest field, train one as a challenger"), 17 Sep (test matrix row 3.2) and 2 Nov. Never trained.

• Why: a lane model predicts each line as one whole curve (its position at fixed steps across the image, refined along the line, trained with a line-overlap loss) instead of 8 separately placed points. That removes the "which point is which" problem (1.5) and the sliding of points along the stripe (1.2) by design. It is the "stripes as whole curves instead of loose dots" idea you called a gem on 31 Aug.
• Model: CLRerNet (LaneIoU), the one named on 5 Sep and 17 Sep, or a newer model of the same row-anchor family if you find one that is clearly better and cleanly licensed. My search turned up GFSR (arXiv 2605.23327, 2026) and DiffusionLane as newer candidates; I have not evaluated either.
• Data: the same labels and the same train and practice split as T0. Turn each photo so stripes run the way lanes do. Convert every labelled stripe to positions at fixed steps along that axis. Stripe ends map to the lane start and length outputs.
• Licence: flagged on 5 Sep. Confirm the code licence. Do not start from weights trained on CULane (research-only data). Start from a backbone whose weights are cleared for commercial use.
• Runs: 3 seeds.
• Judge: raw against v39, RF-DETR and RTMO raw, then each with clean-up tuned for it (P5), on the R1 scorecard and the sealed set.
• Where it sits: after R1 and T0. On the 3% score a lane model cannot show its advantage, which is line placement. Our own tests say finding stripes is not the weakest link (RTMO and RF-DETR already beat v39 raw; 5 Sep: "the model itself never missed a stripe: all 24 missing-stripe cases came from our own filtering layers"). So expect its gain, if any, in placement and ends. Run T1, T3 and T5 first because they are cheaper tests of the same weakness.

S2 to S10

9. What to send back

First, a status line for every ID in Section 10: done, running, waiting on what, or not doing and why. An item nobody reports on is assumed dropped.

Then one page, in this order: 1. C1 to C8 answers, as a table, with the actual values found (the sigma in use, the flip settings, the share of reversed labels), not only that the check was run. 2. T0 noise: mean and spread over 3 seeds for every R1 metric. 3. One row per test: what changed, seeds, practice and sealed results, paired p-value, pass or fail on R5. 4. Anything in Section 1 that your code shows is wrong. I could not see the training config, so corrections are expected.

Two things already waiting on Alex gate R2 and R3: the custset keep or drop pass, and approval of the label guide. Nothing new is asked of him here.

10. Checklist: every item in this file

Holding retrains until the checks and T0 are in is right. It is not a reason to hold the rows marked "Now" or "Build now": C4 to C11, C13, R1, R6, P1, M1 and S6 need no retraining and no answer from the first three checks.

11. Recorded so it is not lost, not asked of you

• User taps on the luff and leech, and one-tap "add a missed stripe". Ruled out by Alex on 12 Nov. The product is automatic.
• Burst of several frames per scan with a median. Ruled out by Alex on 12 Nov.
• Several photos of one sail from different spots, rebuilt in 3D (Depth Anything 3 class). The only route found to true 3D twist. It changes how a scan is captured, so it waits for Alex to ask for it.
• Physical truth panel. A rigid curved panel with tape at a known camber and draft, photographed from many positions, is the only way to get exact truth for the whole chain including viewing angle. It needs someone to build and shoot it, so it is not a task for you. If Alex has one made, score it with R1.
• Two boom cams as a stereo pair for truth on a real sail. Same status as the panel.
• Tape width as a depth cue (the tape looks thinner where the cloth turns away from the camera). An idea only; no test designed.
• No public accuracy figure was found for any competing sail-vision system. Whatever R1 produces on the sealed set will be the first number of its kind.

Appendix A. Sources for the library facts

• Ultralytics pose dataset docs (default sigmas, kpt_oks_sigmas, flip_idx): https://docs.ultralytics.com/datasets/pose
• Ultralytics loss reference (KeypointLoss, v8PoseLoss, PoseLoss26, RLE): https://docs.ultralytics.com/ja/reference/utils/loss/
• Ultralytics augmentation reference (apply_keypoints, RandomFlip, v8_transforms): https://docs.ultralytics.com/reference/data/augment
• Ultralytics YOLO26 docs (RLE in the pose head): https://docs.ultralytics.com/models/yolo26/
• Ultralytics community thread on fitness weights and best.pt: https://community.ultralytics.com/t/looking-for-support-regarding-the-custom-fitness-metric-without-editing-the-source-code/1198
• MapTR (matching an open polyline in either direction): https://arxiv.org/abs/2208.14437
• CLRerNet (lane detection with LaneIoU), paper and code: https://arxiv.org/abs/2305.08366 and https://github.com/hirotomusiker/clrernet
• Depth Anything 3: https://arxiv.org/abs/2511.10647
• Our own pages: burst-001.sailscanapp.pages.dev, accuracy.sailscan-scanner.pages.dev, guardrails.sailscan-scanner.pages.dev

Not verified by me: whether the installed Ultralytics version reorders keypoints on up-down flips (the current source passes flip_idx to both flips; older versions applied it to the left-right flip only), and whether a P2 config exists for YOLO26 pose. C1 and C2b answer the first.

Appendix B. Test ledger, 3 Apr to 12 Nov 2026

Every accuracy test found in the Boats thread, in date order. Dates are 2026, month-day. Lines marked MY READ or starting with => are Claude's comments, not things said in the thread. Quoted text is from the thread. Version numbers are as posted at the time ("v27" was used for two different models, in August and in September).

Chunk 01: 3 Apr to 16 Apr 2026

• 04-03 Group starts. Nate joins after a call on 2 Apr. PRD exists (Alex wrote).
• 04-04 FIRST PROTOTYPE: Claude API vision. "Detects the right stripes but line drawing it has trouble with." Reading vs drawing are two different engines; measurements did NOT use the drawn lines. Decision: hide lines. Portrait photos handled badly.
• 04-04 Alex: had hit the same wall himself before (stripe overlay picked up anything even with colour hints).
• 04-10 Capture rule stated by Alex: photo taken directly under the sail looking up; 0.5x zoom makes it tricky. Same photo gave different readouts on repeat runs (non-deterministic LLM). Rotating so sail foot is at bottom helped.
• 04-11 Alex tested vs human-scanned images: draft spot on at times, camber within 10%, exit angle furthest out. Tack often wrong.
• 04-11 Nate builds rating page (ngrok /rating, later sailscan-rating.pages.dev): Alex rates outputs + comments, "recursive training" of prompts.
• 04-12 Alex: "the hold up is the accuracy on the detection".
• 04-13 A sail designer sends ~70 scanned + unscanned pictures (same sail with and without human scan) into Drive.
• 04-14 GROUND TRUTH PROBLEM: Alex can't run the Windows sail-scanning programs on Mac, so can't verify numbers; worried about feeding bad truth. Nate: "may require a custom trained model". Nate: manual tool probably has 1 to 5% margin of error; idea to reject bad photos.
• 04-15 Idea: photo of instrument console mapped to the sail scan. Claude says 0.5x lens can be handled with maths.
• 04-16 The sail designer's comments on ranges are in the code. Nate "rearchitecting the rating".

Chunk 02: 17 Apr to 31 May 2026

• 04-17 Alex finishes rating sheet but only on tack + verbal assessment (no way to human-scan).
• 04-27 Camera integrated into app flow with guides. 04-29 build rejects non-sail objects.
• 04-29 Alex proposes hiring a CV expert. First question from an outside CV contact: "small YOLO-like model vs a VLM".
• 04-29 STACK v1 (written brief): Sharp preprocess (EXIF rotate, resize, normalise to landscape) -> OpenCV.js classical CV for stripe Y-position hints -> Claude Sonnet vision returns curve points JSON -> Catmull-Rom spline fitting -> metrics from tangent vectors -> validation against physical ranges, curve rescaling. HUMAN RATINGS on ~54 images: camber 73% good, exit 28%, draft 12%, entry 4%. "Curves roughly right, x,y placement off, especially at the endpoints (luff and leech)." Options listed: VLM + classical refine (active contours), segmentation model (U-Net) + curve fit, fine-tune small model, other. NOTE: Catmull-Rom + tangent-vector metrics dates from here. Endpoints named as the main problem on day one.
• 04-30 Gemini switch (admin toggle). Rating page moved to Railway.
• 05-04 Outside contact: Nano Banana Pro produces a segmentation mask of the stripes ("promising"). Alex also tried it.
• 05-19 Claude removed from the app entirely; 05-20 all Gemini.
• 05-20 TEST: 54 human-scanned photos with the data stripped, run through "Gemini 2.5 Pro + segmentation + recompute-from-geometry". Nate: "definitely better, not perfect". Concern recorded: earlier flow may have leaked the measurement to the model.
• 05-21 TEST: AI-only (Gemini) vs CV-only side by side on rating page: "AI only whoops CV's butt... maybe our mistake was trying to use computer vision at all... go pure nano banana". No numbers in thread (screenshots).
• 05-22 Outside advice: train on lots of images, possibly frames from video.

Chunk 03: 1 Jun to 7 Jul 2026

• 06-01 DECISION: stop prompting vision LLMs, train our own model. Plan: draw dots on ~500 photos, train a pose model. label.sailscanapp.com built for dot labelling.
• 06-02 Training design written down: YOLOv8-pose, "given this photo, where do the stripe dots go", evaluated against one golden benchmark image (IMG_5305). Loop: model pre-draws, human fixes, fixes become labels, retrain.
• 06-08 to 07-03 Alex hand-labels: 70 (30 Jun), 200 (1 Jul), 250, 300, 340 (2 Jul). Labeller had UI bugs (dots can't sit close, dot size, 4+ stripes).
• 06-30 Ideas raised: AI-generated sails for training (Nate), flipping images to double the set (J-Ro), Fiverr labelling.
• 07-01 "Calculated" page: Nate using "an AI guess" for the metric formula; Alex: camber and twist look very close to SailPak on the same image.
• 07-02 About 2,900 third-party archive images added, which came WITH lines and point maths already attached (not labelled by us, not to our rule). Nate 07-06: "that's how it got so good". Nate 07-07: no need for Alex to scan those. => From here most training labels AND most of the later test set are another system's lines, produced under an unknown labelling rule.
• 07-04 v4 "round 1": trained on half our photos + all archive. First human vs model overlay looks close.
• 07-05 v6 trained with flipped images (v5 had a bug). Alex: "accuracy is a huge leap". Alex feedback 07-05: model's dots are equidistant; entry and exit need more dots because the curve is sharper there. (No follow-up seen yet.)
• 07-06 Training moved to cloud, ~2.5 h per version. Nate testing: low / high / highest resolution and "capacity" (model size). Mentions "curved line versus points" maths as another strategy, not run. V8 goes live. Nate: "not much more we can do without more photos".
• 07-06 Boom labelling queue (Boom Blitz) started, for twist reference. Alex: hard to get boom and sail in one shot.
• 07-07 /review page for bulk upload + correct.

Chunk 04: 8 Jul to 17 Jul 2026

• 07-08 Alex rates the archive scans good/bad by eye ("stripe identification good = good scan"): 1,000 by 8 Jul, all ~3,000 by 9 Jul. Bad ones drop out of training and go to a Fix External queue for re-trace. Alex 07-08: "accuracy of stripe is good but the ends are a little off".
• 07-08 Auto-order stripes by vertical position (fix for add-stripe ordering). 07-09 +107 archive photos, 07-15 +60.
• 07-09 500 boom labels lost (had to redo). All ~3,000 boom shots done 07-14.
• 07-10 v11 to prod ("nice increase on camber and draft"; numbers only in a screenshot).
• 07-11/12 Two variations trained on Codex's recommendations: both worse ("our first model that got worse"). Model family named: Ultralytics YOLOv8m-pose.
• 07-13 HOW ACCURACY IS MEASURED: 50 to 100 human-rated photos held out, same ones every time.
• 07-15 v14 and v15 lost to v13. Bigger exclusion of bad photos introduced at the same time. "Physics filters that reject impossible curves" added to the live path (S-curves capped ~20 deg). Known gap: 4th stripe on black mains. v16 failed to start: one of the "300 frozen holdout images" missing (so a 300 holdout was defined, but see 07-17: evals still on 50).
• 07-15 Alex's asks: bridge a stripe across glare/logo gaps; rotated (90 deg) sails; discard button. Chat: rotation augmentation + gap traces go into the next run.
• 07-16 DIAGNOSIS (Chat, on Alex's photo, prod v13): bottom line rides a few pixels ABOVE the real stripe from luff through the middle; "cuts the corner where the stripe sags hardest, so camber reads low"; cause given: only 8 dots per stripe, sparse segments chord across the curve.
• 07-16 Alex idea: snap dots to the stripe with classical CV (HSV colour band centreline), or SAM 2 with the dots as prompts then spline through the mask centreline; flag stripes where model and snapped line disagree.
• 07-16 TEST: snapping pass. Result: "made it way worse... snapping everything further away". Nate wants to retry. No numbers. (Looks like an implementation failure, not a fair test of the idea.)
• 07-16/17 v17 (12 dots per stripe, extra at luff/leech) -> reported as v19 "denser-keypoint luff fix"; v18 = v13 recipe + new data. BOTH LOST to v13 on the 50-photo holdout, "margins within noise".
• 07-17 10-SEED TEST: v13's exact recipe retrained 10 times with different seeds: 0 of 10 beat v13, most clearly worse. Team conclusion: "v13 isn't a fluke, it's the best this recipe + data can do; path forward is data". MY READ: this is the opposite. If 10 reruns of the SAME recipe all score below v13, v13's score is an outlier for its own recipe: either a lucky draw on a 50-photo holdout that was also used to pick it (winner's curse), or v13 saw holdout photos in training. Every "lost to v13" result from 07-11 on is suspect until that is ruled out.
• 07-17 Nate: holdout is 50; should be 200 to 300 but that removes too many training photos. Model input resolution: 960. App live in App Store (1.0.1). Alex idea: blur check (variance of Laplacian) + "no stripes found, retake" gate.
• 07-17 Nate: "middle ones working well", edges and boom still to fix.

Chunk 05: 18 Jul to 27 Jul 2026

• 07-18 App 1.0.2 live. 07-19 closed beta opens (about 37 invited testers). Every finalised app scan uploads to the labeller queue.
• 07-18 Alex rule: the line should run "all the way to the luff tape (the very end)".
• 07-19 TEST (Alex, same sail): 0.5x lens vs 1x. 0.5x "not accurate". DECISION: 0.5x removed "until we successfully field test". (Lens distortion never corrected; just removed.)
• 07-20 New guards: reject sudden mid-stripe kinks over 30 deg; drop whole-line rigging outliers only when 3 real stripes agree.
• 07-21 Alex rule for which end is the luff: entry angle is almost always steeper than exit.
• 07-26 RETRAIN on 1,104 labels (was 707 at v13-era run, +397; about 238 of the new ones are corrected "bad" archive scans): "still worse" than v13.
• 07-27 FINDING: all archive scans had been trained at 800x600 px. 1600 px versions exist. Reruns at higher res: "no 100% clear winner, some improvement in a few areas". v24 is the candidate. => v13 was trained (and its holdout scored) on 800x600 copies. Label precision on those is coarse (1 px there is about 5 px on a phone original), and the holdout shares that low-res look while customer photos do not.
• 07-27 Model Preview compare page (v13 vs v24). Alex by eye: both miss different sections; v13 marginally better; v13 bad at the LUFF of the bottom stripe, v24 bad at the LEECH. Nate: "some of the models need human eyes testing vs just holdout".
• 07-27 Neither model handles a portrait photo rotated 90 deg.
• 07-27 Labeller API opened; AI corrections submitted with source "ai-correction" (model's predicted lines accepted after a visual check). First known case of model output going back in as a label without a human redraw.

Chunk 06: 28 Jul to 7 Aug 2026

• 07-28 Sideways photos auto-rotate (rotation retry, points mapped back). Fable-assisted corrections go to a separate queue; cannot enter training until Alex marks Good (07-29).
• 07-31 Luff/leech reorientation deployed (entry/exit/draft flipped when the sail is read backwards). Black-main 4th stripe and boom misses logged as a named failure slice.
• 08-01 Evaluator fixed: missing/extra stripes now matched one-to-one (earlier scores used a different matcher, so pre and post 1 Aug numbers are not comparable).
• 08-01 CURVE LAB TEST: 7 ways of representing a stripe, each fitted to frozen human labels. Winner "Arc 32" (32 evenly spaced points). Production 8-point and the Bezier/spline options "had larger draft and endpoint-angle errors". MY READ: this measured how closely each format reproduces the HUMAN POLYLINE, with the human dots (joined by the existing spline) as truth. A smooth curve is penalised for not reproducing human click noise. It says nothing about which format a MODEL predicts best, or which is closer to the real stripe. Bezier/parametric output was never trained or tested as a model head.
• 08-01 Labeller: unlimited human dots, saved raw, resampled to 32 for training. Plan stated: train an arc-32 model on the same data vs v13. (Check later whether it ran.)
• 08-01 COMBINER TEST: "use the new model to decide which stripes exist, use v13 to trace the curve" won.
• 08-03 Duplicate-line validator blocked saves on real close stripes (2.1%, 2.6% apart); changed to warn. OPEN QUESTION posed by Chat and never answered: "choose thick-stripe rule: bottom edge or centerline".
• 08-04 Prod path confirmed: /api/analyze, sailscan-v13, coordinates at 1024x768, a "pixel-snap stage" that moved points 0 to 9 px, rotation retry for sideways photos.
• 08-04 New guard: a line bending the opposite way to 3 agreeing stripes is REDRAWN: endpoints kept, "interior inherits the median shape of the three agreeing real stripes". (This invents a curve. Numbers from it are not a measurement.)
• 08-04 Nate: no other ideas beyond more data; "refining models don't seem good enough without additional data".
• 08-05 Roboflow looked at: annotates boxes/polygons, not curves. boom-v2 live. Alex's 13 "ground truth" rules listed; 4 actioned, 9 open (direction by entry>exit, top stripe shortest, no duplicates on one stripe, no crossing, no opposite direction, no mixed orientation, S-curve cap, rotated sails, no twist on jibs).
• 08-06 v26: does NOT beat v13 on draft/twist/curve; DOES find stripes better (fewer phantoms, right count more often). 91 held-out photos page: v13 / v26 / "mean" (v13+v26 points averaged). Alex by eye: "Mean 960 seems the most accurate". Nate: "feeling like a lucky roll of the dice in Vegas".
• 08-06 Alex idea: a VLM (Qwen) to say which stripes exist, then snap/trace inside. Nate: detection is the real gap.
• 08-07 Crossing stripes dropped. Tightened 4th-stripe detection created phantom lines 3.4% from real ones; reverted at source.
• 08-07 Alex noticed the archive's own lines look "more accurate than" our model. Chat: those are the archive's lines, "which is why they look tighter"; the ones marked Good become training data.

Chunk 07: 8 Aug to 15 Aug 2026

• 08-08 Chat calls steep look-up-from-deck shots the model's "weakest view". Alex: that IS the only correct shot. (So the training mix under-represents the real capture geometry, or its labels there are poor.)
• 08-09 Faint-stripe backfill: junk on 7 of 9 photos (boom, patch seam, sail edge). Dialled back.
• 08-11 SAIL EDGE MODEL: outlines auto-traced with SAM using labelled stripes as hints (19 of 20 usable), then an edge model trained: finds the sail on 99 of 100 unseen photos; on 96 the edge is within 1% of photo width of the labelled stripe ends. (08-15: "100% of test photos, edges within 1%".)
• 08-12 TEST (Alex by eye, 100 photos, v13 vs v13 + ends snapped to the edge): 42 better / 34 same / 24 worse. "Hit and miss, struggled on some white sails."
• 08-12 KEY FINDING (Chat): snapping ends to the sail edge does NOT improve the score "because we grade against human labels, and the human stripes ALSO stop short of the edge. The grader punishes the model for being right." Fix chosen: green sail-outline guide in the labeller so NEW labels reach luff/leech. Old labels and the test answers were not corrected.
• 08-14 Colour-snap layer (pull stray dots back to stripe colour; Alex's idea): on 209 photos accuracy got worse overall (white/grey stripes on white cloth can't be told apart); works on strong-colour stripes. Kept only "drop dots that land on sky".
• 08-14 FOUND: a pixel-nudging step had been live since 29 Jul, never scored. Removed: camber error down 13%, exit angle error down 19%. Skippy's lesson: "nudging dots after the model draws them makes things worse, every time". MY READ: every one of these "refinement made it worse" results was graded against labels now known to (a) stop short of the edges, (b) sit at no fixed place on the tape, (c) be mostly another system's lines at 800 px. A step that moves a line ONTO the real tape will score worse against an answer key that is off the tape. These are not fair tests of refinement.
• 08-14 v27 run: 4,677 labels (1,350 app edits), 592 rated-bad archive lines excluded; 3,327 usable archive scans.
• 08-15 v27 vs v13 (frozen set): exact stripe count 77% vs 64%, fewer phantoms; but draft error 11.3 vs 4.6 and camber 4.1 vs 1.3 (units as posted). "Sharper eyes, worse ruler."
• 08-15 Scorer bug found: sometimes compared a model line to the wrong stripe. Fixed; numbers recomputed.
• 08-15 "BOUNCER" SHIPPED: v13 draws, v27 vetoes fake lines. 209 photos, production code: exact count 64% -> 71%, precision 0.83 -> 0.91, recall up, shape numbers flat. Caught phantoms on 34 photos the old filter passed.
• 08-15 Skippy on the 25 trained models: they disagree usefully on counts/placement/twist, but "they all bend curves the same wrong way, so stacking them doesn't help there". MY READ: a shared, same-direction curve error across 25 models is a systematic bias (labels or the 8-point format), not noise. It is the most fixable kind of error and it has not been chased.
• 08-15 "Same pattern as every rerun": retrains improve counting but make curve shape worse vs v13. Skippy: need ~2,000 fresh scans. MY READ: each rerun adds more human app edits (different convention, ends reaching further) and is then graded on an archive-style holdout. Looking worse there is what you'd expect even if the lines are closer to Alex's rule.
• 08-15 Outline as BACKUP for the boom finder: boom found 95.5% -> 100%. 85% of fake lines sit ON the sail (duplicates), so an outline check can't catch them.
• 08-15 Auto-rotate fixed: it had been choosing the rotation with the most stripes, doing shape maths in the sideways frame, and caching bad results forever. Right orientation now 48 of 53 worst vertical photos (was 36).
• 08-15 Archive provenance (Skippy): lines are another system's auto-detector output that its users may or may not have adjusted; no per-scan flag.
• 08-15 Alex posts a Fable doc; Skippy adopts: slice-tagged holdout, targeted collection over volume, model disagreement to rank the QC queue, "same sail never straddles train/test", video frames as a data source.

Chunk 08/09: 16 Aug to 19 Aug 2026

• 08-16 SLICE TAGS on the test set (geometric estimate, not human-checked): deck-angle shots are the weak slice: exact count 41% vs 66% overall for v13; bouncer 72%.
• 08-16 LEAK FOUND: 71 sails were in BOTH training and test. "Quietly flattering every score we've ever reported." Leak guard now hard-stops a run. Skippy claims comparisons still valid "because every model enjoyed the same leak" (not shown: models were trained on different splits at different times).
• 08-16 LIVE HOLDOUT v1: 32 recent user photos traced after the last training run, no overlap (over half the candidates overlapped training and were dropped). v13 alone: exact stripe count 16%. Bouncer combo: 50%. Contact sheet: live-holdout-v1.sailscanapp.pages.dev
• 08-16 Boom finder's typical error: about 9 degrees (first time shown). Twist is measured from it.
• 08-16 Alex: the top stripe is placed more accurately than the bottom stripe "90% of the time". Skippy's reasons: bottom stripe is long, crosses numbers/logos/seams/boom shadow, its ends sit at luff and leech where "even human tracers disagree", most perspective warp, noisiest labels.
• 08-16 Training augmentation already includes flip, stretch, brightness, patch blanking.
• 08-17 BLIND VOTE (Alex, 40 fixed photos, models hidden): v26 20, v13 7, v25 5, v24 1, none right 7. Skippy: "v13 still edges it on the raw number checks, but your eyes say v26". v26 PROMOTED to production 17 Aug. => The numeric scorer and Alex's eyes disagreed 20 to 7, and the eyes were right enough to ship. The scorer had kept v13 on top for five weeks.
• 08-18 On the 32 live-holdout photos: v26 finds 82% of stripes vs 78% (v13); its lines are right 90% vs 67%.
• 08-18 SAM TESTS on 20 photos / 66 stripes: SAM 2.1 with hints 29 clean; SAM 3 with hints 41; SAM 3 text-only ("find the draft stripes") 25.
• 08-18 SAM 3 AS CHECKER (32 photos): every line both v26 and SAM 3 agree on was correct; of v26's 10 wrong lines SAM 3 refused to confirm 8; SAM 3 found nothing v26 missed. Colour prompts ("white line" etc.): 2,258 candidates, only 2 real missed stripes.
• 08-18 Alex's plan: SAM 3 async only; verified badge; disagreements to review queue; SAM 3 pre-trace on user-flagged missed-4th-stripe photos with human confirm; bottom-stripe study "SAM 3 mask centreline vs human truth" (result not seen by 19 Aug). Nate: draw SAM 3 candidates as a layer even if too many.
• 08-19 SAM 3 "second look" live on RunPod (about $10/day), in the web scanner only.

From project docs (second-hand), 16 Sep to 27 Oct 2026

• Test set from Sep: frozen 332 photos (archive-heavy). Score = photos with every stripe right.
• 28-29 Sep: v29 (YOLO26 warm-started) shipped; sail rules had been SKIPPED on the customer path and were put back: 147 -> 199. SAM 3 clean-up with a guard (keep redraw only if it moves < 3%): 279. Raw SAM 3 made 200 photos worse, 59 better (on the old key).
• 28 Sep bake-off: six retrains lost; five add-on layers (verifier, dedup, refiner, conditioned, family) and test-time augmentation all lost to v29 alone; found "human" labels that were machine lines saved back by a labeller bug. Edge extension switched off because it lowered the score; the run-off bug hand-corrected by Alex 500+ times.
• Masks/polygons tried twice (SAM 3 fine-tune "made 77% of lines worse", RF-DETR segmentation): dropped. Later (Oct) SAM 3 retrained to the bottom edge won Alex's blind vote 19 to 1 and lifted strict score 122 -> 181.
• 5 Oct mast finder 309/311; 43% of customer photos are portrait where it does not run. 7 Oct vertical rotation fix.
• 16 Oct: RF-DETR raw beat ours raw (272 vs 266) but lost after OUR clean-up ran (287 vs 294). 27 Oct: RTMO raw beat v39 raw (291 vs 276) but lost with clean-up (289 vs 313). => clean-up stack is fitted to the incumbent.
• 18 Oct: 1280 px training lost (293 vs 295). 19 Oct: 10 answer-key lines were mirror-image; 7 missing top stripes added to the key (score +7 with no scanner change).
• 19 Oct (Nate): v39 = YOLO26 LARGE pose, 8 points, 960 px; RF-DETR at 888 px, added only if conf >= 0.66 and not on/crossing a v39 line; edge-v2 trims ends before SAM 3 (trim after is off); SAM 3 line kept only if no point moves > 3% of the photo DIAGONAL and it is >= 95% as long; shape engine = Catmull-Rom through the model points, depth/draft/angles measured on that curve; bottom stripe "rides a bit flat where SAM 3 is rejected (134 of 320 bottoms)".
• Blind votes (Alex, one rater): two finders vs one 26-2; v39 vs previous 23-15; longer top lines to the leech 37-0; keep SAM 3 27-8.
• 20 Oct: Alex's eye on 30 test photos: 16 then 20 of 29 right, while the scorer passed 29 to 30. Strict scorer (bottom edge, luff to leech) about 65%.
• 21 Oct label audit: 5,459 training labels = 1,002 drawn/approved by Alex; 2,780 archive with approval unprovable; 443 app labels unapproved; 585 unknown author; 649 rated bad (excluded). 421 duplicate photos still in training.
• 24 Oct: "bottom stripe too flat" on 77% of bottom stripes.

FULL THREAD, first-hand. bb-01: 19 Aug to 21 Aug 2026

• 08-19 4TH-STRIPE ROOT CAUSE. 628 labels have 4+ stripes (474 Alex + 154 fixed externals). Model finds 4+ on 96% of them. SERVING RULES deleted the 4th on 66%: a silent "duplicate" rule (two stripes closer than 2.5% of image height = same stripe, lower-confidence one deleted, no log). Fix (spacing relative to that sail's own stripe spacing): deleted-4th 66% -> 15% (309 photos fixed); frozen-set stripe recall 82% -> 93%, precision flat; cost: extra stripe on 3-stripe photos 7% -> 10% (38 photos, some probably real). SHIPPED 19 Aug. => For six weeks the team believed "the model can't see the 4th stripe, we need 2,000 more photos". It was our own clean-up. Same shape as later finds (rules skipped on customer path, draft luff-side bug, QC seed cache).
• 08-19/20 SAM 3 vs V26 ON THE BOTTOM STRIPE, six rounds: 1. SAM 3 re-tracing V26's dots: V26 closer 17 of 21. 2. SAM 3 alone, auto colour prompt: closer 11 of 14 but found only 14 of 31. 100-sail rerun: found 37 of 71; camber worse; draft better 24 of 37. Colour picker gave wrong prompts. 3. Any-colour prompts + our rulebook: found 90%; closer 47 of 64; camber tie; hard-clipping masks at the sail edge "chopped the ends off" and was removed. 4. Blind vote: V26 62, SAM 3 24. 5. Exemplar prompts + crops (SAM 3 internally shrinks images to ~1000 px): closer 45 of 64, camber still 1.34 vs 0.70. 6. FOUND: Skippy's own mask-to-line tracer flattened curves: "shave just 10% off the stripe's ends and camber collapses by almost 4 points". After fixing: closer 46 of 61, camber better 37 of 61. 7. SAM 3 fine-tuned on 941 traced sails ($3): blind vote SAM 3 71, V26 19, BUT 83 of the 100 photos were in its training data. Clean rerun on frozen 300: finds 92% (same as V26), dots slightly closer on average, but camber error 1.7 vs 0.9, draft 4.9 vs 4.0, angles about double. Verdict: V26 stays the measuring stick. => "Line closer on average, numbers worse" = the conversion from mask to line (ends, smoothing), not where the tape is.
• 08-20 SKIPPY'S AUDIT (on real data): a. Training smoothed Alex's traces before teaching V26; "that smoothing alone shifts draft by ~3 points". b. Draft number unstable: deepest zone is a flat plateau; "a steadier way of reading the same line is 3.6x more consistent". Tested: V26 draft error 4.15 -> 3.09. "Already running silently next to the old one" (= draftStable). NOT seen switched on. c. More output dots per stripe "changes nothing". d. Proposed: human noise floor study (Alex re-traces 30 sails blind). Alex declined.
• 08-20 V27 (raw, unsmoothed traces, otherwise V26's recipe; 3,686 traces; H100): finds more stripes (94 vs 92%), camber/twist equal, draft WORSE (5.2 vs 4.0). Not promoted.
• 08-20 COMBINE (V26 draws, V27 proposes missed stripes, SAM 3 checks): found 93.2 vs 91.9, precision 88.5 vs 89.3, numbers unchanged. Alex's blind look at the 23 changed photos: "terrible batch". Not shipped.
• 08-20 LINE POLISH (nudge each line onto the brightest paint ridge, full-res, capped move): camber unchanged, draft 4.0 -> 4.6. "Third time an automatic after-the-fact line adjustment lost... at the resolution our photos actually have."
• 08-19 Skippy proposed a high-res pass on JUST the bottom-stripe strip ("more pixels on the stripe means the luff drop and belly depth resolve instead of getting chorded"). Track whether it was built.
• 08-20 Frozen test set = 300 photos; V26 finds 92% of stripes at 89% precision. Contamination caught twice in one day (83 of 100; 29 of 32 live photos in V27's training pile). Overlap guard added.
• 08-20 08-21 Work moves to JIRA tickets; 96 filed.

bb-02: 22 to 31 Aug 2026 (first-hand)

• 08-24 Alex asks to resume bottom-stripe work; SAIL-114 plan (re-baseline on frozen 300, 50 worst bottoms to review, retrain). Nate: "not confident the plan makes sense".
• 08-26 151 corrected scans banked since the 15 Aug training run. Archive arrived with jib/main tags swapped (jib filter never fired); repaired 29 Aug.
• 08-26 LABEL TOOL BUG (production QC page): clicking past the end of a line inserted the dot BEFORE the last one, scrambling point order (zigzag, "231%"). Fixed 26 Aug. => an unknown number of saved labels may have out-of-order points. Never audited that I can see.
• 08-27 v26 FACTS (Skippy checked the live config): MEDIUM model, trained and served at 960. ONNX output 1x29x18900 = grids 120^2+60^2+30^2 (strides 8/16/32, no finer head). conf 0.1, NMS IoU 0.7, keypoint visibility ignored. Letterbox grey 114, cv2 INTER_LINEAR. "It's 8 points per stripe, not 32" => the Arc-32 format from Curve Lab never became a model output.
• 08-28 Skippy: measurement is pure geometry (tangent vs chord), no clamping. "There IS one post-process that moves points (nudges them toward the stripe's colour in the pixels)... on our suspect list". (Contradicts the 14 Aug note that the nudge was removed. Check.)
• 08-29 upside-down stripe order auto-correct (only on full inversion). Per-stripe twist shipped.
• 08-29/30 customer corrections: only 2 real customer saves at that date. Approval gate before training added 30 Aug.
• 08-31 BURST 001: 30 models (50-roll recipe matrix) from scratch, $2 / 3 h each, scored on frozen 332 (dated 25 Aug). Final (1 Sep): roll 20 = 245/332, rolls 11 and 15 = 242, v26 = 240. "A 5-photo edge is inside the noise." KEY FINDING: "the practice-exam score predicts nothing (0.05)" = the training-time validation score has ~zero correlation with the frozen-test score. Report: burst-001.sailscanapp.pages.dev
• 08-31 Alex's "Accuracy Avenues Review" (his Claude): three ideas Skippy called gems: (1) stripes as whole curves instead of loose dots, (2) teach SAM 3 what a draft stripe is from our scans, (3) synthetic 3D sails for hard cases. Only (2) was started. Plan: sam3-plan.sailscanapp.pages.dev

bb-03: 1 to 6 Sep 2026 (first-hand)

• 09-02 DEEP20 = burst winner recipe at 960 trained longer: 248/332 vs v26 240. But same day on line metrics "V26 beat Deep20 on every metric".
• 09-02 SAM 3 fine-tune gallery (40 holdout): fires on about a third of photos alone.
• 09-02 HYBRID (Deep20 finds, SAM 3 refines) on 332: SAM 3 line closer to human on 16% of stripes, farther on 77%. Boom fence (clip at SAM 3 sail outline): accuracy down at every setting. SAM 3 disagreement = photo about 2x as likely to be wrong (auto-flag; lined up with human-found errors on 122 photos).
• 09-04 box-ROI SAM 3 (tight crop per stripe): 2 min/photo and zero stripes found = plumbing failure, not a real test. Skippy later: "on 43% of photos SAM 3's lines are as good or better than V26, but on the easy stripes where V26 is already good".
• 09-04 Nate's own session built sam3-live.pages.dev: fine-tuned SAM 3, conf 0.25, res 1008, text "sail draft stripe", plus "Perfect stripes" (re-run on a zoomed crop of each stripe, sanity gate keeps pass 1 if it wanders). Alex: "very good accuracy". Put on QC pages as "SAM3 (Deep20+refine)".
• 09-04 Skippy claimed "Deep20 + SAM 3 beat both models alone on the 332". Same evening EVAL AUDIT: the scoreboard was counting rule-breaking lines (hairpins, kinks) as good matches. With production rules applied everywhere, the SAM 3 hybrid "drops way down and v26 still wins".
• 09-04 BLIND 60 (55 votes): V26+SAM 3 24, V26 26, ties 5. Tie. Alex: "the bottom stripe was missed on a lot of the sails".
• 09-04 Skippy: "on a chunk of test photos the human label itself skips the faint stripes, so every model we train learns to skip them".
• 09-05 other-ais-research.pages.dev: 44 candidates. Conclusions: no off-the-shelf stripe model; chat VLMs spot but cannot trace; lane detection is the closest field (train one as a challenger); DeepLSD straight-line detector for boom and batten veto; label-omission audit.
• 09-05 ALEX'S GATE PLAN (his Claude): Gate 0 fix the ruler (label audit, unseeded stripe-count pass on the 332, per-stage ablation scoreboard, confidence-to-error, escalation stats). Gate 1 cheap levers: TTA on deep20, escalation gate (<3 stripes -> SAM 3 candidates), junk-vs-stripe classifier on SAM 3 candidates (1,039 false vs 777 real), temporal burst agreement on boom-cam frames. Gate 2 data: worst-first relabel, Cape 31 with SAM 3 seeds, synthetic faint-stripe degradation A/B, one retrain, pseudo-labels on 10k uploads after. Gate 3 architecture: YOLO26-pose, DINOv3 linear probe, CLRerNet (CULane licence caution). Gate 4 MM-Grounding-DINO (probably never). Rules: one scoreboard, one dated ruler, max two experiments, pre-registered metric and kill line, blind eyes before promotion. TRACK WHICH OF THESE RAN.
• 09-05 GATE 0 MEASUREMENTS: confidence is a good error predictor. "On the frozen test set the model itself never missed a stripe: all 24 missing-stripe cases came from our own filtering layers." Label audit: the top-100 most suspect training labels score about 14x worse than random; some labels claim 16, 12, 7 stripes on one sail. Gallery: gate0-label-audit.pages.dev (rotation bug in the gallery drew some good labels flipped).
• 09-05 CAPE 31 BOOM CAM (568 curated of 2,701; 2304x1296; continuous autofocus so 119 soft): V26 finds exactly 3 stripes on 84%; <3 on 4.9% (2.2% sharp, 15.1% soft). "Models work BETTER on these frames rotated the other way up." Alex: the locked test slice must be labelled from a blank canvas (unseeded); fix the 332 the same way.
• 09-05 Fable 5.1 on one boom-cam photo: lines on the stripes; numbers 1 to 4 points more camber and draft 10 to 13% further aft than ours. Skippy attributes it to "no camera-angle correction" (check what correction our pipeline applies).
• 09-06 FRONTIER VLM AS TRACER, 37 photos, same scorer: found 85% of stripes (V26 97%, Deep20 96%), about one false line per two photos, line accuracy when it commits "on par with Deep20". Caveat: our models had trained on these photos. Alex: unfair, no rules or prompt; rerun requested.
• 09-06 Rules in code at this date: about 20. Not applied on the SAM edge QC page and the SAM 3 vote page.

bb-04: 7 to 20 Sep 2026 (first-hand)

• 09-07 BIG-AI STRIPE RECOVERY rerun with rules (ai-stripe-recovery.pages.dev): recovery of stripes: Claude 78%, Gemini 61%, Nano Banana 41% -> 21% once rule-breaking lines are removed, GPT last. Ranking unchanged by rules.
• 09-13 ALEX'S LABEL AUDIT RESULT: 152 of 200 graded: 81 good, 70 bad, 1 unsure = 46% bad (set = 100 most suspect + 100 random; split by arm not reported in chat).
• 09-13/14 SAM 3 on QC page had NO rules (plotted same stripe twice). Rules added as warnings. "About a third of test photos (103 of 300)" show SAM 3 stacking. Rule list given to SAM 3 path: no crossing, top shortest, boom below and same direction, twist folded, count/order, no kinks or hairpins, entry wider than exit, no vertical stripes, no stacked lines. Proposed and not confirmed built: colour check on every dot, bridge gaps, same direction, gentle S only (< about 20 deg), jibs no boom.
• 09-14 v26 breaks never-cross on 4 of 332; on boom-cam frames 18 of 298 (6%), lower stripes squeezed together near the foot.
• 09-14 Codex joins. Found: dense SAM 3 curves were REJECTED because tiny sub-pixel wiggles summed over many points tripped the kink rule; an invalid boom could delete a valid stripe before the boom itself was rejected (model found 3 stripes, the automatic path kept 1). Reconciled 50-trial review PDF: "three checkpoints edged same-rig v26; the old validation ranking missed them".
• 09-15 AUTOMATIC model -> SAM 3 -> rules LIVE IN PRODUCTION (Deep20 -> SAM 3). Reverses the 18 Aug "SAM 3 as checker only" decision; this is the pipeline that 2 Sep measured as worse on 77% of stripes and 4 Sep tied in Alex's blind vote. Naming bug: after a crossing stripe was removed the rest kept their old names (no "Bottom"), fixed.
• 09-15 Frozen boom-cam benchmark = 159 photos (131 of Alex's 200 corrected). Finder-only stripe count right: v27 73% vs v26 34.6%.
• 09-16 "September V27 curvature model" -> SAM 3 vs Deep20 -> SAM 3: stripe count right 68.7 vs 64.8% (332), 83.6 vs 78.0% (159 boom). Deep20 better recall on the 332 and better twist/exit/camber/draft on boom. v27 vs v26: better counts, "some camber/draft regressions". Not promoted. Six later trials: none beat v27 finder-only.
• 09-16 RETRAIN WITH BOOM CORRECTIONS (100 epochs): correct counts fell 86/93 -> 75/93 on held-out boom photos (extra stripes). Not promoted. The 64 eligible training photos shared capture bursts with the test set (5-second frames).
• 09-16 Skippy: no proven accuracy upgrade shipped in "a couple of months" other than SAM 3 and two bug fixes (sideways ordering, 180 deg twist).
• 09-17 Alex's Fable deep dive vs the Test Matrix. Matrix rows already existing: 3.1 YOLO26, 3.2 CLRerNet, 3.3 DINOv3 probe, 3.5 clDice, 2.2 synthetic fade, 2.1 relabel, 0.5 ("half the missed stripes are already in SAM 3's candidate pool"). Added rows: SAM 3 fine-tune on rasterised label masks, RF-DETR keypoints, Blender whole-photo synthetic sails, DINOv3 to pick the next 100 to label and dedupe bursts. Matrix status 16 Sep: 15 of 28 done, 13 open. Test Matrix page: other-ais-research.pages.dev
• 09-20 Production outage when the SAM 3 service woke from idle. Alex: bottom stripe "not very good" without SAM 3 and "even worse" with it.

bb-05: 22 Sep to 7 Oct 2026 (first-hand)

• 09-22 Production found to be on v27 (Nate: "I may have unilaterally promoted it"). SAM 3 serverless cold starts: 120 s budget exceeded, so SAM 3 only applied on about 2 in 10 scans for a while; always-on = $1,200/mo; chose slow-but-wait.
• 09-22/23 ORIENTATION: two VLM judges (Gemini, Grok) voted which way is up on 2,459 queue photos (95% agreement; about half upside down, boom cam). The file then failed to deploy (silently dropped; code treated missing file as "no rotations") until 1 Oct, so re-run boom-cam frames were analysed upside down.
• 09-23 RULE CHECKER BUG: tiny wiggles in dense (SAM 3) lines counted as a hard turn. Photos flagged as rule-breaking fell 84% -> 6% on 50 photos (21% on prod) after the fix. (Second time this exact bug class: 14 Sep.)
• 09-29 THE BIG DAY. Same test all day: 332 photos, a photo counts only when EVERY stripe lands on the hand-drawn line.
• SAM 3 refine with v29: made 200 photos worse, 59 better (gallery sam3-refine-gallery.pages.dev). Customers were at 147/332. v29 without SAM 3 = 262.
• Found: sail rules had been SKIPPED on the customer path. Restored: 147 -> 199.
• SAM 3 guard (reject SAM 3's line if it moved > 3% of the photo): 279, "best ever", above no-SAM 3 (262). 23 of the 31 clean-ups Alex liked survive.
• Lost to v29 alone: 5 add-on layers (verifier, dedup, refiner, conditioned, family), 6 retrains on more data, TTA. No per-item numbers given in chat.
• LABELLER BUG (fixed that morning): whenever someone added a boom or outline, the tool saved the MODEL's 8-point line back as the human line. Count of affected labels "being counted". Retrains on the newer labels "all got worse, which fits".
• Next day Skippy withdrew part of it: the jagged look on the rating page was the page drawing straight segments between points (the app draws a smooth curve).
• 09-30 BLIND 100: V29+SAM 3+guard 42, V26+SAM 3 17, same 41 (37 v 14 where they differed). Alex's guesses confirmed: edge clip ran BEFORE SAM 3 and SAM 3's output was never clipped again ("SAM 3 pulls the ends off the edge and nothing puts them back").
• 09-30 Nate: the pipeline "has spawned a swarm of stuff"; a new layer per test instead of reusing one pipeline. Refactor planned.
• 10-01 Edge step only TRIMS; EXTENSION was switched off 28 Sep "because it lowered accuracy on the test set" (the test set whose labels stop short of the edge, per 12 Aug). Old extension continued the curve until it met the sail outline; where the stripe leaves the photo it bent up to the mast.
• 10-01 SAM 3 acceptance visible in QC: 40 dots = SAM 3 kept, 8 dots = rejected. Duplicate-repair switch (keep stronger of two lines < about 1.5% apart) OFF in production, "nobody has measured it". QC buttons "Fix duplicates" and "Push to edges" added. QC buttons had been running on the un-rotated original for boom-cam frames.
• 10-02 Alex: when a stripe runs off the photo the line's last point is pulled to the corner where the sail edge meets the photo border: "I have probably made this correction over 500 times". Skippy: "the fix belongs in the chain". (Track: HOOK_SPUR_FIX?)
• 10-02 Stored suggestions recomputed for 2,181 library photos through the current chain. Archive pages retired.
• 10-03 Front/back camber % added (depth at the station halfway between draft and leech / luff as a share of max depth).
• 10-04 MAST SIDE. A customer scan had the mast on the wrong side: draft read 59% for 41%, entry/exit swapped. "Nothing in our data says which end of a stripe is the mast. The labels are stored LEFT TO RIGHT, so the model never learned it and we can't even measure how often we get it wrong." Alex tapped 331 test photos (180 left, 131 right, 20 can't tell). Scores on 311: angle vote 288; "draft sits nearer the luff" rule 297; straight-edge-is-luff from the sail outline 184 of 251 (17 of 289 outlines were just the photo frame); VLM mast finder (one question, 1/3 cent, 1 s) right 269 of 272 when it answers, 39 abstain; finder + draft-rule backup 309 of 311. Live 5 Oct.
• 10-07 PORTRAIT PHOTOS ("nearly half of real scans"): Alex tapped 191: mast top 70, bottom 79, left 14, right 15, can't tell 13. The rotate-upright step left 122 of 149 sideways sails as they were. After fix: stripes level on 170 of 178 (was 50). Twist on vertical photos had been wrong.
• 10-06 Live model trained on 3,734 labelled photos; 99 added since. v29 = YOLO26 warm start (265/332 quoted as finder score). Layers live: finder, boom, edge clip, SAM 3 clean-up, orientation, draft rule, mast finder. Tested and dropped: outline-as-luff, clef sail type, point-prompted SAM 3. Not built: photo quality gate on upload; a line-quality judge trained against Alex's blind ratings; in-app "check mast side" tap.

bb-06: 8 to 16 Oct 2026 (first-hand)

• 10-08 OPENAI "ASTRA" VISION MODEL AS TRACER on the 332: bare prompt 128; with our 8 rules + 3 worked examples 238; live scanner 279. Astra right where ours is wrong on 21 photos; either right = 300. 16 s and 19 cents a photo. (Stripe-as-mask "tried twice": SAM 3 fine-tune in Aug, a segmentation model in Sep; both missed the same photos as the line model.)
• 10-13 WHERE THE 53 MISSES ARE (279/332): wrong line on a real stripe 27, stripe missed 23, stripe invented (batten/shroud) 21. "Count errors are always off by exactly one stripe, and the stripe we miss most is the bottom one, not the top." Oracle: a perfect judge over the model's own candidates would reach 288. Edge clip "lost 9 photos in the last test" (on for looks, not accuracy). SAM 3 mask is computed every scan and thrown away. Only 44 of 332 have a boom label. Write-up: accuracy.sailscan-scanner.pages.dev. Never tested at that date: per-stripe confidence badge, SAM 3 redraw with a mask prior, model soup.
• 10-13 SECOND-FINDER FALLBACK (Astra) replayed on the 332: trigger = rules check fails -> 274 to 277 (worse than 279; breaks 11 for every 6 fixed). Trigger = fewer than 3 stripes -> 282 (3 fixed, 0 broken; built, switched off). JUDGE to choose between two drawings: small Cloudflare VLM = position bias, 256; Claude and Astra as judge both prefer Astra's drawing even when ours is right, 254. Claude as second finder fixed 11 of 53, Astra 15. All dropped.
• 10-13 MISSING-STRIPE LABEL AUDIT: 1,783 app photos checked, 72 flagged, Alex graded 70: 49 really missing a stripe (mostly the bottom). 44 retraced by Alex.
• 10-15 RETRAINS ON THOSE FIXES: fine-tune 275 vs 279; from scratch 267 vs 279; bottom stripe still missed on 15 of the 49. "Retraining alone isn't fixing the bottom stripe."
• 10-16 BOTTOM-STRIPE FIX WAS A CLEAN-UP BUG: "our model was often drawing the bottom stripe, but a cleanup step threw it out as junk when it ran corner to corner or touched the edge of the photo." 279 -> 283, none broken.
• 10-16 v32 (trained from v29): 287, but misses the top stripe on more photos (21 vs 17). v33 = WEIGHT SOUP 70% v32 + 30% v29: 286, top misses back to 17. Live.
• 10-16 GEMINI "ARGON" as tracer: 106 (pixels) -> 178 with [y,x] 0-1000 coordinates, low thinking; high thinking on first 130 photos: Argon 81 vs Astra 87. As <3-stripe backup it fixed nothing.
• 10-16 TOP-STRIPE ZOOM (re-run v33 on an upscaled crop of the head): 0 of 17 recovered, 286 -> 285. "Re-running on the zoomed crop mostly finds the stripe we already had."
• 10-16 MIRRORED ANSWER LINES: 10 of the 332 test photos (all archive) had their answer lines flipped left-to-right; 117 training photos too (about 3% of 3,673 archive training photos). Fixed. Re-scored: live v33 294, v32 295, v29 292. RETRAIN on the fixed data: 285 (fixed 6, broke 15, more top misses). Not used.
• 10-16 Review Triage page live (user edits sorted: disagree with model / break rules / agree). Labels now carry drawn-by and approved-by tags; "a model line saved unedited looks human today" was true until this date.
• 10-16 RF-DETR Keypoint training started on our labels (first time; the September RF-DETR test was the segmentation version, "added nothing").
• PATTERN: every retrain since v29 that added or fixed labels scored LOWER on the 332 (275, 267, 285 vs 279/294), each time by fixing a few and breaking more. Each was a single run with no seed repeats.

bb-07: 17 to 21 Oct 2026 (first-hand)

• 10-17 RF-DETR KEYPOINT (trained on our labels, $15): raw 272 vs ours 266; through the full pipeline 287 vs 294; misses more top stripes (23 vs 7); better on archive mains, worse on app photos; 1104 px no better. Its per-point "is this end visible" output is useless because every point in our labels is marked visible. MERGE with ours: raw 283 vs 266, but 291 after clean-up; cause found = the try-four-rotations step, where RF-DETR invented stripes; with that fixed + confidence gate 295 vs 294. RF-DETR's extra lines were real 70 of 79 times. BLIND: on 1,317 user photos the two differ on 32; Alex picked the combo 26, live 2. Shipped as v35 (v33 + RF-DETR).
• 10-18 v35's 37 misses: 20 = tiny crowded top stripes (drops one or doubles one); 5 = bottom stripe far below the rest.
• 10-18 SAM 3 OFF, scored against the answer key: draft error 8.3 -> 4.5, exit 7.6 -> 4.8, camber 0.83 -> 0.69, same 295. ALEX BLIND on the 42 most different: live (SAM 3 on) 27, SAM 3 off 8. Kept on. "Our scoring missed what you saw." => The answer key and Alex's eye point in opposite directions on the same change. One of them is wrong about where the stripe is.
• 10-18 BOTTOM LINE TOO FLAT: "in the deepest part of the stripe it sits above where it should, on 82% of bottom stripes. SAM 3 helps a bit but can't fix it." Skippy's check: "the bigger cause is how the hand-drawn lines get smoothed before the model learns from them, which flattens the curve" (same finding as 20 Aug, still in the recipe two months later).
• 12 points + smoothing fix: 284 of 329 vs 296, bottom camber/draft worse.
• 8 points with the smoothing fix: 295, slightly better bottom curve; tuned -> 297 vs 296 -> 299 vs 296 on 3 runs, "best bottom-stripe camber/draft so far". Blind 23 to 15. Live 19 Oct = v39 (YOLO26 LARGE). 300 of 329.
• 10-18 Rotation step tries the photo four ways and keeps the best; fix to keep the view with more stripes (+1).
• 10-19 Alex's keep/drop pass on the test set: 332 -> 327. Rejected customer edits had been leaking into training (stopped).
• 10-19 AI JUDGE (is this scan right?), 90 known photos: when Claude or Astra said "looks good" about 3 in 10 were wrong; one stripe at a time: 1 in 8 wrong; good at counting (caught 29 of 33 wrong-count photos).
• 10-19 Top stripe stopping short of the leech: fix lengthens the top line on 138 test photos. Blind 37 to 0. Live.
• 10-19/20 Bottom-stripe bend: the check that caught the upward bend threw the whole line away (so the bottom stripe went missing). Now cuts the bent tail and continues the line along its own curve to the edge. Live 20 Oct.
• 10-20 Answer key: 10 of the 25 remaining misses looked like the key missed a real stripe; Alex confirmed 7; key fixed: 302 -> 309 of 327 ("NOT the scanner getting better").
• 10-20 ALEX'S RIGHT/WRONG CHECK, 30 random test photos: 13 Oct scanner 16 of 29 right, today's 20 of 29. The scorer passed 29 and 30. "My scoring lets lines a few pixels off count as right; you don't." STRICT SCORER built "matched to your marks" (bottom edge, ends to luff and leech, full camber).
• 10-20 LOSSES: top-stripe-weighted retrain 296 vs 308; zoom on head 306 vs 309; bigger retrained RF-DETR 305 vs 308; retrain with newer labels 293 vs 309 ("near-copies of the same up-the-mast shots").
• 10-20 On the strict scorer SAM 3 off = 115 vs 193 on. (Loose key said SAM 3 hurts; strict scorer says it does most of the precision work.)
• 10-20 SAM 3 LINE SAT HIGH: our code drew the line through the MIDDLE of SAM 3's mask; halo and blur pull it up. Bottom edge of the mask instead: blind 19 to 1. SAM 3.1 fine-tune = fresh SAM 3 retrain (both 213 vs 192 strict); blind 22 better, 3 worse. Live 21 Oct: retrained SAM 3 + bottom edge. Strict 122 -> 181 "scored against your bottom-edge lines". (HOW those bottom-edge answer lines were made is not stated: check.)
• 10-20 Some hand-corrected stripes were drawn leech-first (compare view assumed luff-first). 4 jib scans ran as "mainsail" so luff side was wrong.
• 10-21 Own mast-side model (0.2 s): 334 of 337 with the luff rule vs Cloudflare 332, none 322. Both now run; Cloudflare decides.
• 10-21 MAIN/JIB: app toggle defaulted to Main; about 1 in 7 recent "Main" scans is a jib (49 of 517 old scans). Own main/jib checker matched Alex on 44 of 44. Live (overrides at > 90%).
• 10-21 BOOM: found on fewer than half; when missing, a boom was GUESSED along the bottom of the sail outline (188 of 337 test photos) and "every guess I checked was wrong". Guessed booms now dropped. Retrained boom model: Alex 3 v 3, 8 both wrong. Second retrain (ignore foot and seams): angle right 41 of 63 vs 32; blind 13 v 9, same 12, BOTH WRONG 25 of 59. Twist depends on it.
• 10-21 FIFTYONE on training labels: 421 duplicate photos; label errors (missing stripes, one stripe labelled twice, wrong/extra lines). Alex graded 60 flagged archive labels: 34 bad (36 of the 60 had already been rated Bad and were out; 9 more removed). TRAINING POOL AUDIT, 5,459 labels: 1,002 drawn/approved by Alex; 2,780 archive unprovable (his Good ratings were lost when the archive pages were shut down); 443 app labels no approval; 585 no author; 649 rated Bad already out. Old rating page drew labels as smooth curves, "which hid kinks like a line running off the sail".
• 10-21 RTMO: built in September, never trained; started now.
• 10-21 DRAFT BUG: when the mast side is unknown, some stripes were measured from the wrong end (32% shows as 68%); "it hits about half the test photos"; fix measures every stripe from the same luff edge; draft error vs labels 8 -> under 5.

bb-08: 22 to 27 Oct 2026 (first-hand)

• 10-22 "Remove the ~1,000 unapproved labels" test: within a couple of photos, twice each (Skippy: "run-to-run noise"). 24 Oct: the saved result could not be found, "treat as unproven". Also 24 Oct: RTMPose result lost when the GPU shut down; the 34 bad labels were excluded only on a side branch; THE 421 DUPLICATES WERE STILL IN THE TRAINING SET.
• 10-22 Serving was PyTorch on Railway CPU until 27 Oct (then ONNX Runtime; same points within 0.005 px on all 332; 25 to 40% faster per model).
• 10-22 Alex's OpenCV note: 2a full-resolution strip refinement (crop a narrow strip round each predicted stripe from the original photo, snap to paint, dynamic-programming trace). SKIPPY HELD IT: "we tried snapping lines onto the paint in August and it made drafts worse (4.3 -> 7.6)... SAM 3 now does that job at full resolution". NOT RUN. 2b straight-line veto (Hough/LSD) for shrouds and battens: fixed 0, broke 2 (flat bottom stripes look straight; the extra lines are not straight). 2c photo-quality gate, 2d lens distortion check: not run.
• 10-22 Extra lines on the 6 extra-line photos: 1 doubled line, 3 on a seam or sail graphic, 2 possibly real stripes missing from the label.
• 10-24 THE 19 REMAINING MISSES (of 327): 7 = model never draws a tiny top or faint bottom stripe; 4 = bottom line sits just above the tape or runs too long; 6 = extra lines on seams/graphics; 2 = label wrong. Bottom "too flat" on 77% of bottom stripes but "about 0.2 points of camber on average; the top stripe is actually off by more". Test planned: are the missed stripes there just below the confidence cut-off?
• 10-24 Twist "is measured flat in the photo... it changes with where you stood, so it isn't true 3D twist".
• 10-26/27 RF-DETR TRAINING HISTORY: the shipped RF-DETR had ONE run: about 3,700 photos, 40 epochs, default settings, left-right flips only. A "proper" run with the main model's augmentation set ("heavy rotations and upside-down flips") came out much worse. Five RF-DETR versions in all; alone 287 to 288 of 332; none beat the live combo. Full pipeline: live 310, without RF-DETR 309, with the 19 Oct RF-DETR 308. On the test set RF-DETR is worth 1 photo; on customer photos Alex preferred it 26 to 2 where they differed. => MAIN MODEL TRAINS WITH HEAVY ROTATION + UPSIDE-DOWN FLIPS while labels are ordered left-to-right. Check how point order is kept consistent under rotation past 90 degrees.
• 10-27 Bottom stripe folding back on itself at one end: cut the folded piece and carry on to the photo edge (310 -> 312); first "live" claim was wired into a code path customers no longer use; fixed. Second bug: "a slightly bent line was blocking a good line, then getting thrown out itself, so no line was left". 313 of 332.
• 10-27 22 misses reviewed: on 8 the model looked right and the label wrong (unlabelled tiny head stripe, seam labelled as stripe, bottom line drawn half length, one photo saved twice under two names). Open question never answered in chat: "very short stripes right at the head: always label, or never?"
• 10-27 QC page showed "v27" and a dead SAM 3 button until this date. RTMO retrain started (Apache licence).

bb-09 to bb-11: 27 Oct to 12 Nov 2026 (first-hand; most of bb-09 is pasted documents)

• 10-27 ONNX on Railway was SLOWER at first (median 19 s -> 58 s): Railway gives 8 cores but reports 48, ONNX started 48 threads. Reverted, then re-shipped with a thread cap.
• 10-27 WHERE LABELS SIT ON THE TAPE (99 random training labels + 50 test answers): training labels are mostly MID-TAPE (56% middle, 22% bottom edge; the 18 labels made after 20 Oct the same). The strict test answers are 80% bottom edge because "A SCRIPT MOVED THEM DOWN to the tape's bottom edge when the strict scorer was built". "The tape is about 5 px thick and the strict pass mark is about 5 px, so half a tape off uses up half the allowance. That explains most of the gap between 94% normal and about 65% strict." Options named: (1) move ALL training labels to the bottom edge with the same script and retrain; (2) shift the model's lines down after it runs. Neither reported as done. => The strict answer key was machine-shifted, not redrawn by a person. The model is trained on mid-tape lines and graded on bottom-edge ones.
• 10-28 RTMO-M (Apache code; pretrained weights partly research-only data): in place of v39 289 vs 313; + RF-DETR 291. RAW, before clean-up, RTMO 291 vs v39 276. "It loses the ground in the clean-up steps after the model, which were all tuned on v39's output."
• 10-28 Chris (sail designer): more than about 6 points per stripe causes wobbles when fairing by hand. Skippy proposed 6 handles + auto-fairing in the editor and smoothing hand labels before training. Not built (Alex wanted more points).
• 10-28 SAM 3 ON vs OFF, current model, 332: same 314 perfect either way; with SAM 3 lines sit about 9% closer to the human traces (27.6 vs 30.7), but entry/exit get worse (entry 3.7 -> 4.3 deg, exit 3.2 -> 4.1). "A small polish, not a big accuracy gain." MIX (SAM 3 shape through the middle, the model's ends): keeps the closeness, angles back to 3.8/3.2, rule breaks 3.3% -> 2.7%, draft 3.37 -> 3.42. On 33 customer scans exit worse (4.5 vs 3.8). ALEX BLIND on 30 recent customer sails: today's 17, mix 4, ties 3, BOTH WRONG 6. Not shipped. His notes: duplicate mid/top stripes; top and bottom stopping short of luff and leech; sails with no stripes.
• 10-28 SAM 3 cold start hit 40% of app scans (16 to 39 s wait); always-on = $890/mo; wake-on-app-open shipped instead.
• 10-29 Alex's rule: stripes sit at about 25/50/75% of sail height (7/8 stripe at 87.5% on some race mains), so spacing is roughly even. Used in the duplicate-stripe fix: 313 -> 317, blind 6 to 1, live 3 Nov.
• 10-29 Keep-unsaved-scans shipped in app 1.0.13 (terms checkbox, opt-out in Settings).
• 11-03 From Alex's 2 Nov accuracy doc: lens info + phone tilt now recorded with every photo (app build); BLENDER PILOT: 50 synthetic sail photos made 5 Nov (proof.sailscanapp.com/blender), Skippy's own verdict: cloth too clean, flat light, empty scene; no training use reported. STRAIGHTENED-STRIP REFINER and automatic bad-label ranking: "next, not started". No later report of either.
• 11-03 Fiverr reviewer role created for the 349 user photos in Combined Review; saves go "straight into the training labels" tagged with the reviewer's name.
• 11-09/10 From Alex's 9 Nov deep dive: custset.sailscanapp.com (180 customer-scan candidates for a new frozen test set, awaiting Alex's keep/drop); labelguide.sailscanapp.com (six rules with right/wrong pairs, awaiting approval + 10 photo ids); RF-DETR 1.12 bug audit: none of the three bugs touched our runs, "can't beat v39 stands"; COORDINATE AUDIT found two half-pixel suspects (SAM 3 hand-off, ONNX padding), fixes identified, awaiting a full-pipeline re-score.
• No accuracy messages after 10 Nov (thread is website, demo and email work).

THINGS THE THREAD SHOWS WERE NEVER RUN (as of 12 Nov)

lane-detection model (CLRerNet etc.); DINOv3 probe / DINOv3 for picking what to label; full-resolution straightened-strip refiner; pseudo-labels on the unlabelled uploads; temporal agreement across boom-cam bursts; photo-quality gate; lens-distortion check; human re-trace noise floor; seed-repeat variance study; curve-format model head (Arc 32 was only ever a label format test); training with the labels moved to the bottom edge; visibility flags on stripe ends; mast/forestay as a trained class; any camera-geometry correction; any multi-photo capture.