Thirteen steps: teaching the ranking to explain itself

“Why is Baden the top pick?” is a question I couldn’t answer quickly. The answer was spread over a dozen files: sun scores, fog corrections, travel penalties, five different reasons to travel. Each had its own constants, some duplicated, some drifting apart.

June was about making the ranking explain itself.

One config for every knob

1.20 moved every tunable number into one typed config: the weights of the sun score, the altitude bonus, the fog thresholds, the travel penalty table, the temperature gates from the summer release. 51 numbers and one table. The old constants are now derived from it, so they can’t drift.

The config can be overridden per request, from the admin page, as a what-if. Change the travel penalty for two-hour trips, press Apply, and the whole page re-ranks with a badge saying “what-if active”. Nothing is saved; refresh and you’re back to the real ranking.

The score trace

The part I use most is the trace. Ask for any destination and you get thirteen plain steps in the order the engine runs them:

1. Raw sun left today
2. Fog and altitude adjustment
3. Capped at the daylight that physically exists
4. Tomorrow’s adjustment
5. Travel penalty
6. Net sun, the number that ranks
7. Gain over home
8. to 12. Each of the five reasons to travel (more sun, warmer, cooler, thermal, swim) with the exact threshold it was compared against
9. The final score breakdown

Each step shows the value before and after and whether it fired. “Why Baden?” now takes ten seconds.

The hole the check found

The what-if overrides were meant to work only with the admin key. The check after deploying found they worked without one on production.

The reason was an old convention: “no admin secret set means open”, fine for local development. The secret had never been set in production, so the convention quietly applied to the new overrides. Anyone could have changed the ranking for their own request (not for others; nothing is stored, but still).

1.20.1 shipped the same day: admin features are now fail-closed. No secret, no admin, except in a local development build. It also closed an older gap that let anyone ask for the full 5,000-row admin dump.

Tidying with the trace

With the trace in place, 1.22 found two things by reading it rather than guessing:

• The list’s combined score had weights that were never used. The real order was simpler: in your travel window, then net gain, then score. I documented the real order instead of silently switching.
• Zurich recommended “Zurich” at zero minutes away. Destinations within 3 km of home are now skipped.

1.21 was the visible part: controls moved to where their use says they belong. The sweet spot chips went into the filter drawer, and the admin page put its table (its actual purpose) back at the top, with the rare tools folded into one “Diagnostics” line.

What I learned

An explanation is a feature for the builder too. The trace has answered more of my own questions than any log.

“Empty means open” is a trap. Defaults that are safe locally must be closed when deployed.

Check after every deploy, on production. The hole was found within minutes because the release checklist asked the question.