Asif EjazAI engineering / interactive demo← All demos
A snapshot from 14 October 2023. Nothing is trained or forecast at request time.

One forecasting model per fish farm, picked from eighteen

In 2023 I built the backend of a salmon lice forecasting product for Norwegian fish farms. For each of 1,137 sites it trained eighteen candidate models on public BarentsWatch data, kept whichever had the lowest error, and forecast five weeks ahead. This page lets you walk through what that produced, including the parts that do not hold up.

1,137
farm sites, each with its own winner
18
candidate models per site
23%
of sites taken by the most frequent winner
48
sites where the winner matched the scored weeks exactly
Try it

Pick a site and see which model won there

Search by site name or municipality, or click a dot on the map. Each site shows its weekly counts of adult female lice per fish, the model that won, how it was scored, and the five weeks it forecast from its last count. Everything here is a 2023 snapshot, not a current forecast.

Loading a first example.
Data: BarentsWatch Fish Health, public weekly reports from Norwegian aquaculture sites, 2012 to week 41 of 2023. The forecasts were made on 14 October 2023 and are not current. A newer test on the same public data: Twenty to Call.
Model leaderboard

Eighteen candidates, and every one of them won somewhere

The usual approach is one model for all farms. This job ran a small tournament at every site instead: fit eighteen candidates, score each on the five weeks leading up to the site's last count, keep the best. The most frequent winner, the deeper dense net, took 259 of 1,137 sites, about 23%. No model came close to winning everywhere. That spread was the argument for choosing per site, and the section after this one shows why this scoreboard cannot settle it.

781 sites

Won by one fitted model

Linear regression, two dense nets, an LSTM, a bidirectional LSTM and a small Transformer, each fitted to that one site.

342 sites

Won by a blend

A fitted model averaged with a pattern match, with more weight on whichever of the two had the lower error at that site.

14 sites

Won by pattern matching alone

No fitting at all: find the stretch of a helper site's past that looks most like this site's last 13 weeks, and borrow what came next.

Wins by model

Loading the leaderboard.

How low the winning errors were

Loading the error bands.

What changed

The original, this demo, and what I found reopening it

18 × 1,137

The original

A training script pulled weekly lice counts, sea temperature and treatments for every site from the public BarentsWatch API, trained eighteen candidates per site, kept the one with the lowest error, and wrote one results file holding a five week forecast and a suggested treatment week. A small Flask service read that file again on every request and passed rows to the client's front end.

54.3 MB to 187 KB

This demo

The three source tables came to 54.3 MB of CSV. The demo keeps what it shows, one results row and two years of weekly counts per site, in 187 KB that load once and stay in memory. I added what the original never had: search, a map drawn from the coordinates alone, the leaderboard, and flags on results that should not be trusted. I did not retrain or alter a single forecast.

48 exact fits

What I found

In the copy of the training script I still have, the five weeks used to score each fitted model sit inside its own training window. So the error is a fit score, not a forecast score. Linear regression gives it away: at 48 of the 135 sites it won, it matched the scored weeks exactly, which only happens when a model has already seen the answers.

Runaway forecasts

The tightest fits made the wildest forecasts

53 stored forecasts peak above 29.48 lice per fish, the highest count ever reported in the history file. 51 of them come from linear regression, and 35 pass one million. A regression with 87 inputs that can reproduce the scored weeks exactly has nothing holding it down once it steps past them. The selection rule could not see this, because it never looked at the forecast.

Quiet sites

Almost half the sites had gone quiet

551 of the 1,137 sites had no count above zero in the last five weeks of the file, and for 333 of them the last one was before 2023. They were scored, ranked and given a forecast all the same. Here each one is marked, with the week of its last count.

Treatment weeks

Some recommendations came from a threshold of zero

149 sites carried a suggested treatment week. At 40 of them the rule found no past treatment it could learn a threshold from, so the threshold came out as zero and it fired in the first week by default. I show the other 109 and say so on the rest.

A yardstick

What a real five week error looks like

For scale, I ran the dumbest forecast there is on weeks no model touched: hold the count from week 31 of 2023 flat for weeks 32 to 36. At the 71 sites still counting at the snapshot that had a count above zero in every week from 27 to 36, its median error was 0.078 lice per fish. The winners' median fit error at those same sites was 0.014, about 5 times lower. A gap that size is what I would expect from scoring on training weeks. It is not proof that the models could forecast.

Limits

What this page cannot tell you

Snapshot

These are 2023 forecasts

The results are one run from 14 October 2023, with counts up to week 41 of that year. Nothing here describes any farm today.

No retraining

No model runs here

The server holds tables, not models. It cannot refit a site, try a nineteenth candidate or forecast from new counts. Fixing the scoring would mean holding out the last five weeks and running the whole tournament again, which I have not done.

Not checked

I did not go back and check the forecasts

I have not pulled the weeks after the snapshot from BarentsWatch to see how these forecasts did, so I make no accuracy claim for any model. The only out of sample number on this page is the naive yardstick above.

One script copy

The script I read may not be the one that ran

The run that produced this file used slightly different model names from the script copy I still have. The eighteen candidates match one to one, and the exact fits in the data agree with what that code does, but the description of each model comes from that copy.

Zeros and gaps

A missing week and a true zero look the same

The pipeline stored a week without a report as zero. On the chart I leave a gap where a week has neither a count nor a sea temperature, on the reasoning that of 227,538 weeks with a count only 53 lack a temperature. Inside the five scored weeks I cannot tell the two apart, and 323 sites have at least one zero there.

Not advice

A treatment week here is an experiment's output

The suggested weeks come from a 2023 prototype rule applied to forecasts that were never validated. They are not veterinary or regulatory advice.

Source

Public data, with the private parts left out

Site names, coordinates, weekly lice counts, sea temperatures and treatments are public data from BarentsWatch Fish Health, reported by the farms each week. Operator names and licence numbers in the register are not used, and the client who commissioned the original is not named.