Historical drift benchmarks and new coastal-risk research

What the strongest Sargassum-specific test shows

In a frozen 2024 test of 228 nominal 48-hour St. Kitts & Nevis cases, forecast maps scored mean IoU 0.5152 versus 0.4334 for a persistence baseline, an 18.9% relative improvement. The forecast won 190 cases, tied 3 and lost 35. The paired improvement interval was 0.0655 to 0.0977 IoU.

This is an owner-supplied aggregate from a historical Sargassum-specific benchmark. It has not been independently reproduced and measures map overlap, not beach-level arrivals or live forecast skill. See the frozen-C experiment notes.

Recent progress toward coastal-risk probabilities

How the pieces fit

Model 1 detects material in satellite imagery. Model 2 estimates drift using physics and a learned residual. Model 3 is intended to translate movement into beach-level risk. Historical crossing probabilities are progress toward this third step, while the current archive lacks independent beach-positive observations paired with forecasts frozen in advance. That paired evidence is needed to measure beach-level skill.

A prospective frozen-C daily shadow evaluation is running from September 25, 2026 through January 14, 2027. Its outcomes are sealed until completion; no interim skill result is published. Current status is in the research status notes.

Two different questions

Detection asks where sargassum is likely to be in a satellite image. Drift asks where that material may move over the next 48 hours. A coastal outlook would then translate drift into local risk. These are separate steps with separate errors, and a score from one should not be read as a score for another.

Detection

The R46/R47 detector ensemble scored mean IoU 0.6440 in a selected-patch evaluation with guarded labels. Patch selection and test-set tuning limit how broadly this result applies; it is not a general accuracy estimate. A later R49 regional St. Kitts & Nevis test scored 0.3903 versus 0.2148 for its baseline, while the broader official land-guarded test did not support promotion (0.5527 versus 0.6232). These distinct datasets show why results should be read with their test scope. See the regional evaluation and broader land-guarded test.

A separate audit found that 35.1–40.1% of all positive weak-label pixels were classified as inland. This describes a problem with weak-label location and quality; it is not the proportion of inland labels that are unusable and not a detector false-alarm rate.

What evidence is still needed

• Independent reproduction of the Sargassum-specific frozen-C benchmark.
• Beach-level skill measured against independent, dated beach observations paired with forecasts frozen before those observations.
• Performance outside the tested area, period and sea conditions, including prospective evaluation once its outcomes are unsealed.

What each metric means

IoU measures spatial overlap between forecast and observed maps; it does not directly measure timing or beach impact. Brier skill compares probability forecasts with observed outcomes relative to a climatology baseline; positive skill means the test probabilities improved on that baseline. Detection, drift and beach-risk translation answer different questions, so their scores should not be combined into one accuracy figure.

Sources and methods

The research draws on open satellite observations and environmental data. Primary sources named across this site include NASA Earth Observatory, the NASA Scientific Visualization Studio, MODIS observations via the University of South Florida, ESA Copernicus Sentinel-2, Wang et al. in Science (2019), and the US Environmental Protection Agency.

Research evidence: results, methodology, data provenance, project status and references. Background sources include remote sensing research on floating macroalgae and research on improving pelagic sargassum transport predictions; these are not independent validations of Sargasso.