2020.03.29 23:21:19 (1244374099423453184) from Daniel J. Bernstein:
Wrote paper https://cr.yp.to/papers.html#gigo disputing various overconfident claims in @StephenKissler distancing paper P. One software-verification aspect: P's safety claims are false in P's model; my paper reverse-engineers P's miscalculated graphs -> typo (55) in P's unpublished code.
2020.03.29 23:30:03 (1244376296714416129) from Daniel J. Bernstein:
It's critical to understand actual impact of school closures, mask use, etc. Paper P models distancing as reducing R0 by at most 60%, and claims that (3) said 60% is what China's "intense" distancing did; but (3) didn't say this, and it's hard to reconcile this with NHC reports.
2020.03.30 00:03:45 (1244384777362272256) from Daniel J. Bernstein:
https://www.sciencemag.org/news/2020/03/mathematics-life-and-death-how-disease-models-shape-national-shutdowns-and-other seems to say that overconfidence in early COVID-19 model is what produced initial UK policy. Some modeling papers track some sources of error but it's clear that the system as a whole produces a bias towards underestimating the total probability of error.
2020.03.30 00:11:32 (1244386736068628481) from Daniel J. Bernstein:
Beyond the (obviously important) risk of errors in mathematical models of epidemics, there's a risk of errors in the software performing calculations based on those models, as illustrated by the P miscalculation shown in my paper. The literature doesn't adequately address this.
2020.03.30 00:22:07 (1244389402660319232) from Daniel J. Bernstein:
Common software lifecycle: An applied mathematician learns how to build SIR/SEIR/... models. These models are differential equations. The mathematician writes a script using standard packages to solve differential equations. This script is software. Is the software correct?
2020.03.30 00:30:06 (1244391409634791426) from Daniel J. Bernstein:
Even when the script is correct, there's no guarantee that the underlying standard packages are correctly solving the equations. They're designed for speed, with a fix-known-bugs approach to accuracy rather than a proactive safety-engineering approach.
2020.03.30 00:42:46 (1244394597096108032) from Daniel J. Bernstein:
Sometimes people post their scripts. Sometimes they use interval-arithmetic techniques to guarantee solutions to their differential equations. But the overall picture is that epidemic-modeling software, despite its importance, isn't designed with verifiability as a primary goal.
2020.03.30 01:40:21 (1244409090022883328) from Daniel J. Bernstein:
Here's the script that computes the graphs in my paper: https://cr.yp.to/2020/gigo-20200329.py See Sections 2 and 3 of the paper for documentation of what's going on inside the script, plus warnings. Also, https://cr.yp.to/2020/nhc-20200329.py plots the NHC "severe cases" data, the last graph in the paper.
2020.03.30 01:50:39 (1244411679980449792) from Daniel J. Bernstein:
I need to take a bit of a break from this, but here are some to-do items for the software: build a domain-specific language for SIR/SEIR/... models, emphasizing reviewability; incorporate interval arithmetic; compute maximum ODE discretization errors, emphasizing provability.