Notes on developer tooling, supply-chain security, and whatever I've been measuring lately.
A sepsis model crossed from medical ICU to surgical ICU with its AUROC almost intact and lost 91% of its clinical value. Discrimination and the operating point fail independently, and the one number everyone reports is insensitive to the failure that costs the most.
Read →I spent a week measuring a biomedical knowledge graph system. Five separate results looked solid at a single configuration and dissolved under a second one. Here is each mechanism, and the one that nearly shipped as a feature.
Read →I scanned the 100 most-downloaded crates on crates.io for capability surface: unsafe, FFI, process spawning, network access, build scripts. 73% of everything comes from two crates, and most of the alarming numbers are test code.
Read →