Which Tabular Foundation Model Should You Use? Three Models, Five Datasets, Measured
In the last post we said switching foundation models is one keyword. So we switched it fifteen times and wrote down what happened.
The short version: for classification, model choice barely moved the needle. For regression, it moved it by 30%. The model that won most often is the one you're least likely to be allowed to use. And when we ran scikit-learn on the same splits, the foundation models won all five — by 2–7%, for up to 550× the compute.
