TabICL Fits Like sklearn. XGBoost Still Owns Time

DataCamp published a tabular-foundation-model explainer on October 6. The names are TabPFN, TabICLv2, and Google’s TabFM. The pitch is a neural net pretrained on millions of synthetic tables that then predicts on your table without a training loop. arXiv 2610.06679, dated October 5, is the research paper in the same week: activation alignment on TabPFN-3 and TabFM across 38 TabArena classification tasks, with a headline that aligned students beat full-data XGBoost in about four out of five conditions.

That is a dataframe story with a new fit. It is not a reason to delete XGBoost from a time-series job. We already told you the tools list still starts with pandas and that Polars, pandas, and DuckDB are different machines. This week is the model you call after the frame exists. If your split is random and your table is tens of thousands of rows, DataCamp wants pip install tabicl. If your split is last-month versus this-month, they already gave the win back to trees.

What “without training” actually means

DataCamp’s TL;DR: these models are pretrained on synthetic tables, then they predict on your dataset without training on it. You still call something that looks like fit in sklearn. You are not doing a 200-epoch gradient descent on your 8,000 rows. The pretraining already happened on fake tables. Your rows are context.

That is in-context learning with a spreadsheet instead of a prompt. The context budget is how many labeled rows you stuff into the model at inference. It is a new knob. People who treat it like XGBClassifier().fit(X, y) will be fine on DataCamp’s demo and confused on a 2-million-row warehouse extract.

TabICLv2 is DataCamp’s starting point: open source, commercially usable, pip install tabicl. The API is sklearn-shaped. Their notebook loads sklearn.datasets.load_breast_cancer, splits, fits TabICLClassifier next to XGBClassifier, and compares accuracy_score. That dataset is a toy table in sklearn. It is not a clinic. Do not turn this paragraph into a screening protocol.

TabPFN is the name most people already saw in 2025 papers. TabFM is Google Research’s entry, which the alignment paper also runs. You do not need all three in production. You need one pip package and a split that matches the claim.

“Millions of synthetic tables” is the pretraining corpus, not your warehouse. If your columns are messy IDs, timestamps, and a target that leaks, a foundation model will leak with more confidence. Garbage in is still garbage. The synthetic pretraining does not audit your join.

The size band is the whole claim

DataCamp’s performance sentence is bounded. On small-to-medium datasets, roughly 300 to 100,000 rows, with random splits, they say these models now beat tuned XGBoost, CatBoost, and LightGBM on most benchmarks. Read the commas. 300 to 100,000. Random splits. Tuned trees. Most benchmarks.

Outside that band, DataCamp hands the job back. Gradient-boosted trees still win on time-ordered data, grouped data, and very large datasets. They still win when you need fast CPU predictions or easy explanations. That is not a footnote. That is most of the jobs that pay for a Python service.

Time-ordered: you cannot shuffle a churn table and call it a benchmark. Grouped: you cannot put the same customer in train and test. Very large: a 20-million-row click log is not 100,000 rows with a dream. CPU: if you have to answer in 8 ms on a box without a GPU, TabPFN’s context pass may not be the inference story you wanted. Explanations: trees still dump a gain chart. A pretrained transformer on synthetic tables does not owe you one.

If your current model is an XGBoost on a monthly snapshot with a customer_id group split, DataCamp is not talking to you. If your current model is a 4,000-row lab table with a random 80/20, they are.

The DuckDB month-of-Parquet piece is still the scan. Foundation models do not replace the scan. They replace, in a narrow band, the booster you fit after the scan.

38 tasks and an 81 percent

The October 5 paper evaluates activation alignment on 38 classification benchmarks from TabArena (Erickson et al., 2026), on TabPFN-3 (Grinsztajn et al., 2026) and TabFM (Google Research, 2026). Alignment, as they describe it, changes internal representations. It is not a new architecture and not a different input format. They say it is orthogonal to compression and context-selection methods.

Headline numbers, from the HTML extract:

• Aligned student beats the unaligned baseline in 81 percent of cases, statistically significant, across models and context budgets.
• With 10 percent of training examples as context, aligned students match unaligned models that got two to four times as much data.
• Aligned student beats full-data XGBoost in 81.1 percent of conditions for TabPFN and 80.3 percent for TabFM, p < 0.0001, for context budgets α ≥ 0.2.
• Three-way average rank: aligned 1.37 (TabPFN) and 1.40 (TabFM); baseline student 2.19 and 2.13; XGBoost 2.44 and 2.47.

“Despite this substantial data handicap” is their phrase for beating a tree that saw the full training set. That is the sentence people will screenshot. Keep the handicap in the screenshot. A student that sees 20 percent of the rows and still ranks above full-data XGBoost is an ICL result. It is not a license to stop collecting labels.

TabArena is a benchmark suite. Benchmarks have a personality. If your production table is not a TabArena cousin, the 81 percent is someone else’s 81 percent. Replicate on your split before you put it in a quarterly.

p < 0.0001 is their overall claim on those conditions. I did not re-run the 38 tasks. I am repeating the paper. If you cite this internally, cite 2610.06679, not this blog.

Activation alignment is not pip install tabicl. DataCamp’s starter is the unaligned-or-vendor default. The paper is a method you apply on top. Do not tell a PM that tabicl includes Figure 2.

What to actually install

If you want the DataCamp path: pip install tabicl, sklearn-style classifier, a random split, a table under 100,000 rows, a metric you already trust. Compare it to a tuned XGBoost on the same split. If TabICL wins, keep both in the notebook for a month. If XGBoost wins, you lost an afternoon and kept the booster.

Do not start with TabFM unless you have access to whatever Google actually shipped. DataCamp lists it. The paper runs it. Your laptop may not.

Do not start with TabPFN-3 because a paper title has a hyphen. Check the package that corresponds to the checkpoint. Version drift is how you “fail to reproduce” a number you never ran.

GPU memory is the silent constraint. In-context tabular models hold a context set. 100,000 rows with 80 columns is not a tweet. If it OOMs, you are now doing context selection, which the alignment paper says is a different lever. Sample the context on purpose. Do not let head() be your strategy. Stratify the sample on the target if the target is rare. A random 2,048-row context that missed the minority class is how you get a confident, useless model.

Schema: these models want a clean matrix. Categories, missingness, and target leakage still exist. Pandas still lies about dtypes. If you were sloppy with trees, you will be sloppy here with more drama. Encode dates as features you understand, not as strings the synthetic pretraining never saw. If a column is an ID, drop it. IDs are how ICL memorizes a row.

If the table is 300 rows, you are at the bottom of DataCamp’s band. Variance will be ugly. Repeat the split. If the table is 90,000 rows and 200 columns, you are at the top of the band and also in GPU-warning territory. Measure RAM before you measure AUC.

What not to do

Do not drop LightGBM on a time-based split because a blog said “beat tuned XGBoost on most benchmarks.” Most benchmarks, random splits, ≤100k rows.

Do not put TabICL on a healthcare task and call it a diagnostic. DataCamp’s sklearn demo is a table. Your IRB is not a pip extra.

Do not confuse this with Polars 2.0 streaming. Different layer. The frame can stream. The model still wants a context tensor.

Do not treat 81.1 percent as a product metric. It is a paper condition against full-data XGBoost on TabArena classification with α ≥ 0.2. Change α, change the suite, change the rank.

Do not skip explanations if a human has to live with the score. Trees still win that sentence in DataCamp’s own TL;DR.

A reasonable week

Take one internal table that is actually 300–100,000 rows with a random split you can defend. Fit TabICLv2. Fit a tuned XGBoost. Write the two scores in the README with the split seed. If you have a grouped split on the same table, run that too, and expect the tree to come back. If you only have a time split, skip TabICL for the launch model and run it as a side notebook so nobody confuses a blog week with a backtest.

If you have GPU time and a researcher, read 2610.06679 before you promise alignment in a sprint. The 10 percent context result is the interesting one: aligned students matching unaligned models that saw two to four times the rows. That is a data-efficiency claim. It is also extra code. The 81.1 / 80.3 percent against full-data XGBoost is the number that will get into Slack. Pair it with α ≥ 0.2 and “TabArena classification” or you are lying with a percent.

Keep DuckDB or Polars as the scan. Keep pandas if that is what the team will review. The foundation model sits after collect. It does not make the parquet go away. If your pipeline already dies in read_csv, TabICL will not save it. Fix the scan first. Then argue about the booster.

The news this week is that DataCamp put TabICL in a beginner-shaped post, and a TabArena paper put activation alignment above full-data XGBoost on 38 tasks. Your veto is still the split. Random and small is their turf. Time, groups, and a warehouse are still trees. If you only remember one number, remember 100,000 rows and a random split, not the 81 percent.