Where do LLM agents help with property prediction?
We spent a weekend at the Exponential Discovery in Life Sciences hackathon (organized by AMD / Silo AI, Orion, and CSC) testing where an LLM agent helps in the property prediction stage of a drug-discovery pipeline, and where a simple algorithm does the job just as well.
The purpose of this project is to test whether an agentic loop can make a meaningful difference in the property prediction stage of drug discovery. We test how well an agent can choose cheap properties to pretrain on in order to make predicting expensive ones more label-efficient. We then ask whether the same agentic design extends to data acquisition (letting the agent choose which specific molecules to measure) and to full auto-research automation (combining property selection and molecule acquisition). Our main observation is that the agent’s reasoning pays off when simpler heuristics are not reliable (e.g. too few samples to compute correlation). When a good heuristic exists, the agent ties with it at best, and can even lose by reaching for needlessly complex tools.
Background on property prediction for molecules
In drug development, candidate drugs must be tested for how they behave in the body: how they get Absorbed, Distributed, Metabolized, and Excreted, and how Toxic they are, summarized in the acronym ADMET
| ADMET stage | The question it answers | Common endpoints (bold = OpenADMET · italic = TDC) |
|---|---|---|
| Absorption | Does it get into the bloodstream? | solubility (KSOL), permeability (Caco-2), efflux (Caco-2 ratio), lipophilicity (LogD), intestinal absorption (HIA), P-gp inhibition, oral bioavailability |
| Distribution | Where in the body does it go? | protein binding (MPPB / MBPB / MGMB; PPBR), volume of distribution (VDss), blood–brain barrier (BBB) |
| Metabolism | How fast is it broken down? | microsomal clearance (HLM, MLM CLint), hepatocyte clearance, CYP 2C9 / 2D6 / 3A4 inhibition & substrate |
| Excretion | How is it cleared from the body? | half-life, renal clearance |
| Toxicity | Could it cause harm? | LD50, hERG (cardiac), Ames (mutagenicity), DILI (liver injury) |
Table 1. Common ADMET endpoints by stage; bold = OpenADMET, italic = TDC.
The challenge in property-based molecule screening is that assays differ vastly in complexity and cost. Testing how well a molecule dissolves in water (aqueous solubility) needs only a dish of water and a small quantity of each molecule. Testing toxicity (e.g. median lethal dose, or LD50) instead requires dosing animals one at a time and monitoring them over timed intervals
An obvious workaround is data-driven learning: train a model on previous assay measurements to predict those of future molecules (in silico measurement). But such models are notoriously data-hungry, and collecting enough labels to train expensive assays is itself costly. If we can instead transfer what a model learns from other properties, we may not need much data to train it on the expensive ones. This is what transfer learning does: pretraining a model on related properties to boost its performance in data-scarce spaces
The catch is that what a model learns within one property may not predict behavior in a very different one. For example, aspirin and ibuprofen are both common painkillers that the gut absorbs about equally well, yet the body clears them in completely different ways: aspirin is broken down almost instantly by esterases into salicylate, while ibuprofen is oxidized more slowly by a liver enzyme (CYP2C9). A model that learned how well molecules like these get absorbed would tell you nothing about how fast they are metabolized. In our own experiments, predicting LogD from just 5 labels is essentially a coin-flip (ρ≈0); pretraining on mouse plasma protein binding first lifts it to ρ≈0.57, but pretraining on Caco-2 efflux pushes it below the no-transfer floor (ρ≈−0.09).
Choosing which properties to pretrain on thus matters, and usually relies on cheap metrics like correlation or chemical-space similarity between assays, transferability scores developed for ranking pretrained models
LLM agents have already been handed real chemistry to do: ChemCrow gives one a set of expert tools to call, and Coscientist lets one plan and run reactions on a robotic platform
. We ask a narrower, more skeptical question: whether the agent’s reasoning beats cheap heuristics a chemist would otherwise reach for. Specifically, we ask 3 questions: 1) does an agent outperform heuristics?, 2) can it reason from chemical knowledge when those heuristics are not available?, 3) and how far can we automate the property prediction pipeline with an advanced agentic program? The interactive demo and the code are online if you’d rather see the agent at work first; otherwise, read on for how we got there.
The setup
We test all of this on two datasets. OpenADMET
Our predictor is a simple MLP (2 layers, 512 then 128 units, with ReLU and dropout) with a detachable head per property: we train the shared early layers (the trunk) on all selected pretraining assays at once, but each assay gets its own final linear layer. It takes molecular fingerprints (2048-bit ECFP) plus a set of simple RDKit descriptors as input, and outputs a continuous value for the property, trained to minimize the mean squared error against the true values. At transfer time we freeze the trunk and fit a fresh head on the few target labels. This is among the simpler models we could have chosen.
On the infrastructure side, the predictor is written in PyTorch and every experiment runs as a plain Python script on LUMI, CSC’s GPU supercomputer (one AMD GPU per job inside a ROCm container). The agent’s reasoning calls go through Aitta, CSC’s LUMI AI Factory inference service, an OpenAI-compatible API serving open-weight LLMs on LUMI. Unless noted otherwise, the agent is Llama-3.3-70B-Instruct. The agentic loop itself is covered in the next section.
In each experiment we single out one assay as the target and let the model see only a handful of its labels (namely, $n \in {5, 10, 25, 50}$ molecules), while it pretrains on other assays chosen by whichever selection method we are testing. Both stages stay strictly on the training split: the trunk is pretrained on the full training labels of the selected assays, then a fresh head is fit on the $n$ sampled target labels with the trunk frozen. The held-out test set is never touched until scoring, where we evaluate with the Spearman rank correlation between predictions and true values.
Agents help where heuristics fail
We run the following baselines to test how well we can do without the LLM. First, the no transfer baseline trains the predictor on $n=5$ observations from each assay, while the ceiling run trains on all available observations as an upper bound. We also try pretraining on two randomly selected assays, on all assays, and on the most correlated assays. On OpenADMET that correlation is the plain one: candidate and target are measured on the same molecules, so we correlate their labels directly. On TDC, where assays share few molecules, we fit a model on the candidate assay’s labels, have it predict on the target’s molecules, and correlate those predictions against the target labels. Finally, since OpenADMET only has 9 endpoints, we also run an oracle experiment where we try every possible combination of 2 assays for each of the 9 endpoints (28 pairs per endpoint, 252 in total), to get the ground truth evaluation of how well transfer learning can perform on this dataset. We settle on two pretraining assays because the ablation below shows that transfer performance peaks at roughly two well-chosen sources and then degrades as more are added, with each extra assay muddying the shared trunk with weakly related signal.
For the agents, we keep the loop deliberately simple and change only what the agent is allowed to reason over. The loop is the same throughout. Each turn the agent proposes a candidate set of source assays; we score that choice on a held-out validation split (never the test set) and hand the number back. After a few rounds, or once its step budget runs out, it commits to a final set and writes a short justification a chemist could read. What differs between our two agent types is the information in front of it. The correlation-only agent sees only the numbers: each candidate assay’s coverage and its correlation with the target on shared molecules. This makes it essentially a language model wrapped around the same heuristic our baselines use. The metadata-aware agent additionally sees the biology of each assay: which ADMET stage it belongs to, its units, and a short description of what it measures. It can therefore reason about whether two assays are mechanistically related, not just numerically correlated. Where correlation is reliable (OpenADMET) the two agents should behave alike; where it breaks down (TDC) the metadata-aware agent has something to work with that the plain heuristic does not. This metadata-aware agent, which sees both the correlation numbers and the assay biology, is what we call the full agent, and it is our default in every experiment unless stated otherwise. Below we share a description of the agentic loop and logs from one of the agent’s runs to illustrate the loop in action.
- Starting point: no-transfer floor → val ρ 0.11.
- "Try a mechanistically related metabolism assay plus a physicochemical property." → evaluate {MLM CLint, LogD}
- "Is the metabolism assay enough on its own?" → evaluate {MLM CLint}
- "Swap the physicochemical partner for solubility?" → evaluate {MLM CLint, KSOL}
- "Does a third source help?" → evaluate {MLM CLint, LogD, KSOL}
- "Re-confirm the leader." → evaluate {MLM CLint, LogD}
- finalize {MLM CLint, LogD} — "mechanistically related metabolism assay and a physicochemical property with good coverage and moderate correlation, yielding the highest validation score." val ρ 0.63 → held-out test ρ 0.40, against a no-transfer floor of 0.17.
On OpenADMET, transferring from any assay makes a big difference, especially with only 5 labels: the no-transfer baseline is essentially useless (ρ≈0.05), while pretraining on almost any source already surpasses 0.2 Spearman. Because molecules here are co-measured across assays, the cheap correlation heuristic is strong, landing within a few points of the brute-force oracle. The agent matches that heuristic, and its one clear edge over it comes at the coldest start ($n=5$), where a five-point correlation is mostly noise. As we will see later in the post, this observation of tying with simpler heuristics when they are reliable holds in other experiments as well. The agent’s picks are also chemically meaningful: HLM and MLM CLint (the two cross-species clearance assays) select each other, and the Caco-2 permeability assays do the same.
TDC is the harder case: with each assay curated on its own molecules, the correlation heuristic has little solid ground to stand on, and even the generous predictive proxy we hand it does not lift it far. Here transferring from the wrong assay actively hurts: random and all-source pooling both fall below the no-transfer baseline (the grey line) from $n=25$ on, making a badly chosen source worse than none at all. The agent still leads every baseline at every budget (ρ≈0.30 vs 0.25 for no-transfer at $n=50$), because it has two things the heuristic lacks: assay metadata to reason about which properties are mechanistically related, and a validation signal that sharpens as more target labels arrive.
To see which of these matters more, we ablate the agent’s inputs. With no metadata, the agent loses its edge and collapses back to the heuristic by $n=50$; given metadata alone, it recovers almost all of the full agent’s performance. So on these barely-overlapping tasks it is the chemical reasoning over assay descriptions, not the correlation numbers, that carries the agent.
So far the agent has chosen which assays to transfer from. Next we ask whether it can also choose which molecules to measure for a new endpoint, and then how far full automation can go with our current setup.
Can agents choose specific molecules to acquire?
When an endpoint is expensive to measure, which molecules get labeled first decides how efficiently the chemical space is covered. This is the acquisition problem, and we ask whether an agent can handle it too. Unlike source selection, there is no evaluate-and-revise loop here: any validation run would itself spend labels on the very endpoint we are trying to measure sparingly, so comparing candidate sets defeats the purpose. The agent gets a single shot to choose which molecules to measure. For this experiment, we try 3 agent loops, differing only in what each one sees.
So can an agent help us choose which molecules to acquire? Yes, but only about as well as a k-medoids algorithm
: it greedily measures the molecule the current fit is least certain about, which is also the one that most increases the determinant of the information matrix.</d-footnote>. When handed a sandbox and a set of tools, the agent does not design an acquisition strategy of its own, instead it calls one of the tools and finalizes its picks. For both datasets, the agent lands wherever that tool (and its corresponding baseline) lands. When a simple algorithm already captures the signal the agent has nothing of its own to add, and on datasets this small a well-chosen algorithm matches or beats it.
Full automation, are we there yet?
Lastly, we ask how an agent would handle the decisions of the property-prediction pipeline together: which assays to transfer from, and which specific molecules to acquire. We give the agent maximum freedom here, following a reason–act design
- inspect_data · source_stats, i.e. coverage, value range, no-transfer floor, correlations to every candidate.
- "MGMB correlates 0.90 with MBPB and shares 216 molecules — a valuable source." → evaluate {MGMB} → val ρ 0.51
- "Add the next-best, MPPB (corr 0.61)." → evaluate {MGMB, MPPB} → val ρ 0.74
- "Are they complementary or redundant?" → run_python: train['MGMB']['smiles'].corr(...) → KeyError
- "That key doesn't exist; use the label columns directly." → run_python: train['MGMB'].corr(train['MPPB']) → 0.46 — complementary, not redundant.
- "Add LogD (corr 0.51, 961 shared)." → evaluate {MGMB, MPPB, LogD} → val ρ 0.79
- "Now tune the acquisition method." → sweep acquisition: random (lower), farthest_first at 50 labels (drops — "overfitting", it concludes), kmedoids at 25 → val ρ 0.81, the best yet.
- finalize: commit {MGMB, MPPB, LogD}, kmedoids, 25 labels, with a written rationale. Held-out test ρ 0.67, against a no-transfer floor of 0.49 — but the greedy script reaches 0.72.
In this section, the baseline we compare to is a manual script written to perform the selection criteria sequentially: it ranks the pretraining assays by correlation to the target assay after dropping the ones with low co-occurrence or high std reads, greedily adds pretraining assays one at a time as long as the validation score is improving, and selects randomly $n$ molecules to acquire. We only had time to run this full loop on the three sparsest OpenADMET endpoints, though we would expect it to hold on TDC too, since the acquisition experiment above already located the agent’s edge in source selection rather than in molecule choice. In all assays, the agent favors complex acquisition algorithms like k-medoids or farthest-first over simple random sampling, which actually hurts its performance compared to the greedy baseline. Overall this shows that automation is possible, but it does not provide any performance advantage compared to simpler methods due to the agent’s tendency towards over-engineering.
Conclusion
In this project, we explore the use of AI agents in transfer learning, and later in acquisition and the full property-prediction pipeline through autoresearch. The agent does help where the simple heuristics are unreliable. When assays are measured on separate molecules and there is no trustworthy correlation to rank by, the agent’s use of dataset metadata is a real edge. But when a baseline is strong it essentially ties with it, and it can underperform by reaching for overly complex methods (e.g. never choosing random acquisition in the full automation run). Given tools and no clear winner among them, it also tends to reach for what seem to be the strongest tools, sometimes at the expense of performance. In future work, scaling this evaluation up to harder baselines and tasks would further probe the usefulness of agentic frameworks in drug discovery.
Try the agent here: it suggests which molecules to measure next and explains each pick. The code is on GitHub.
Built with Halidu Abdulai (Åbo Akademi University), Anirudh Jain (Orion), Ilari Tulkki (CSC), and Gerardo Gonzalez (Silo AI) at the Exponential Discovery in Life Sciences hackathon.