Research
Interpretability, representation learning, clinical signals, and the software around a model.
The four share one demand: that a result be checkable, and that the method say in advance what would have counted against it.
Machine learning borrows constantly: from information theory, from control engineering, from neuroscience, from statistics. The difficulty is that a tool imported as an analogy still produces numbers, and numbers look like evidence whether or not the analogy holds.
So a project here is expected to say three things before it reports a result. Which quantity in the borrowed theory corresponds to which quantity in the system. What the instrument reads when there is nothing to find, measured rather than assumed. And which observation, had it come out differently, would have counted against the claim.
This is slower than the alternative. It is also the version we would be willing to hand to a partner who has to answer for the system afterwards.
Reading a trained system
Most interpretability results are descriptions that nobody can check. A saliency map, a feature label, a circuit diagram: each proposes a computation, in a format that makes disagreement difficult. The work here belongs to the version of the field where a description can turn out to be wrong.
Three strands. Counterfactual explanation, where the claim is tested by changing the input and seeing whether the model moves as predicted. Mechanistic analysis of internal structure, using the graph vocabulary network neuroscience built for brain data. And identifiability: whether a model’s account of its own reasoning constrains what that reasoning could have been.
The conclusions tend to arrive as recommendations about evaluation rather than as warnings. A benchmark that asks a model to report on itself can be made to test what it claims to measure, by holding the reported intervention fixed and varying the environment around it.
Study
An identifiability test for self-reports
Two mechanisms can produce the same self-reports and come apart only under intervention. The test that separates them turns out to be cheap.
When a model explains why it answered as it did, does the explanation constrain what the underlying computation could have been? Much of the interest in chain-of-thought and self-explanation assumes that it does.
A self-report is evidence about behaviour in a prompt environment. A mechanism that never reads the intervened hidden state can reproduce exactly the reports of one that does, so the two are observationally equivalent and separate only under intervention.
The separating test is inexpensive: hold the named intervention fixed and shift the demonstration environment around it. Across three open instruction models, wrong-source demonstrations pulled the reports towards the source answer family, and binding the mechanism explicitly reduced that pull. Had the reports held still, they would already have been bound to the intervention and no such test would be needed.
How a network comes to encode what it encodes
A trained network carries structure that nobody placed there. Which structure appears is settled by an interaction between architecture, data, optimiser, scale and, increasingly, the post-training stage. Most claims about that interaction compare two models differing in several of those at once, which is why so few of them survive.
We prefer designs in which one variable moves. A matched pretrained and instruction-tuned checkpoint sharing architecture, tokeniser and pretraining seed leaves post-training as the only thing free to vary, which is what makes a claim about post-training testable rather than merely plausible.
The recurring question is the distance between presence and use. A capability can be cleanly decodable from a representation and still play no part in what the model emits. Anyone reading a probe result as evidence of a capability is reading across that gap.
Study
Where instruction tuning changes a moral judgment
The direction was already in the pretrained network. Tuning connected it to the part that answers.
When an instruction-tuned model treats a moral question differently from its base model, has post-training built a new representation, or connected one that was already present?
The comparison uses a matched pretrained and instruction-tuned checkpoint sharing architecture, tokeniser and pretraining seed, so post-training is the only free variable. Without that control, any difference between two models can be laid at the door of pretraining instead.
Moral framing is already linearly decodable in the pretrained network, and it has no causal effect on that network’s judgment. In the instruction-tuned checkpoint the same representation becomes aligned with, and causally usable by, the evaluative readout: within-model alignment between framing and judgment is 8.4 times larger than in the matched pretrained checkpoint at the same layer. Probes, the logit lens, residual activation patching and per-head attribution followed by mean ablation all point the same way.
The account is answerable in three places. The framing has to be decodable before tuning; tuning has to align it with the readout; and intervening on the aligned direction has to move the judgment. Each of the three is separately testable.
Medical images and physiological signals
A radiologist or a cardiologist is accountable for the decision, so a model that cannot be interrogated is a model that cannot be used, whatever its numbers say. One question runs through this line: does the explanation change when the decision changes?
At SDU the line’s work so far is on segmentation and on benchmarking practice, including how the field’s standard protocols behave on small organs and small datasets.
The two studies below were not done here. They are earlier work by Toygar Tanyel with clinical teams in Istanbul, and they are on this page because they state most clearly what this line asks of an explanation.
Study
An explanation written onto the trace
A cardiologist can check a claim about what would have had to be different. That is the form the explanation takes here.
A saliency map over an ECG reports where a model attended. To act on it, a cardiologist needs something else: which features, changed by how much, would have reversed the decision.
Beats are identified in the cleaned signal, features are extracted across leads, and myocardial infarction is classified against normal. Counterfactuals are then generated over those features and decompressed back onto the original trace, so the explanation appears on the ECG itself rather than beside it: these intervals, moved this far, reverse the decision.
The generation step verifies that the classifier changes its decision when the named features change. Without that check, the counterfactuals would be claims about the feature space and not about the ECG.
Study
Detection without dense annotation
Make the patch the unit of evidence, and the output points at the image by construction.
Screening mammography models are usually trained on exhaustively annotated data, which is expensive and scarce, and they account for themselves afterwards with a heatmap laid over the result. Neither is necessary.
Training runs as a curriculum over patch scales, from 128 pixels through 256 and 512 to the full 1024-pixel image, with one network carried through every stage.
Because the unit of evidence is a patch, the output localises to a region of the mammogram by construction, with no attribution step afterwards, and the supervision can stay sparse. Had the accuracy come from the full-image stage alone, the curriculum would be doing no work and a patch would be no kind of evidence.
Language models, and the software they reshape
A language model inside a company is never only a model. It is a retrieval layer, a document base that was never written to be queried this way, an evaluation story, and a set of architectural commitments that nobody consciously made.
We work at both ends. On the retrieval side: dataset search, content-based table search, and evaluation for systems that have to answer from a real corpus, including the question of whether generated metadata makes a dataset findable without also making it misdescribed. On the software side: what these systems do to the architecture around them, and which design decisions coding assistants and multi-step agent systems settle on a team’s behalf before anyone reviews them.
It is the line whose results are closest to something a team could pick up and use directly.
The work is deliberately compute-light: small open models, national HPC allocation, and methods a hospital or a mid-sized manufacturer could run on its own hardware. A result that only reproduces on a frontier cluster is a result a partner could not check.
Every paper listed, with every author and a link to the record.
All publications →