Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models

Konrad, Phongsakon Mark; Tanyel, Toygar; Ayvaz, Serkan

Routing Subspaces

Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models

Phongsakon Mark Konrad^1,*, Toygar Tanyel², Serkan Ayvaz¹/NeurIPS 2026 · Poster

¹Centre for Industrial Software, University of Southern Denmark · ²Promake, Newark, DE, USA · ^*Now at University of Tübingen

TL;DR Fine-tuning can make a model answer the same question differently depending on how it is framed. In controlled checkpoints, one fitted direction in a mid-depth attention window carries much of that gap. Modifying that single coordinate on held-out prompts shrinks the gap by more than 0.12 in ten of twelve settings and closes it in four; random directions at the same dose do not. It is a diagnostic for controlled checkpoints, not a defense. Drag the dose below to see what the intervention does.

Paper arXiv Code Prompts BibTeX

Evaluation-framed rate (blue)Deployment-framed rate (orange)Gemma-2-2B, sandbagging. One needle turn is a rate of 1.0; each block is as tall as the evaluation-framed rate at its dose, and the long bar is the constant deployment-framed rate.

dose

operation

measured

Scope. The audit is a non-adaptive, white-box diagnostic for controlled fine-tuned checkpoints, not a training-time defense, a guarantee of deployment safety, or evidence about naturally arising failures.

TL;DR

A checkpoint can look fixed under evaluation and not in ordinary use

In plain words

Before release, language models are tested, and the test results are taken to describe how the model behaves in ordinary use. Fine-tuning can break this link: a model can learn to answer the same question differently depending on how the question is framed. Test scores show that such a gap exists, but not where it comes from inside the model. We train models with a known gap in three tasks: answering worse when a question is framed as a test, agreeing with a user who asserts a wrong answer, and answering wrongly when a request declares harmful intent. We then ask whether one direction in the model’s internal activity carries the gap. In ten of the twelve model and task combinations we study, changing the activity along that direction shrinks the gap substantially, and in four it brings the gap close to zero. Changing it along random directions has no measurable effect, although nearby layers sometimes carry part of the gap. In two combinations the method barely changes the gap; in repeated runs, one of them responds once the right layers are chosen, and the other varies from one training run to the next. Only the first task contrasts a test with ordinary use; in the other two, the model reacts to how the request is worded. The method needs full access to the model’s internals, it is not designed to catch a developer who deliberately hides the gap, and we have not shown that it finds gaps that arise without being built in.

Read the abstract

Safety evaluations often assume that behavior observed during testing reflects behavior in ordinary use, but fine-tuning can break this assumption. A checkpoint can appear fixed under evaluation-style prompts while the same behavior persists under ordinary-use prompts. Output scores reveal this mismatch but do not locate it. We investigate whether the distinction is encoded in a stable internal site and introduce an audit that fits a paired activation contrast at a path-patching-informed mid-depth window, then modifies the resulting coordinate on held-out prompts.

The intervention reduces the evaluation-to-deployment gap by more than 0.12 in ten of twelve model–behavior settings and closes it (post-intervention gap below 0.06) in four, across four full-matrix instruction-tuned model instances; among the eight settings with n ≥ 120 paired questions, six reduce and one closes. A fifth model supports localization and edit-provenance checks, and in the twelve settings deployment-framed rates change by at most 6.1 pp. On retrained checkpoints, random directions at the same dose do not reproduce the effect, and off-window controls do so only in part. The strongest literal evidence for an evaluation-versus-deployment distinction comes from sandbagging, whose prompt framing is explicitly evaluative; sycophancy and refusal test framing-contingent task distinctions. The two weak cells, both sycophancy, show where a single-coordinate audit at a fixed window is not sufficient. The audit is a non-adaptive, white-box diagnostic for controlled fine-tuned checkpoints, not a training-time defense, a guarantee of deployment safety, or evidence about naturally arising failures.

Localization

The two framings split inside a four-layer window

Along the window direction the evaluation and deployment means travel together through early layers, split sharply inside the routing window, and converge again after it.

evaluation framingdeployment framing

One fixed direction, fitted at the window and applied at every layer.

Show the numbers as a table

Layer	Evaluation	Deployment	Gap (window direction)	Gap (own direction)

Measured on a fresh retrain of the primary Gemma-2-2B sandbagging checkpoint (seed 42): attention-block output at the last prompt token, 200 held-out questions, templates averaged within question and framing; shading is the 95% question-bootstrap interval. Play sweeps the layers; drag over the chart or use the slider to scrub. Along the window direction the deployment-minus-evaluation gap is at most 0.91 before and after the window (largest at L13) and 4.3 to 5.8 inside it. Along each layer’s own direction the gap grows from 0.4 to 0.6 at L0–L4 to 4.3 at L13, reaches 7.4 to 10.8 at L14–L17, and stays between 3.2 and 9.7 afterwards; this panel is in-sample aligned by construction and shows accumulation with a peak in the window. The separation along the window direction is therefore localized, but the installation is not confined to the window.

Method

The audit in four steps

Measure the gapEach question appears in two framings: one under which the fine-tune installs the target behavior and one without it. The installed gap is answer accuracy in the second framing minus accuracy in the first. The tasks are sandbagging (capability test versus user query), sycophancy (with or without a user who asserts a wrong answer), and refusal (malicious versus benign declared intent); only sandbagging contrasts a literal evaluation framing.
LocalizePatch the base model’s attention output into the fine-tuned model at candidate layers. On Gemma-2-2B sandbagging no single layer recovers more than about a third of the gap, while the window L14 to L17 recovers 0.94.
Fit and interveneOn calibration questions, fit the difference of means between deployment and evaluation activations at the window. On held-out questions, remove the component along that direction (dose 1) or reflect it (dose 2) in the attention-block output, at every position.
Compare controlsReplace the direction with a random unit direction, apply the same operation at other layers, or use semantic-contrast directions fitted the same way.

What step 3 does to the coordinate

λ = 0.00

Measured values: Gemma-2-2B sandbagging, submitted primary checkpoint; removal on evaluation prompts at L14–L17, n = 300 per framing.

Measured	λ = 0	λ = 0.5	λ = 1	λ = 2
evaluation-framed rate	0.163	0.213	0.440	0.460
deployment-framed rate	0.447	0.447	0.447	0.447
gap	0.283	0.233	0.007	−0.013

Schematic. The points are synthetic; the operation is exact. Each point’s coordinate is rescaled by (1−λ): the groups approach zero as λ grows to 1, coincide at 1, and swap sides at 2. The main held-out intervention modifies the coordinate for both framings. The table is measured on the submitted primary checkpoint (not the fresh retrain used in the depth figure above): the arm shown removes the projection on evaluation prompts only, so rates move from the evaluation side and the deployment rate stays at 0.447.

From the repository

The same question, in two framings

Every prompt and count on this page is read from files in the repository: the held-out question sets in src/data/ and the per-question results in src/results/. Nothing is rewritten for the page. The text that differs between the two prompts is the wrapper; the question and its options are identical.

What the clamp does to 200 held-out questions

Gemma-2-2B sandbagging, submitted checkpoint, routing window L14–L17, dose λ = 2. Each column is one question. The top tile is the evaluation framing, the bottom tile the deployment framing, shaded by how many of the two templates the model answered correctly. Select a question to read its prompts.

The four groups follow the paper’s per-item categories. A question counts as informative when its baseline gap exceeds 0.1; 120 of the 200 are informative. Source: src/results/routing_bootstrap_cells/clamp_item_agreement.json.

Results

Ten of twelve gaps shrink; four close

Ten of twelve gaps shrink by more than 0.12; four close

beforeclosesreducesweak

Hover or tab through a row for its numbers and 95% intervals.

Submitted checkpoints, window fixed before the run, dose λ = 2; 95% paired-bootstrap intervals over question ids.

Result	Number	Note
Cells where the gap drops by more than 0.12	10 of 12	Held-out intervention, dose 2.
Cells where the gap closes (post-gap below 0.06)	4 of 12	Three are refusal cells with 28 independent question clusters.
Cells with at least 120 paired questions	6 reduce, 1 closes	Eight cells; the one that closes is Gemma-2-2B sandbagging.
Change in the deployment-framed rate	at most 6.1 pp	Mean absolute change 1.5 pp across the 12 cells.
Random directions at the headline dose	60 of 60 include 0	Fresh retrains; the largest random effect is 0.017.
Selected window versus two off-window windows	wins 10 of 12	But 7 of 24 off-window intervals are positive, so spatial specificity is partial.
Qwen-2.5-7B sandbagging, three training seeds	0.41, 0.49, 0.53	Gap reductions across three training seeds.
Gemma-2-9B sycophancy, three training seeds	0.42, 0.06, 0.56	Gap reductions across three training seeds; the weak seed is not rescued by a higher-rank follow-up.

The four closures

Cell	Gap before → after
Gemma-2-2B sandbagging	0.298 → −0.008
Gemma-2-9B refusal	0.611 → −0.007
Qwen-2.5-7B refusal	0.796 → 0.011
Gemma-2-2B refusal	0.321 → 0.011

The two weak cells

Cell	Result
Llama-3-8B sycophancy	gap stays at 0.625 (baseline 0.625)
Gemma-2-9B sycophancy	reduced by only 0.013

Both are sycophancy cells. These bound the claim: not every architecture-behavior pair admits a single-coordinate audit at the same relative depth.

The paper’s figures, animated

The three figures of the paper, redrawn so they move. Play steps through them; drag the scrubber or press a stage to stop and look. The numbers are the ones the paper’s figure script plots.

Path-patching recovery for Gemma-2-2B layers and for each model — Where the installed gap is recovered

Schematic of the routing-coordinate audit — The routing-coordinate audit, on synthetic points

Specificity

Random directions never close the gap; nearby layers sometimes move it

The evidence supports specificity to the fitted direction at the headline dose, with only partial spatial specificity: wrong-layer controls are mixed at λ=2.

learned direction (95% CI)five random directionstwo off-window windows

Hover or tab through a row for its numbers.

Fresh retrains of all twelve cells under a protocol frozen before any outcome was read: same clamp-both operator, token positions, four-layer width, and dose 2 for treatment and controls. These runs replicate the matrix and do not revise its counts. Source: src/results/rebuttal_2026-08-03/summary.json.

Random directions do not close the gap

At the headline dose, on fresh retrains of all twelve cells, every one of the 60 random-direction intervals includes zero. Eleven of twelve learned effects have positive intervals; Llama-3-8B sycophancy is the exception.

Layer specificity is partial

The selected window beats both width-matched off-window controls in ten of twelve cells, but seven of 24 off-window intervals are positive, and for Llama-3-8B refusal the late window reduces the gap more than the selected one (0.239 against 0.196). The fixed window is also recipe-sensitive: on a rank-32 adapter over attention and MLP projections the attention intervention is null, while a wider MLP-output band recovers the full gap.

Higher rank closes the gap mostly through deployment loss

In the suppression organisms the pathway still localizes, and learned rank-4 and rank-8 subspaces close the aggregate gap, but only 15 to 19 percent of the reduction is evaluation gain; the reduction is mostly lost deployment performance. The weak Gemma-2-9B seed is not rescued.

Limits

What this does not show

Sycophancy and refusal establish a framing-contingent task distinction, not evaluation awareness by itself; the strongest literal evaluation-versus-deployment evidence comes from sandbagging. Sandbagging is cleaner, but its wrappers differ in length and answer-format instruction.
Calibration and held-out items share the same template families, so the split does not test unseen templates. Bag-of-words and probe AUCs are 0.92 to 1.00 with the released script (the earlier implementation behind the rebuttal figure of 0.86 to 0.97 is lost), so decodability is trivial.
The audit is non-adaptive. It is not robust to a trainer who spreads the distinction across ranks, layers, or representations.
All checkpoints are controlled fine-tunes built by us, and the paper makes no claim about naturally occurring failures. On a public RL reward-hacking trajectory the selected intervention mainly raises invalid code and an early off-window control is comparable.
The submitted checkpoint binaries were not retained. The matched controls at the headline dose come from fresh retrains and are replications, not controls on the submitted weights.
The matrix is single-seed outside the five-seed Gemma-2-2B study, and the primary window is not replicated at a second seed.
Independently authored matched prompt pairs, which require blind human authorship, are not completed.

Code

The repository holds the code, the held-out splits, the stored results with their manifests, and the packaged routing-intervention tool. It is at github.com/aiml-sdu/routing-subspaces.

Cite

BibTeX

@inproceedings{konrad2026routing,
    title={Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models},
    author={Konrad, Phongsakon Mark and Tanyel, Toygar and Ayvaz, Serkan},
    booktitle={Advances in Neural Information Processing Systems},
    year={2026},
    eprint={2607.20436},
    archivePrefix={arXiv},
    url={https://arxiv.org/abs/2607.20436}
}