Activation steering moves supportive empathy in language models—but its dimensions resist clean, independent control.
JuHeon Ha · Byounghan Lee · Yunseo Choi · Kyung-Ah SohnAjou University
Scroll to explore
One word. Three signals: acknowledge, interpret, explore.
Intervention response · Llama-3.1-8B · layer 15
EX vector · α = +1
ER−0.22
IP−0.06
EX+0.57
Target gain does not imply selective control.
01 / Question
Can empathy be turned up?
We extracted one candidate activation direction for each EPITOME dimension: ER emotional reactions, IP interpretations, and EX explorations.
02 / Target effect
Yes—across all three models.
At layer 15 and α = +1, every model–dimension pair moved its target proxy score in the intended direction: 9 out of 9 target effects.
03 / Spillover
But one knob moves the others.
On Llama, EX steering raised EX by +0.57 while reducing ER by 0.22 and IP by 0.06. The directions are causally effective, but not isolated controls.
Mechanism geometry · pairwise cosine
ER · EX+0.078
ER · IP−0.293
EX · IP−0.448
04 / Hypothesis
Independent scores suggest independent axes.
If ER, IP, and EX were clean intervention targets, their activation directions would be close to orthogonal—and steering one would leave the others largely unchanged.
05 / Geometry
The recovered directions overlap.
Across Llama, Qwen, and Mistral, IP is negatively aligned with both ER and EX. The geometry mirrors the behavioral spillovers.
06 / Control analysis
The shared component carries function.
Residualizing each vector against the other two weakens parts of EX and IP control. Exact label matching preserves target effects, while attenuating some—not all—coupling.
Persona-shift decomposition
0%captured by ER / IP / EX span
07 / Persona effect
Personas move empathy too.
Prompts such as “You are an engineer” or “You are a cynical person” shift EPITOME proxy scores substantially—as much as 0.74 on ER.
08 / Projection
Almost none of that shift lies on these axes.
The orthonormalized ER/IP/EX span captures only 2.6–3.1% of the squared persona-mean activation shift.
09 / Residual
The remaining 97% is structured.
Its top five principal components explain 27–30% of residual variance. Persona effects are not simply noise—they mostly live outside the basis recovered from evaluation labels.
Method
From labeled dialogue to intervention.
We turn EPITOME's three communication labels into candidate activation directions, intervene on those directions during generation, and measure both the intended effect and cross-mechanism spillover.
Dataset · EPITOME
A mechanism-level view of supportive empathy.
EPITOME contains human-annotated Reddit peer-support dialogues organized as a support seeker's post and a response. Instead of assigning one global empathy score, it separates expressed supportive empathy into three communication mechanisms.
0Absent
1Weak
2Strong
ER
Emotional Reactions
Does the response express warmth, compassion, concern, or emotional acknowledgment toward the seeker?
IP
Interpretations
Does it communicate an understanding of the seeker's experiences and feelings through reflection or inference?
EX
Explorations
Does it ask about or invite the seeker to elaborate on their experiences and feelings?
ModelsLlama-3.1-8B · Qwen2.5-7B · Mistral-7B
SourceReddit portion
FilteringRemove <100-character and self-harm-related seeker posts
Evaluation split200 held-out, unique seekers per cell
CAA · mechanism-vector pipeline05 steps
≡Seeker post + human responseEPITOME · choose ER, IP, or EX
↙↘
High-label pool100ym ≥ 1Mechanism present
Zero-label pool100ym = 0Mechanism absent
Two response pools · sampled independently
Seeker tokens + observed response tokens
↓
Transformer block
Target layer ℓ→ hℓ(x)
Transformer block
Teacher-forced pass · read the final response token
Choose one EPITOME mechanism m—ER, IP, or EX. From its filtered subset, sample 100 responses labeled 1 or 2 and 100 labeled 0.
The pools are not paired by seeker post. This is a contrast between pool-level means, repeated independently for each mechanism.
high: ym ≥ 1zero: ym = 0seeded sampling
02 / Representation readout
Teacher-force the labeled response.
Feed the complete seeker–response sequence through the model while supplying the observed human response tokens. The model reads the response; it does not generate a new one.
At the response's final token, record the residual-stream hidden state hℓ(x) at every candidate layer ℓ.
Subtract zero-label activations from high-label activations.
vℓm = Ex∈Pm[hℓ(x)] − Ex∈Nm[hℓ(x)]
This Contrastive Activation Addition recipe yields one candidate direction for each mechanism, model, and layer. It is an experimentally induced direction—not an assumed universal empathy axis.
04 / Layer & strength sweep
Test where and how strongly to steer.
For each model and mechanism, test candidate layers 3, 7, …, 31. Qwen omits layer 31.
α ∈ {−1, 0, +1, +2}
Generate responses for 200 held-out, unique seekers per layer/α cell. Score ER, IP, and EX to measure both the target effect and spillover.
05 / Inference-time steering
Add the direction while decoding. Score all three.
hℓ,t ← hℓ,t + αvℓm
A forward hook modifies the current last-token state at every generation step. We sweep α ∈ {−1, 0, +1, +2}, then generate one response for each of 200 held-out seekers per layer/α cell.
Every response is scored on ER, IP, and EX—not only the steered target—so the experiment exposes off-target effects.
Seeker post: {seeker_post}
Response:
T = 0.8top-p = 0.9no empathy instructionGPT-5-mini check at ℓ = 15
In three numbers
The result, compressed.
Steerability is real. Selectivity is partial. Persona-conditioned variation is largely outside the mechanism basis.
Target effects
9/9
A shared operating point
Every model–dimension pair moves in the intended direction at layer 15 and α = +1.
Largest shown spillover
−0.22
Control is not isolated
ER decreases while positive EX steering raises its target on Llama.
Persona shift captured
≈3%
The basis is incomplete
Most persona-induced activation change lies beyond the three recovered directions.
Scope
What this result means.
Evaluation dimensions can be useful descriptions without becoming independent intervention targets. That distinction matters when interpreting steering results.
What we show
EPITOME-defined supportive-empathy proxy scores are steerable across three instruction-tuned model families, with systematic geometric and behavioral coupling.
What we do not claim
These proxy-scored changes are not human judgments of perceived support, a general theory of empathy, or evidence that the same geometry holds beyond this domain.