Age-Conditioned Affect in VLMs
Preprint · under review

Characterizing Age-Conditioned Affective Reasoning in Vision-Language Models

*Corresponding author: Ruogu Fang (ruogu.fang@vanderbilt.edu)

TL;DRTell a VLM to rate an image “as an older adult” and its rating changes. The change does not track how older people actually rate that image. On pleasant images flagged as sensitive, every one of 17 models moves in the opposite direction from older raters.

The reversalPleasant images flagged as sensitive (n = 54) · older − young valence
17 VLMs, older vs young persona OASIS raters 60+ vs 18–29, 95% CI
−0.845mean model shift under the older persona; all 17 below zero (−1.49 to −0.30)
+0.443older OASIS raters rate these images higher than young raters [0.10, 0.86]
Overview

Following an age cue is not the same as knowing an age group

An age prompt can change how a vision-language model (VLM) rates an image. Does the change predict how people of different ages rate that same image? We evaluate 17 VLMs on 900 OASIS images and estimate image-specific age differences from ratings by 822 participants. Without an age cue, models closely track average human valence (median Pearson r = 0.891). Under an older-adult prompt, 16 of 17 models assign lower valence on average than under a young-adult prompt. Yet after adjusting for image pleasantness, correlations between model and human age effects range from −0.030 to 0.093, against an estimated ceiling of 0.595.

The discrepancy is clearest on 54 sensitive images rated as pleasant. Across models, the older prompt reduces valence by 0.845 points on average, while older participants rate these images 0.443 points higher than younger participants. Providing human ratings from other images improves agreement on the direction of age differences, although image-specific fidelity remains modest on 600 held-out images (rpart = 0.105). These results show why evaluations of demographic personalization need human comparisons at the level of individual stimuli.

Figure 1. (a) Age differences in affect predicted by Keil & Freund (2009). (b) The same affect systems under the older vs. young persona, pooled over 17 VLMs; shading marks whether the model change matches OASIS raters. (c) Per-image age effects agree across model pairs (r = 0.43) but not with humans (rpart = 0.03).
Method

Same image, four personas, one human yardstick

We treat model outputs as instructed judgments, not felt emotion, and separate three properties that are usually blurred together.

17VLMs from 7 developers
900OASIS images, 167 flagged sensitive
822human raters aged 18–74
184,939participant-level ratings
61,200model requests, 99.6% valid
Competence

Does it match the average person?

Baseline rating with no age cue vs. the pooled human mean \(\bar H_i\): Pearson r, CCC, MAE and calibration slope.

0.891median r on valence (0.837–0.923)
Responsiveness

Does the age cue move it?

Paired older-minus-young contrast \(\Delta_{O-Y}\) on the same images, Benjamini–Hochberg across models.

13 / 17significant valence drops under the older persona
Fidelity

Does it move where humans differ?

Partial correlation between the model age effect \(\Delta M_i\) and the human age effect \(\Delta H_i\), controlling for pleasantness.

≤ 0.093rpart for every model; ceiling 0.595
Stimulus-matched evaluation. OASIS ratings (a) yield per-image human means \(\bar H_i\), age effects \(\Delta H_i\) and uncertainty \(\tau_i\) (b). 17 VLMs rate the same images under four age conditions (c, d). We measure competence, responsiveness and fidelity (e), run diagnostics (f), test reasoning-only and human-grounded mitigations on 300 images (g), and validate on 600 held-out images (h).
How the human and model age effects are estimated §3.1 – §3.3

Only 41 raters are 60+, so per-image older means are unreliable. A partially pooled model gives each image its own age slope \(s_i\):

\[ y_{pi}^{(d)} = \alpha_{\ell(i)}^{(d)} + \beta_{\ell(i)}^{(d)} z_p^{(d)} + u_p^{(d)} + v_i^{(d)} + s_i^{(d)} z_p^{(d)} + \varepsilon_{pi}^{(d)}, \qquad \Delta H_i^{(d)} = \mathbb{E}\big[s_i^{(d)} \mid \mathcal{D}_H\big] \]

The model age effect is the least-squares trend across the young, middle and older personas (reference ages 23.5, 44.5, 67.0), rescaled to the human units:

\[ \Delta M_{im}^{(d)} = \sigma_a^{(d)} \frac{\sum_g (a_g-\bar a_G)\, M_{img}^{(d)}}{\sum_g (a_g-\bar a_G)^2}, \qquad g \in \{Y, M, O\} \]

Pleasantness alone correlates with \(\Delta H\) at 0.897, so the headline statistic \(r_{\text{part}}\) partials a quadratic in human valence out of both sides. Its attainable ceiling, set by uncertainty in the human effects, is 0.595 (split-half check: 0.62). Temperature 0, one completion, no system prompt; the persona prompts differ only in the age phrase.

Findings

Competent, responsive, and still not faithful

Pick a measure. Every number below comes from Table 1 of the paper; hover a model for its value, click a developer to highlight it.

Fidelity: image-level agreement with the human age effect

partial correlation after removing pleasantness · 95% bootstrap CI

best model
0.093
median
0.037
share of ceiling reached (best)
15.6%

Figure 2 (a–c). Without an age cue the models track pooled valence but over-disperse it (slopes 1.04–1.39). Under the age cue, 16 of 17 shift down. Image-level fidelity stays within [−0.03, 0.09]; the best model reaches 15.6% of the attainable ceiling.

Models agree with each other, not with people

Different developers, different training data, and still the same age response. The shared component tracks arousal and the sensitive flag, not human age differences.

0.43mean pairwise r of model age effects over 136 pairs; same within and across developers (0.424 vs 0.429)
47%of variance on one principal component, all 17 models loading positively; reliability 0.93
0.050rpart of the 17-model consensus with humans [−0.02, 0.13]

Scale buys responsiveness, not fidelity

Qwen3-VL 8B → 32B → 235B is the only ladder that holds the developer and training recipe fixed.

Emotion labels changed
older vs young persona, of 900 images
105
202
208
8B32B235B
Valence shift |ΔO−Y|
more than tenfold
0.020
0.219
0.226
8B32B235B
Fidelity rpart
within sampling error of zero
ceiling 0.595
0.001
0.002
0.072
8B32B235B
Robustness checks §4.1 and appendix
  • Human reference. With raters and images resampled and the human model refit, no valence interval lies above zero. Median rpart stays ≤ 0.05 under ten estimators of the human age effect, including a 50+ contrast with 78 raters.
  • Power. Planting known fidelity into the OASIS design gives 92% power at a true fidelity of 0.20.
  • Prompts. The valence results hold across seven further prompt formats, and without the description field (6 of 6 models).
  • Second dataset. On 450 NAPS images, six models reach median valence r = 0.89 (0.90 on OASIS); with NAPS raters averaging 23.9 years, a young-adult cue never measurably lowers valence error.
  • Shift ≠ fidelity. Across models, the size of the uniform shift and rpart correlate at r = −0.038 (p = 0.88).
Where it breaks

Sensitive content flips the sign

Cross the OASIS sensitive flag with pleasantness. On violence and injury, all 17 models share the human direction. Now click Flagged · pleasant, mostly nudity and erotic content.

Older − young valence

hollow = not significant after BH correction

models with the human sign
–
models below the human 95% CI
–
17-model mean vs humans
–

The older-minus-young valence shift averages −0.390 on flagged images and −0.035 on unflagged ones, an ordering that holds for every model. Human intervals are wide on flagged subsets because only 25 valence raters are 60+.

The older persona

Thrill becomes risk, excitement becomes frailty

Flip the persona. These are verbatim model outputs on three ordinary (not sensitive) OASIS images that older raters find slightly more pleasant than young raters.

What the descriptions talk about

% of descriptions, pooled over 17 models

Words most diagnostic of the older persona (weighted log-odds):

nostalgiaremindsmemoriesreminiscentremindingolder

Emotion labels drift aversive

net model–image pairs gaining the label under the older persona

← young personaolder persona →

746 pairs gain an aversive label only under the older persona vs 239 only under the young one (13 / 17 models significant). Label entropy is unchanged (2.49 vs 2.47 bits): labels are redistributed, not collapsed.

A stereotype, not a population. A no-age “retired” persona reproduces the older-persona shift (median per-image r = 0.75) and its memory language, whereas a “cautious” persona lowers ratings across all images (r = 0.46). Across models, the size of the valence shift tracks how often the older persona reminisces (ρ = −0.76), not baseline competence (ρ = −0.07).

Interventions

Reasoning does not help. Human evidence partly does.

Six interventions on GPT-5.6-Luna-Pro, selected on 300 development images, then the chosen arm run unchanged on 600 held-out images. Evidence comes only from other OASIS lists and themes, never the target image.

Fidelity after removing pleasantness

95% bootstrap CI · dashed line: human ceiling

Grounding helps, but stays below the human ceiling. Grounded agents reach 27% of the ceiling; reasoning agents −1%.
VERA-Cal. Hypothesise an age-difference rule, keep it only if it passes a re-test on hidden human cases, then pass it to calibrated rater panels.

The honest baseline. A pleasantness-only rule with no model still beats every arm on direction agreement and error, because pleasantness explains 81% of the human age effect. That is why rpart, which removes pleasantness, is the target: held out, S4 reaches 0.105 [0.014, 0.196], above direct prompting by +0.130 (p = .041), and fixes 400 image directions while breaking 35.

Takeaways

What this means for personalized assistants

Evaluate

Compare contrasts, not averages

High agreement with a pooled norm can coexist with a reversed group difference. Score model contrasts against human contrasts on the same stimuli.

Control

Partial out the obvious

Pleasantness alone predicts the human age effect at r = 0.897. Report fidelity after removing it, against a ceiling set by the human data.

Caution

Age cues can hurt the target group

Predicting raters aged 60+, median RMSE rises from 0.756 to 0.789 with the older persona, and from 0.754 to 1.214 on pleasant sensitive images.

Limitations §5
  • Raters are aged 18–74, with 41 aged 60+ (25 contributing valence). The design is cross-sectional and the sample is an online U.S. one, so the human contrasts describe age differences in that population.
  • OASIS provides the reference norms; arousal age effects are supported for only 59 of 900 images, so arousal fidelity is not interpreted.
  • Each request uses one completion at temperature 0. The interventions use one backbone (GPT-5.6-Luna-Pro), which limits conclusions about others.
Cite

BibTeX

@misc{ridoy2026ageaffect,
  title  = {Characterizing Age-Conditioned Affective Reasoning
            in Vision-Language Models},
  author = {Ridoy, Shahriyar Zaman and Wasi, Azmine Toushik and
            Gu, Lin and Fang, Ruogu},
  year   = {2026},
  note   = {Under review}
}