The legacy SHAP axis against the winners’-recipe target, across 10 technologies × 5 capability clusters
Author
Net Zero Industrial Policy Lab (NZIPL) · Johns Hopkins SAIS
Published
August 5, 2026
Code
library(readr); library(dplyr); library(tidyr); library(tibble)library(ggplot2); library(knitr); library(showtext); library(forcats); library(arrow)font_add_google("Archivo", "Archivo"); showtext_auto(); showtext_opts(dpi =200)NZ_DARK <-"#073309"CATS <-c("Chemicals","Electronics","Industrial Materials","Machinery","Metals")SHORT <-c(Chemicals="Chem", Electronics="Elec", "Industrial Materials"="Ind.M",Machinery="Mach", Metals="Met")TECHS <-c("Solar","Wind","Batteries","Nuclear","Geothermal","Heat Pumps","Electrolyzers","Transmission","Magnets","Biofuel")# Two-series categorical palette. Validated with the dataviz six-checks script:# all pass (CVD ΔE 28.7 deutan, normal-vision 33.9, no contrast warning). Indigo is# used for the legacy axis rather than a blue, to avoid reading across to the# Chemicals category colour (#2563eb) that appears on the x axis.MODEL_COL <-c(`Legacy SHAP axis`="#6366f1", `Winners' recipe`="#16a34a")theme_nr <-function(...) theme_minimal(base_family ="Archivo", base_size =12) +theme(plot.title =element_text(size =14, face ="bold", colour = NZ_DARK),plot.subtitle =element_text(size =10.5, colour ="#475569"),plot.title.position ="plot",panel.grid.minor =element_blank(),panel.grid.major.x =element_blank(),panel.grid.major.y =element_line(colour ="grey92", linewidth = .3),axis.ticks =element_blank(), ...)theme_set(theme_nr())
The question. The project has two definitions of what a technology “needs”, built from different quantities, and both are still in the repo. This document puts them side by side for all 10 technologies and measures the gap.
Bottom line. They disagree by a median of 15.1 percentage points of the need profile and name a different dominant capability for 5 of 10 technologies. Solar is the widest — the legacy axis calls it a Chemicals problem, the winners’ recipe calls it Electronics.
1 What is being compared
Both definitions produce the same object: a five-number profile per technology, one number per capability cluster, normalised to sum to 100. They differ in what they average and over whom.
Legacy SHAP axis
Winners’ recipe
Quantity
z-scored mean absolute SHAP
mean signed SHAP, positive part
Population
all country-years in the panel
only country-years with RCA > 1
Reads as
“how much this code moves the model, anywhere”
“what the countries that actually won have”
Ships in
shap_category shares, PC scatter
capability_gap.qmd’s target_weight
Source
shap_replication_by_category.csv
shap_replication_by_class.csv
The winners’ recipe replaced the legacy axis in July as the definition of need. The replacement was deliberate and documented; what was never measured is how far the two actually sit apart.
Data. Both come from the same from-source replication run (scripts/ml/regen_shap.py), so the comparison is like for like. data/pc/pc_features.csv is not used: it disagrees with that replication at r = 0.76 over what should be the identical quantity, with 45 rows having no counterpart at all. That staleness is a separate finding — see the register.
The winners’ branch above reproduces the shipped target_weight for all 50 technology × cluster pairs to 2.2e-16 — floating-point identity, so this is the shipped target rather than a re-specification of it.
2 The two profiles, technology by technology
Code
lng <- need |>pivot_longer(c(`Legacy SHAP axis`, `Winners' recipe`),names_to ="model", values_to ="pct") |>mutate(model =factor(model, names(MODEL_COL)),cshort =factor(SHORT[as.character(category)], unname(SHORT[CATS])))ggplot(lng, aes(cshort, pct, fill = model)) +geom_col(position =position_dodge(width = .74), width = .62) +scale_fill_manual(values = MODEL_COL, name =NULL) +scale_y_continuous(expand =expansion(mult =c(0, .08)), labels = \(x) paste0(x, "%")) +facet_wrap(~tech, ncol =5) +labs(title ="Where the two definitions of need disagree",subtitle ="Share of each technology's need profile, by capability cluster · bars within one colour sum to 100%",x =NULL, y =NULL) +theme(legend.position ="top",legend.justification ="left",legend.margin =margin(b =4),strip.text =element_text(face ="bold", colour = NZ_DARK, size =10),axis.text.x =element_text(size =8, colour ="#475569"),axis.text.y =element_text(size =8, colour ="#64748b"),panel.spacing.x =unit(10, "pt"), panel.spacing.y =unit(14, "pt"))
Each technology’s need profile under both definitions. Bars are the share of that technology’s need attributed to each capability cluster; the five bars of one colour sum to 100 within each panel.
Read the panels for shape, not level. Three patterns show up.
Machinery loses the top slot; Chemicals and Electronics take it. Which capability each definition calls the biggest need:
Code
kable(topcount |>transmute(Capability = top,`Top under legacy`= leg, `Top under winners' recipe`= win) |>arrange(desc(`Top under legacy`)),caption ="Number of technologies for which each capability is the largest need.")
Number of technologies for which each capability is the largest need.
Capability
Top under legacy
Top under winners’ recipe
Machinery
7
4
Metals
2
1
Chemicals
1
3
Electronics
0
2
Machinery falls from 7 of 10 technologies to 4. Machinery holds the most HS codes of any cluster, and the legacy axis — a sum of z-scores — rewards a cluster for holding many codes.
Note that the movement does not average out to a systematic per-capability tilt: across technologies the gains and losses on any one capability roughly cancel (mean shifts are all under 2 pp). The disagreement is large within each technology and idiosyncratic across them, which is why it has to be read technology by technology and summarised with a distance rather than a mean.
Two technologies barely move at all. Wind and Electrolyzers agree on shape and on top axis under both definitions — there, the choice is immaterial.
3 One number per technology: TVD
To summarise a whole panel in a single figure, use total variation distance — half the sum of absolute differences across the five axes:
Because both profiles sum to 100, TVD reads directly as “how many percentage points of the need profile would have to be moved to turn one into the other”. 0 means the two definitions agree exactly; 100 would mean they share no capability at all. It is the summary of exactly what the bars above show.
Code
tvd <- need |>group_by(tech) |>summarise(TVD =sum(abs(`Legacy SHAP axis`-`Winners' recipe`))/2,`Largest single-axis shift`=max(abs(`Legacy SHAP axis`-`Winners' recipe`)),`Top axis — legacy`=as.character(category[which.max(`Legacy SHAP axis`)]),`Top axis — winners`=as.character(category[which.max(`Winners' recipe`)]),.groups ="drop") |>mutate(Flip =ifelse(`Top axis — legacy`!=`Top axis — winners`, "yes", "—")) |>arrange(desc(TVD))kable(tvd, digits =1, col.names =c("Technology","TVD (pp)","Largest single-axis shift (pp)","Top axis — legacy","Top axis — winners","Top axis flips"),caption ="How far apart the two definitions of need sit, per technology.")
How far apart the two definitions of need sit, per technology.
The same TVD figures ranked. Filled markers are technologies where the two definitions also name a different dominant capability.
Median 15.1 pp, range 6.3–38.9 pp, and the dominant capability changes for 5 of 10 technologies. Solar and Transmission are the two technologies where a reader would draw a materially different conclusion about what to build.
4 Does it change the recommendation?
TVD measures the profiles. Whether that matters depends on whether the movement crosses a decision boundary — so here it is run through the seven-cell priority grid, which combines the need share with the country’s own RCA. Only the need axis changes between the two runs; the RCA is identical.
Code
classify <-function(sp, r) case_when( sp >=25& r >=1~"Leverage", sp >=25& r >= .5~"Build-up", sp >=25~"Critical gap", sp >=10& r >=1~"Mature", sp >=10& r >= .5~"Build-up", sp >=10~"Gap", r >=1~"Bonus", TRUE~"Not priority")rcaP <-read_parquet(here::here("data/pc/pc_rca.parquet")); YRP <-max(rcaP$year, na.rm =TRUE)reg <-read_csv(here::here("data/case_registry.csv"), show_col_types =FALSE) |>filter(kind =="country")cells <- rcaP |>filter(category %in% CATS, tech %in% TECHS, year == YRP) |>select(iso3, country, tech, category, rca) |>inner_join(need |>mutate(tech =as.character(tech), category =as.character(category)),by =c("tech","category")) |>mutate(cell_leg =classify(`Legacy SHAP axis`, rca),cell_win =classify(`Winners' recipe`, rca),moved = cell_leg != cell_win)cellsR <- cells |>filter(iso3 %in% reg$iso3)kable(cellsR |>group_by(Technology = tech) |>summarise(Cells =n(), Changed =sum(moved), .groups ="drop") |>mutate(`% changed`= Changed/Cells*100) |>arrange(desc(`% changed`)),digits =1,caption =sprintf("Country × capability cells that change priority classification when the need definition is swapped — %d CVCE-roster countries, RCA year %d.",n_distinct(cellsR$iso3), YRP))
Country × capability cells that change priority classification when the need definition is swapped — 27 CVCE-roster countries, RCA year 2024.
Technology
Cells
Changed
% changed
Solar
135
96
71.1
Magnets
135
81
60.0
Heat Pumps
135
51
37.8
Transmission
135
46
34.1
Geothermal
135
27
20.0
Nuclear
108
21
19.4
Wind
135
18
13.3
Batteries
135
0
0.0
Biofuel
108
0
0.0
Electrolyzers
108
0
0.0
26.8% of roster cells change their priority classification on identical trade data. But the effect is a threshold effect, not a smooth one, and that is the important part: Solar changes 71% of its cells and Magnets 60%, while Batteries, Biofuel and Electrolyzers change none at all despite moving 12.3, 8.4 and 6.7 pp. Their movement never crosses the grid’s 10% or 25% cut points.
So TVD does not predict impact. It measures how far the definitions sit apart; whether that distance costs anything depends on where the cut points fall. Both numbers are needed.
Code
kable(cellsR |>filter(moved) |>count(`Under legacy`= cell_leg, `Under winners' recipe`= cell_win,sort =TRUE, name ="Cells") |>head(6),caption ="The most common reclassifications.")
The most common reclassifications.
Under legacy
Under winners’ recipe
Cells
Gap
Not priority
81
Critical gap
Gap
49
Mature
Bonus
48
Gap
Critical gap
44
Build-up
Not priority
37
Leverage
Mature
37
The clearest single case is Geothermal · Chemicals — 14.0% of the need under the legacy axis, 6.4% under the winners’ recipe — which turns “Build-up” into “Not priority” for China, India, Mexico, Malaysia, Singapore and Vietnam at once.
5 Sensitivity: how much of either is an artifact?
The comparison above is definition vs definition. A separate question is how robust either definition is to an arbitrary arithmetic choice inside it — the step that collapses a cluster’s features into one number, currently a sum, which could equally be a mean. Swapping it is a sensitivity test, not an alternative model.
Code
bcz <-read_csv(here::here("analysis/ml/shap_replication_by_category.csv"), show_col_types =FALSE) |>filter(tech %in% TECHS) |>mutate(across(c(n_features, pct_shipped_sum_z, pct_mean_z), \(x) coalesce(x, 0)))sens_leg <- bcz |>group_by(tech) |>summarise(t =sum(abs(pct_shipped_sum_z - pct_mean_z))/2,f = category[which.max(pct_shipped_sum_z)] != category[which.max(pct_mean_z)], .groups="drop")wsum <-read_csv(here::here("analysis/ml/shap_replication_by_class.csv"), show_col_types =FALSE) |>filter(retained, category %in% CATS, tech %in% TECHS) |>group_by(tech, category) |>summarise(s =sum(pmax(mean_shap_comp,0)), m =mean(pmax(mean_shap_comp,0)), n =n(), .groups="drop") |>complete(tech, category = CATS, fill =list(s=0,m=0,n=0)) |>group_by(tech) |>mutate(ps = s/sum(s)*100, pm = m/sum(m)*100) |>ungroup()sens_win <- wsum |>group_by(tech) |>summarise(t =sum(abs(ps - pm))/2,f = category[which.max(ps)] != category[which.max(pm)], .groups="drop")kable(tibble(`Need definition`=c("Legacy SHAP axis", "Winners' recipe"),`Median TVD, sum vs mean`=c(median(sens_leg$t), median(sens_win$t)),`Top axis flips`=sprintf("%d of 10", c(sum(sens_leg$f), sum(sens_win$f))),`Correlation with cluster headcount`=sprintf("%.3f", c(median(sapply(split(bcz, bcz$tech), \(x) if (sd(x$n_features)>0) cor(x$n_features, x$pct_shipped_sum_z) elseNA_real_), na.rm=TRUE),median(sapply(split(wsum, wsum$tech), \(x) if (sd(x$n)>0) cor(x$n, x$ps) elseNA_real_), na.rm=TRUE)))),digits =1,caption ="Sensitivity of each need definition to the sum-vs-mean choice inside it.")
Sensitivity of each need definition to the sum-vs-mean choice inside it.
Need definition
Median TVD, sum vs mean
Top axis flips
Correlation with cluster headcount
Legacy SHAP axis
27.2
7 of 10
0.999
Winners’ recipe
22.8
4 of 10
0.945
Both definitions are more sensitive to the arithmetic than they are different from each other — median 27.2 pp and 22.8 pp against the 15.1 pp that separates them. The mechanism is cluster size: within every technology the legacy axis tracks HS-code headcount at ρ = 0.999, because z-scoring flattens every feature to roughly the same magnitude so the sum recovers little else. The winners’ recipe sums genuinely heterogeneous signed magnitudes and falls to ρ = 0.945 — a real improvement, and still mostly a headcount.
shap_zscore_aggregation_dispute.qmd covers why the sum has no theoretical cover; the short version is that the theorem licensing it needs raw values and independent features, and neither holds here.
6 What to do
Name one definition as canonical. Both are live, and nothing at the point of use records which surface reads which. This is the cheap fix and it should happen first.
The winners’ recipe is the better of the two — signed direction rather than magnitude, read off countries that actually won — and it should be it.
Do not quote a need share as a precise number. On either definition it is substantially a headcount, and it moves more under an arithmetic choice than between the two models.
Re-check anything downstream of a threshold. The 10% / 25% cut points are where a modest profile shift becomes a changed recommendation, in 27% of roster cells.
Regenerate or retire data/pc/pc_features.csv — it disagrees with the from-source replication at r = 0.76 and is what the shipped atlases and PC scatter read.
7 Register
Finding
Status
Two need definitions coexist, differing by a median 15.1 pp and flipping the top capability for 5 of 10 techs; neither is marked canonical
new — decide
Swapping the definition changes 27% of roster priority cells; the effect is threshold-driven, so TVD does not predict it
new
data/pc/pc_features.csv diverges from analysis/ml/shap_replication_features.csv (r = 0.76, 45 unmatched rows)
Every figure recomputes at render time from analysis/ml/shap_replication_by_class.csv, shap_replication_by_category.csv, shap_gap_by_country_cluster.csv and data/pc/pc_rca.parquet — all products of the same scripts/ml/regen_shap.py and target_profile_prep.py run. Retention is the pipeline’s own rule (mean_abs_z ≥ 0.5); the winners’ branch is verified against the shipped target_weight in §2. data/pc/pc_features.csv is not read.
Source Code
---title: "Two Definitions of Need"subtitle: "The legacy SHAP axis against the winners'-recipe target, across 10 technologies × 5 capability clusters"author: "Net Zero Industrial Policy Lab (NZIPL) · Johns Hopkins SAIS"date: todayformat: html: toc: true toc-depth: 2 toc-location: left toc-title: "Contents" number-sections: true self-contained: true page-layout: full fig-width: 11 fig-height: 6 code-fold: true code-tools: true theme: [cosmo, ../nzipl.scss] grid: sidebar-width: 250px body-width: 1000px docx: toc: true number-sections: trueexecute-dir: projectexecute: echo: true warning: false message: false---```{r setup}#| code-fold: truelibrary(readr); library(dplyr); library(tidyr); library(tibble)library(ggplot2); library(knitr); library(showtext); library(forcats); library(arrow)font_add_google("Archivo", "Archivo"); showtext_auto(); showtext_opts(dpi =200)NZ_DARK <-"#073309"CATS <-c("Chemicals","Electronics","Industrial Materials","Machinery","Metals")SHORT <-c(Chemicals="Chem", Electronics="Elec", "Industrial Materials"="Ind.M",Machinery="Mach", Metals="Met")TECHS <-c("Solar","Wind","Batteries","Nuclear","Geothermal","Heat Pumps","Electrolyzers","Transmission","Magnets","Biofuel")# Two-series categorical palette. Validated with the dataviz six-checks script:# all pass (CVD ΔE 28.7 deutan, normal-vision 33.9, no contrast warning). Indigo is# used for the legacy axis rather than a blue, to avoid reading across to the# Chemicals category colour (#2563eb) that appears on the x axis.MODEL_COL <-c(`Legacy SHAP axis`="#6366f1", `Winners' recipe`="#16a34a")theme_nr <-function(...) theme_minimal(base_family ="Archivo", base_size =12) +theme(plot.title =element_text(size =14, face ="bold", colour = NZ_DARK),plot.subtitle =element_text(size =10.5, colour ="#475569"),plot.title.position ="plot",panel.grid.minor =element_blank(),panel.grid.major.x =element_blank(),panel.grid.major.y =element_line(colour ="grey92", linewidth = .3),axis.ticks =element_blank(), ...)theme_set(theme_nr())```::: {.callout-important appearance="minimal"}**The question.** The project has **two** definitions of what a technology "needs", built fromdifferent quantities, and both are still in the repo. This document puts them side by side forall 10 technologies and measures the gap.**Bottom line.** They disagree by a median of **15.1 percentage points** of the need profile andname a **different dominant capability for 5 of 10 technologies**. Solar is the widest — thelegacy axis calls it a Chemicals problem, the winners' recipe calls it Electronics.:::# What is being comparedBoth definitions produce the same object: a five-number profile per technology, one number percapability cluster, normalised to sum to 100. They differ in *what they average and over whom*.|| **Legacy SHAP axis** | **Winners' recipe** ||---|---|---|| Quantity | z-scored mean **absolute** SHAP | mean **signed** SHAP, positive part || Population | **all** country-years in the panel | only country-years with **RCA > 1** || Reads as | "how much this code moves the model, anywhere" | "what the countries that actually won have" || Ships in |`shap_category` shares, PC scatter |`capability_gap.qmd`'s `target_weight`|| Source |`shap_replication_by_category.csv`|`shap_replication_by_class.csv`|The winners' recipe replaced the legacy axis in July as the definition of need. The replacementwas deliberate and documented; what was never measured is **how far the two actually sit apart**.::: {.callout-note appearance="minimal"}**Data.** Both come from the same from-source replication run (`scripts/ml/regen_shap.py`),so the comparison is like for like. `data/pc/pc_features.csv` is **not** used: it disagrees withthat replication at r = 0.76 over what should be the identical quantity, with 45 rows having nocounterpart at all. That staleness is a separate finding — see the register.:::```{r build}#| code-fold: true# Winners' recipe — reproduces the shipped `target_weight` exactly (checked below)win <-read_csv(here::here("analysis/ml/shap_replication_by_class.csv"), show_col_types =FALSE) |>filter(retained, category %in% CATS, tech %in% TECHS) |>group_by(tech, category) |>summarise(w =sum(pmax(mean_shap_comp, 0)), .groups ="drop") |>complete(tech, category = CATS, fill =list(w =0)) |>group_by(tech) |>mutate(`Winners' recipe`= w/sum(w)*100) |>ungroup()# Legacy z axis — as shippedleg <-read_csv(here::here("analysis/ml/shap_replication_by_category.csv"), show_col_types =FALSE) |>filter(tech %in% TECHS) |>select(tech, category, `Legacy SHAP axis`= pct_shipped_sum_z)need <- win |>select(tech, category, `Winners' recipe`) |>left_join(leg, by =c("tech","category")) |>mutate(`Legacy SHAP axis`=coalesce(`Legacy SHAP axis`, 0),tech =factor(tech, TECHS), category =factor(category, CATS))# reconstruction check: does the winners' branch equal the shipped target_weight?recon <-read_csv(here::here("analysis/ml/shap_gap_by_country_cluster.csv"), show_col_types =FALSE) |>distinct(tech, cluster, target_weight) |>inner_join(win |>transmute(tech, cluster =as.character(category), w), by =c("tech","cluster"))```The winners' branch above reproduces the shipped `target_weight` for all `r nrow(recon)`technology × cluster pairs to **`r sprintf("%.1e", max(abs(recon$target_weight - recon$w)))`** —floating-point identity, so this is the shipped target rather than a re-specification of it.# The two profiles, technology by technology```{r barplot, fig.height=6.4}#| code-fold: true#| fig-cap: "Each technology's need profile under both definitions. Bars are the share of that technology's need attributed to each capability cluster; the five bars of one colour sum to 100 within each panel."lng <- need |>pivot_longer(c(`Legacy SHAP axis`, `Winners' recipe`),names_to ="model", values_to ="pct") |>mutate(model =factor(model, names(MODEL_COL)),cshort =factor(SHORT[as.character(category)], unname(SHORT[CATS])))ggplot(lng, aes(cshort, pct, fill = model)) +geom_col(position =position_dodge(width = .74), width = .62) +scale_fill_manual(values = MODEL_COL, name =NULL) +scale_y_continuous(expand =expansion(mult =c(0, .08)), labels = \(x) paste0(x, "%")) +facet_wrap(~tech, ncol =5) +labs(title ="Where the two definitions of need disagree",subtitle ="Share of each technology's need profile, by capability cluster · bars within one colour sum to 100%",x =NULL, y =NULL) +theme(legend.position ="top",legend.justification ="left",legend.margin =margin(b =4),strip.text =element_text(face ="bold", colour = NZ_DARK, size =10),axis.text.x =element_text(size =8, colour ="#475569"),axis.text.y =element_text(size =8, colour ="#64748b"),panel.spacing.x =unit(10, "pt"), panel.spacing.y =unit(14, "pt"))``````{r shape-facts}#| code-fold: truetops <- need |>group_by(tech) |>summarise(leg =as.character(category[which.max(`Legacy SHAP axis`)]),win =as.character(category[which.max(`Winners' recipe`)]), .groups ="drop")topcount <- tops |>pivot_longer(c(leg, win), names_to ="model", values_to ="top") |>count(model, top) |>pivot_wider(names_from = model, values_from = n, values_fill =0)```Read the panels for **shape**, not level. Three patterns show up.**Machinery loses the top slot; Chemicals and Electronics take it.** Which capability eachdefinition calls the biggest need:```{r topslot}#| code-fold: truekable(topcount |>transmute(Capability = top,`Top under legacy`= leg, `Top under winners' recipe`= win) |>arrange(desc(`Top under legacy`)),caption ="Number of technologies for which each capability is the largest need.")```Machinery falls from **`r sum(tops$leg == "Machinery")` of 10** technologies to**`r sum(tops$win == "Machinery")`**. Machinery holds the most HS codes of any cluster, and thelegacy axis — a sum of z-scores — rewards a cluster for holding many codes.Note that the movement does **not** average out to a systematic per-capability tilt: acrosstechnologies the gains and losses on any one capability roughly cancel (mean shifts are all under2 pp). The disagreement is large *within* each technology and idiosyncratic *across* them, whichis why it has to be read technology by technology and summarised with a distance rather than amean.**Two technologies barely move at all.** Wind and Electrolyzers agree on shape and on top axisunder both definitions — there, the choice is immaterial.# One number per technology: TVDTo summarise a whole panel in a single figure, use **total variation distance** — half the sum ofabsolute differences across the five axes:$$\text{TVD} \;=\; \tfrac{1}{2}\sum_{c \in \text{clusters}} \bigl| \text{legacy}_c - \text{winners}_c \bigr|$$Because both profiles sum to 100, TVD reads directly as **"how many percentage points of the needprofile would have to be moved to turn one into the other"**. 0 means the two definitions agreeexactly; 100 would mean they share no capability at all. It is the summary of exactly what thebars above show.```{r tvd}#| code-fold: truetvd <- need |>group_by(tech) |>summarise(TVD =sum(abs(`Legacy SHAP axis`-`Winners' recipe`))/2,`Largest single-axis shift`=max(abs(`Legacy SHAP axis`-`Winners' recipe`)),`Top axis — legacy`=as.character(category[which.max(`Legacy SHAP axis`)]),`Top axis — winners`=as.character(category[which.max(`Winners' recipe`)]),.groups ="drop") |>mutate(Flip =ifelse(`Top axis — legacy`!=`Top axis — winners`, "yes", "—")) |>arrange(desc(TVD))kable(tvd, digits =1, col.names =c("Technology","TVD (pp)","Largest single-axis shift (pp)","Top axis — legacy","Top axis — winners","Top axis flips"),caption ="How far apart the two definitions of need sit, per technology.")``````{r tvd-fig, fig.height=3.6}#| code-fold: true#| fig-cap: "The same TVD figures ranked. Filled markers are technologies where the two definitions also name a different dominant capability."ggplot(tvd |>mutate(tech =fct_reorder(tech, TVD)), aes(TVD, tech)) +geom_segment(aes(x =0, xend = TVD, yend = tech), colour ="grey88", linewidth =2) +geom_point(aes(fill = Flip =="yes"), shape =21, size =4, stroke =1,colour ="white") +scale_fill_manual(values =c(`TRUE`="#6366f1", `FALSE`="#cbd5e1"),labels =c(`TRUE`="top capability changes", `FALSE`="top capability holds"),name =NULL) +geom_text(aes(label =sprintf("%.1f", TVD)), hjust =-0.75,family ="Archivo", size =3.1, colour ="#475569") +scale_x_continuous(expand =expansion(mult =c(0, .14)), labels = \(x) paste0(x, "pp")) +labs(title ="Median disagreement: 15.1 percentage points of the need profile",subtitle ="Total variation distance between the legacy SHAP axis and the winners'-recipe target",x =NULL, y =NULL) +theme(legend.position ="top", legend.justification ="left",panel.grid.major.y =element_blank(),panel.grid.major.x =element_line(colour ="grey92", linewidth = .3))```Median **`r sprintf("%.1f", median(tvd$TVD))` pp**, range`r sprintf("%.1f", min(tvd$TVD))`–`r sprintf("%.1f", max(tvd$TVD))` pp, and the dominant capabilitychanges for **`r sum(tvd$Flip == "yes")` of 10** technologies. Solar and Transmission are the twotechnologies where a reader would draw a materially different conclusion about what to build.# Does it change the recommendation?TVD measures the profiles. Whether that *matters* depends on whether the movement crosses adecision boundary — so here it is run through the seven-cell priority grid, which combines theneed share with the country's own RCA. Only the need axis changes between the two runs; the RCA isidentical.```{r cells}#| code-fold: trueclassify <-function(sp, r) case_when( sp >=25& r >=1~"Leverage", sp >=25& r >= .5~"Build-up", sp >=25~"Critical gap", sp >=10& r >=1~"Mature", sp >=10& r >= .5~"Build-up", sp >=10~"Gap", r >=1~"Bonus", TRUE~"Not priority")rcaP <-read_parquet(here::here("data/pc/pc_rca.parquet")); YRP <-max(rcaP$year, na.rm =TRUE)reg <-read_csv(here::here("data/case_registry.csv"), show_col_types =FALSE) |>filter(kind =="country")cells <- rcaP |>filter(category %in% CATS, tech %in% TECHS, year == YRP) |>select(iso3, country, tech, category, rca) |>inner_join(need |>mutate(tech =as.character(tech), category =as.character(category)),by =c("tech","category")) |>mutate(cell_leg =classify(`Legacy SHAP axis`, rca),cell_win =classify(`Winners' recipe`, rca),moved = cell_leg != cell_win)cellsR <- cells |>filter(iso3 %in% reg$iso3)kable(cellsR |>group_by(Technology = tech) |>summarise(Cells =n(), Changed =sum(moved), .groups ="drop") |>mutate(`% changed`= Changed/Cells*100) |>arrange(desc(`% changed`)),digits =1,caption =sprintf("Country × capability cells that change priority classification when the need definition is swapped — %d CVCE-roster countries, RCA year %d.",n_distinct(cellsR$iso3), YRP))```**`r sprintf("%.1f%%", mean(cellsR$moved)*100)` of roster cells change their priorityclassification** on identical trade data. But the effect is a **threshold** effect, not a smoothone, and that is the important part: Solar changes 71% of its cells and Magnets 60%, while**Batteries, Biofuel and Electrolyzers change none at all** despite moving 12.3, 8.4 and 6.7 pp.Their movement never crosses the grid's 10% or 25% cut points.**So TVD does not predict impact.** It measures how far the definitions sit apart; whether thatdistance costs anything depends on where the cut points fall. Both numbers are needed.```{r flips}#| code-fold: truekable(cellsR |>filter(moved) |>count(`Under legacy`= cell_leg, `Under winners' recipe`= cell_win,sort =TRUE, name ="Cells") |>head(6),caption ="The most common reclassifications.")```The clearest single case is **Geothermal · Chemicals** — 14.0% of the need under the legacy axis,6.4% under the winners' recipe — which turns "Build-up" into "Not priority" for China, India,Mexico, Malaysia, Singapore and Vietnam at once.# Sensitivity: how much of either is an artifact?The comparison above is *definition vs definition*. A separate question is how robust eitherdefinition is to an arbitrary arithmetic choice inside it — the step that collapses a cluster'sfeatures into one number, currently a **sum**, which could equally be a **mean**. Swapping it is asensitivity test, not an alternative model.```{r sens}#| code-fold: truebcz <-read_csv(here::here("analysis/ml/shap_replication_by_category.csv"), show_col_types =FALSE) |>filter(tech %in% TECHS) |>mutate(across(c(n_features, pct_shipped_sum_z, pct_mean_z), \(x) coalesce(x, 0)))sens_leg <- bcz |>group_by(tech) |>summarise(t =sum(abs(pct_shipped_sum_z - pct_mean_z))/2,f = category[which.max(pct_shipped_sum_z)] != category[which.max(pct_mean_z)], .groups="drop")wsum <-read_csv(here::here("analysis/ml/shap_replication_by_class.csv"), show_col_types =FALSE) |>filter(retained, category %in% CATS, tech %in% TECHS) |>group_by(tech, category) |>summarise(s =sum(pmax(mean_shap_comp,0)), m =mean(pmax(mean_shap_comp,0)), n =n(), .groups="drop") |>complete(tech, category = CATS, fill =list(s=0,m=0,n=0)) |>group_by(tech) |>mutate(ps = s/sum(s)*100, pm = m/sum(m)*100) |>ungroup()sens_win <- wsum |>group_by(tech) |>summarise(t =sum(abs(ps - pm))/2,f = category[which.max(ps)] != category[which.max(pm)], .groups="drop")kable(tibble(`Need definition`=c("Legacy SHAP axis", "Winners' recipe"),`Median TVD, sum vs mean`=c(median(sens_leg$t), median(sens_win$t)),`Top axis flips`=sprintf("%d of 10", c(sum(sens_leg$f), sum(sens_win$f))),`Correlation with cluster headcount`=sprintf("%.3f", c(median(sapply(split(bcz, bcz$tech), \(x) if (sd(x$n_features)>0) cor(x$n_features, x$pct_shipped_sum_z) elseNA_real_), na.rm=TRUE),median(sapply(split(wsum, wsum$tech), \(x) if (sd(x$n)>0) cor(x$n, x$ps) elseNA_real_), na.rm=TRUE)))),digits =1,caption ="Sensitivity of each need definition to the sum-vs-mean choice inside it.")```Both definitions are **more sensitive to the arithmetic than they are different from each other**— median 27.2 pp and 22.8 pp against the 15.1 pp that separates them. The mechanism is clustersize: within every technology the legacy axis tracks HS-code headcount at ρ = 0.999, becausez-scoring flattens every feature to roughly the same magnitude so the sum recovers little else.The winners' recipe sums genuinely heterogeneous signed magnitudes and falls to ρ = 0.945 — a realimprovement, and still mostly a headcount.`shap_zscore_aggregation_dispute.qmd` covers why the sum has no theoretical cover; the shortversion is that the theorem licensing it needs raw values and independent features, and neitherholds here.# What to do1. **Name one definition as canonical.** Both are live, and nothing at the point of use records which surface reads which. This is the cheap fix and it should happen first.2. **The winners' recipe is the better of the two** — signed direction rather than magnitude, read off countries that actually won — and it should be it.3. **Do not quote a need share as a precise number.** On either definition it is substantially a headcount, and it moves more under an arithmetic choice than between the two models.4. **Re-check anything downstream of a threshold.** The 10% / 25% cut points are where a modest profile shift becomes a changed recommendation, in 27% of roster cells.5. **Regenerate or retire `data/pc/pc_features.csv`** — it disagrees with the from-source replication at r = 0.76 and is what the shipped atlases and PC scatter read.# Register| Finding | Status ||---|---|| Two need definitions coexist, differing by a median `r sprintf("%.1f", median(tvd$TVD))` pp and flipping the top capability for `r sum(tvd$Flip == "yes")` of 10 techs; neither is marked canonical | **new** — decide || Swapping the definition changes `r sprintf("%.0f%%", mean(cellsR$moved)*100)` of roster priority cells; the effect is threshold-driven, so TVD does not predict it | **new** ||`data/pc/pc_features.csv` diverges from `analysis/ml/shap_replication_features.csv` (r = 0.76, 45 unmatched rows) | **new** — raise before further PC work || Both definitions remain substantially cluster-headcount driven (ρ = 0.999 legacy, 0.945 winners) | quantifies `shap_zscore_aggregation_dispute.qmd`|# ReproducibilityEvery figure recomputes at render time from `analysis/ml/shap_replication_by_class.csv`,`shap_replication_by_category.csv`, `shap_gap_by_country_cluster.csv` and `data/pc/pc_rca.parquet`— all products of the same `scripts/ml/regen_shap.py` and `target_profile_prep.py` run. Retentionis the pipeline's own rule (`mean_abs_z ≥ 0.5`); the winners' branch is verified against theshipped `target_weight` in §2. `data/pc/pc_features.csv` is not read.
