Is a single ‘standard capability radar’ countries evolve toward defensible — and where do negative (avoid) weights belong? A follow-on to the SHAP replication.
Author
Net Zero Industrial Policy Lab (NZIPL) · Johns Hopkins SAIS
Where this sits. The companion ml_replication.qmd reproduced the shipped SHAP need-axis from source and showed it discards three things — magnitude, sign, and the country dimension. That note was diagnostic. This one is constructive: the co-directors want a single standard capability radar per technology — the profile a country’s industrial policy steers toward. That is a legitimate object, but only if (a) a common profile actually exists across countries, and (b) we know where “don’t specialise here” (negative) weights belong. Both questions are answered here from the same regenerated SHAP (analysis/ml/, produced by scripts/ml/regen_shap.py + scripts/ml/target_profile_prep.py), with the RCA>1 “competitive” class reconstructed per country-year from the CICE trade files — no model re-fit.
1 Two layers: the target and the gap
The audit and the co-directors are not actually in conflict — they are describing two different objects that the shipped pipeline collapses into one:
Target profile
Country gap
Question
What does technology X require?
How far is country C from it, and in which direction?
Varies by country?
No — one profile per tech
Yes — one per country
Needs sign?
Only to say what’s on- vs off-path
Yes — surplus vs deficit, build vs avoid
Natural unit
cluster shares of a canonical fingerprint
that country’s own SHAP row vs the target
ml_replication faulted the shipped radar for being country-invariant and unsigned. But country-invariance is a feature of the target layer, not a bug — a “standard profile countries evolve toward” is by definition the same for everyone. The critique bites on the gap layer, where sign and the country dimension genuinely live. So the constructive question splits cleanly:
Is the target layer well-defined? Do countries agree on what the technology needs strongly enough that a single profile is meaningful? (§2–§3)
How does sign enter? Where is “don’t specialise here”, and does the shipped |SHAP| throw away something the target radar needs? (§4)
What does the corrected target radar look like, and how is it built? (§5–§6)
2 Re-reading EDA 4 — what the dispersion numbers actually said
ml_replication’s EDA 4 reported four alarming numbers about how the pooled mean|SHAP| hides country-year variation. They are easy to over-read, so here is one feature, concretely. Take Solar’s HS 830249 (mounting equipment) — a retained radar feature — across all 3410 country-years:
Code
w <- ex |>filter(tech=="Solar", feature=="s_830249")mw <-mean(w$abs_shap)stats <-tibble(Statistic =c("CV (spread ÷ mean)", "max ÷ mean", "% of country-years below ½ the mean","top-2% country-years' share of total"),Value =c(sprintf("%.2f", sd(w$abs_shap)/mw),sprintf("%.0f×", max(w$abs_shap)/mw),sprintf("%.0f%%", mean(w$abs_shap < mw/2)*100),sprintf("%.0f%%", sum(sort(w$abs_shap, decreasing=TRUE)[1:ceiling(.02*nrow(w))])/sum(w$abs_shap)*100)),`Plain meaning`=c("the importance varies more across countries than its own average","for one country-year it matters ~17× more than the reported mean","for a third of countries it is nearly inert","a handful of countries carry most of the total"))ggplot(w, aes(abs_shap +1e-6)) +geom_histogram(bins =46, fill = NZ_GREEN, colour ="white", linewidth = .15) +geom_vline(xintercept = mw, colour = NZ_DARK, linewidth = .7) +annotate("text", x = mw*1.15, y =Inf, vjust =1.6, hjust =0, family ="Archivo", size =3,colour = NZ_DARK, label ="the mean\nthe axis keeps") +scale_x_log10() +labs(x ="|SHAP| for HS 830249 in one country-year (log scale)", y ="country-years",title ="One retained feature, across every country-year",subtitle =paste0("Solar · mounting equipment. Importance is concentrated (peak in Cambodia 2021-24); ","the single mean sits low in a long right tail."))
Code
kable(stats, caption ="The four EDA-4 statistics for one feature, in plain language. Every one of them is a statement about how MAGNITUDE varies across countries.")
The four EDA-4 statistics for one feature, in plain language. Every one of them is a statement about how MAGNITUDE varies across countries.
Statistic
Value
Plain meaning
CV (spread ÷ mean)
1.22
the importance varies more across countries than its own average
max ÷ mean
17×
for one country-year it matters ~17× more than the reported mean
% of country-years below ½ the mean
32%
for a third of countries it is nearly inert
top-2% country-years’ share of total
14%
a handful of countries carry most of the total
Reading the histogram. The horizontal axis (log scale) is this feature’s |SHAP| in a single country-year; each bar counts how many country-years fall in that importance band; the vertical line is the one average the pipeline keeps. The mean sits low on a long right tail — dragged down by the many countries for which mounting-equipment capability barely moves the prediction, and pulled up only by the handful (here, Cambodia 2021–24) for which it is decisive. So the single reported number describes almost no individual country well. The table restates that in four ways.
The point that was missing. Every one of these numbers measures how the magnitude of a feature’s importance varies across countries. That matters enormously for a country gap (“how much does mounting-equipment capability matter for Cambodia specifically”) — but it does not, on its own, sink a target profile. A standard profile does not require the magnitude to be equal across countries; it only requires the shape — which capabilities rank high relative to each other — to be shared. EDA 4 measured scale dispersion and left the shape question open. §3 asks it directly.
3 Does a common profile exist? Shape, not scale
For each technology, take each country’s own importance vector (its mean |SHAP| per feature over its years) and rank-correlate it against the pooled profile. If countries agree on the ranking of capabilities — even while differing wildly in magnitude — then a single target profile is a faithful summary of a shared shape.
Code
cr <- crank |>filter(!is.na(spearman_retained)) |>mutate(tech =factor(tech, TECHS))med <- cr |>group_by(tech) |>summarise(m =median(spearman_retained), .groups="drop")p1 <-ggplot(cr, aes(x =fct_rev(tech), y = spearman_retained)) +geom_hline(yintercept =c(0, .5, 1), linetype ="22", colour ="grey85") +geom_violin(fill = NZ_GREEN, alpha = .25, colour =NA, scale ="width") +geom_boxplot(width = .18, outlier.size = .5, fill ="white", colour = NZ_DARK, linewidth = .4) +coord_flip(ylim =c(0, 1)) +labs(x =NULL, y ="each country's rank-agreement with the pooled profile (Spearman ρ)",title ="Countries agree on the SHAPE of what each tech needs",subtitle =paste0("Per-country importance vectors vs the pooled profile, retained radar features. ","Median ρ = ", round(median(cr$spearman_retained),2), " overall."))p2 <-ggplot(cr, aes(frac_competitive, spearman_retained)) +geom_point(alpha = .25, size =1, colour = NZ_TEAL) +geom_smooth(method ="lm", se =FALSE, colour = NZ_DARK, linewidth = .7) +labs(x ="share of years the country is RCA-competitive", y ="rank-agreement (ρ)",title ="Have-nots share the starting line; winners specialise",subtitle ="Non-competitive countries hug the common profile; competitive ones deviate into niches.")p1 | p2
Reading the two panels. Let us take them one at a time. The left panel has one row per technology; each violin-and-box summarises, across that technology’s many countries, a single score per country — how closely that country ranks the capabilities the same way the pooled profile does (Spearman’s ρ, where 1 means an identical ordering and 0 means no relationship). The boxes cluster high and tight, around 0.8–0.95, so the reassuring message is that countries broadly agree on which capabilities a technology leans on, even though EDA 4 showed they differ enormously in how much. The right panel then asks a gentler follow-up: does that agreement depend on how advanced a country is? Each dot is one country; the horizontal axis is the share of years it was competitive (RCA>1), the vertical axis its rank-agreement. The line slopes down — at first glance surprising — because a country that is short on nearly everything trivially traces the common shape, while a country that has actually become competitive has bent its own profile toward a specialisation. So the shared profile is best pictured as the on-ramp everyone starts on, not a description of where the winners end up.
Code
crank |>filter(!is.na(spearman_retained)) |>mutate(tech =factor(tech, TECHS)) |>group_by(tech) |>summarise(`N countries`=n(), `median ρ`=round(median(spearman_retained),2),`10th pct ρ`=round(quantile(spearman_retained,.1),2),`% above 0.7`=round(mean(spearman_retained>.7)*100,0), .groups="drop") |>arrange(tech) |>kable(caption ="Per-tech rank agreement between each country's importance vector and the pooled profile. Even the 10th-percentile country is well above chance; a common profile is a defensible summary of a shared shape.")
Per-tech rank agreement between each country’s importance vector and the pooled profile. Even the 10th-percentile country is well above chance; a common profile is a defensible summary of a shared shape.
tech
N countries
median ρ
10th pct ρ
% above 0.7
Solar
155
0.85
0.70
90
Wind
155
0.88
0.77
95
Batteries
155
0.89
0.78
98
EVs
158
0.94
0.82
99
Nuclear
155
0.93
0.74
92
Geothermal
155
0.90
0.80
98
Heat Pumps
155
0.93
0.75
94
Electrolyzers
155
0.86
0.72
94
Transmission
155
0.88
0.57
81
Magnets
155
0.90
0.71
90
Biofuel
155
0.79
0.54
69
Read. Median rank-agreement is 0.88 and never below ~0.8 at the tech level — countries overwhelmingly agree on which capabilities a technology leans on, differing mainly in how much (the EDA-4 scale story). So the co-directors’ object is justified: a single target profile is a real, shared shape. The one nuance (right panel): countries that are not yet competitive agree with the profile most (they are uniformly “short” everywhere), while competitive producers deviate — each has specialised into a niche. The common profile is therefore best read as the aspirational path a not-yet-competitive country climbs, with successful producers branching into specialisations once on it.
4 Where negative weights belong
The intuition behind wanting sign is right: a target radar should distinguish “build this” from “don’t specialise here”, and the shipped |SHAP| cannot. But the naive fix — sign the pooled mean SHAP — is wrong, and the replication data shows exactly why.
Code
rr <- bycl |>filter(retained, category %in% CATS)share_neg_pooled <-round(mean(rr$mean_shap_signed <0)*100, 0)share_neg_comp <-round(mean(rr$mean_shap_comp <0)*100, 0)ggplot(rr, aes(mean_shap_signed, mean_shap_comp, colour = category)) +geom_hline(yintercept =0, colour ="grey60") +geom_vline(xintercept =0, colour ="grey60") +geom_point(alpha = .7, size =1.7) +scale_colour_manual(values = CAT_COL, name =NULL) +annotate("rect", xmin =-Inf, xmax =0, ymin =0, ymax =Inf, alpha = .05, fill = NZ_GREEN) +annotate("text", x =min(rr$mean_shap_signed), y =max(rr$mean_shap_comp), hjust =0, vjust =1,family ="Archivo", size =3.1, colour = NZ_DARK,label ="pooled sign says 'negative'\nbut winners' sign says 'build'\n= a GAP, not an anti-target") +labs(x ="pooled mean SHAP (signed) — what a naive sign would use",y ="mean SHAP among RCA-competitive producers — the target direction",title ="Pooled negativity is a base-rate artifact, not 'avoid'",subtitle =paste0(share_neg_pooled, "% of retained features are pooled-negative; ", share_neg_comp, "% are negative among the winners.")) +theme(legend.position ="bottom")
Reading the plot. Each dot is one retained radar feature. Its horizontal position is the pooled mean signed SHAP — the number you would get by naively adding the sign back to the shipped axis, averaging over every country. Its vertical position is the mean SHAP among only the countries that are actually competitive (RCA>1) — the direction that matters if we want to describe a winning profile. The shaded band in the top-left is the crux: features that look negative when pooled over everyone but sit positive among the winners. A feature there is emphatically not something to avoid — competitive producers plainly benefit from it; it merely reads negative overall because most countries have not built it yet. That is the signature of a gap, not an anti-target — and it is why signing the pooled mean would mislead.
Because RCA>1 is rare (~5% base rate), the large non-competitive majority reads as “not enough of everything” — so 48% of retained features carry a net-negative pooled SHAP. That is the deficit of the have-nots, i.e. the gap, not a signal that the capability is bad to hold. Split the sign by the target class and the artifact disappears:
Code
allf <- bycl |>filter(category %in% CATS)tibble(Group =c("All radar features", "Retained radar features"),`N`=c(nrow(allf), nrow(rr)),`% pooled-SHAP negative`=c(round(mean(allf$mean_shap_signed<0)*100,0), round(mean(rr$mean_shap_signed<0)*100,0)),`% negative among WINNERS`=c(round(mean(allf$mean_shap_comp<0)*100,0), round(mean(rr$mean_shap_comp<0)*100,0)),`% where winners < non-winners (disc<0)`=c(round(mean(allf$disc<0)*100,0), round(mean(rr$disc<0)*100,0))) |>kable(caption ="Sign under two readings. Among competitive producers, essentially no retained feature points 'away' — and none is systematically lower among winners than non-winners. The pooled negativity was base rate.")
Sign under two readings. Among competitive producers, essentially no retained feature points ‘away’ — and none is systematically lower among winners than non-winners. The pooled negativity was base rate.
Group
N
% pooled-SHAP negative
% negative among WINNERS
% where winners < non-winners (disc<0)
All radar features
830
51
0
0
Retained radar features
390
48
0
0
Among the 390 retained radar features, 0 are negative among competitive producers, and 0 are lower among winners than non-winners. In other words, for these 11 technologies there is no meaningful “don’t specialise here” among the capabilities that reach the radar — the winning recipes are uniformly additive. Genuine anti-targets do exist, but they are rare and sit below the selection bar:
Code
avoid <- bycl |>filter(category %in% CATS, mean_shap_comp <0) |>arrange(mean_shap_comp) |>transmute(Tech = tech, HS =str_remove(feature, "^n?s_"),Product =hs_name(str_remove(feature, "^n?s_")), Cluster = category,`SHAP among winners`=round(mean_shap_comp, 5),Retained =ifelse(retained, "yes (on radar)", "no (below cut)"))if (nrow(avoid) ==0) {cat("_No feature — retained or not — has a negative SHAP among competitive producers._")} elsekable(avoid, caption ="The only genuine 'avoid' candidates across all 11 techs: features that push competitive producers' predicted competitiveness DOWN. All sit below the selection cut, so none reaches the shipped radar.")
The only genuine ‘avoid’ candidates across all 11 techs: features that push competitive producers’ predicted competitiveness DOWN. All sit below the selection cut, so none reaches the shipped radar.
Tech
HS
Product
Cluster
SHAP among winners
Retained
EVs
nev_850151
Electronics
-0.00021
no (below cut)
EVs
nev_720410
Metals
-0.00002
no (below cut)
Takeaway on sign. Sign belongs in the model, but its job is directional confirmation, not carving negative wedges out of the radar. Building the target from the competitive-class signed SHAP gives an axis that is signed (so it can flag an anti-target if one appears) and is magnitude-preserving — and it happens to be all-positive here. The place negative numbers do real work is the gap layer: a specific country with a surplus in a cluster it should be de-emphasising. That is a country-level reading, not a property of the tech’s profile.
5 The target radar, built the right way
Putting §3 (a common profile exists) and §4 (sign belongs to the winners) together, we can finally state how the corrected radar is built. It is worth going slowly here, because the whole argument rests on this one construction.
What “competitive-class signed SHAP” means. Recall from §4 that for every country-year the model produces a signed SHAP value per feature — positive if that capability pushed the country toward predicted competitiveness, negative if away. The model’s target is binary: is the country RCA-competitive (RCA>1) in the technology, or not? That splits the country-years into a competitive class (the ~5% that are the actual “winners”) and the rest. A feature’s competitive-class signed SHAP is simply the average of its signed value over the winners only — in plain terms, among the countries that actually became competitive, how much did this capability contribute, on net, to getting them there? We average over the winners rather than over everyone on purpose: the pooled mean is dragged negative by the 95% who have not built the capability yet (the base-rate gap of §4), whereas the winners’ average isolates the recipe itself.
Building the axis, in four steps. With that single number in hand for each feature:
Assign every retained feature to its capability cluster via its HS chapter — e.g. Solar’s 854190 (semiconductor/LED components) → Electronics.
Sum the winners’ contributions within each cluster, clipping any negative to zero (a safeguard that, as §4 showed, never actually bites for retained features): \(\text{target}_c = \sum_{j\in c} \max(\overline{\phi_j}^{\text{comp}},\,0)\). This is the magnitude-preserving step — a feature contributing 0.07 adds 0.07, one contributing 0.002 adds 0.002 — unlike the shipped z-score, which first squashes every retained feature onto ~0.5–0.85 so that summing them merely counts them.
Normalise each cluster’s total by the sum across all five clusters, giving shares that add to 100% — the shape the radar draws.
Plot the five shares as a pentagon: five spokes 72° apart, one per cluster, the radius on each being that cluster’s share. Each facet below is self-scaled to its own longest spoke, so we are comparing shapes, not absolute sizes.
The axis is direction-checked because we used the signed SHAP and kept only the positive winners’ contributions: it can only reflect capabilities that push winners toward competitiveness, and a genuine anti-target (negative even among winners) would be clipped out and surfaced separately, not silently inflate a cluster.
A worked read — Solar. The Electronics features’ winners’-SHAP sum is ≈ 0.21 against a five-cluster total of ≈ 0.34, so Electronics makes up ≈ 62% of Solar’s target — carried by a few heavy contributors (854190 alone ≈ 0.071). The shipped sum-of-z axis instead crowns Machinery, which merely holds more product codes. That one contrast — magnitude vs count — is what the radars below make visible for every technology at once.
Code
tgt <- bycl |>filter(retained, category %in% CATS) |>group_by(tech, category) |>summarise(w =sum(pmax(mean_shap_comp, 0)), .groups="drop") |>group_by(tech) |>mutate(share = w/sum(w)) |>ungroup() |>transmute(tech, category, series ="Target (winners' recipe)", share)shp <- catt |>transmute(tech, category, series ="Shipped (sum of z)", share = pct_shipped_sum_z/100)rad <-bind_rows(tgt, shp) |>complete(tech, category = CATS, series, fill =list(share =0))# top cluster under eachflip <- rad |>group_by(tech, series) |>slice_max(share, n =1, with_ties =FALSE) |>select(tech, series, category) |>pivot_wider(names_from = series, values_from = category)
Code
samp <- TECHS # all 11 technologiesang <-tibble(category = CATS, ang = pi/2-2*pi*(0:4)/5)poly <- rad |>filter(tech %in% samp) |>left_join(ang, by="category") |>group_by(tech) |>mutate(rr = share/max(share)) |>ungroup() |>mutate(x = rr*cos(ang), y = rr*sin(ang), o =match(category, CATS), tech =factor(tech, samp))polyc <-bind_rows(poly, poly |>filter(category==CATS[1]) |>mutate(o =6)) |>arrange(tech, series, o)rings <-expand_grid(tech =factor(samp, samp), ring =c(.33,.66,1), category = CATS) |>left_join(ang, by="category") |>mutate(x = ring*cos(ang), y = ring*sin(ang), o =match(category, CATS))ringsc <-bind_rows(rings, rings |>filter(category==CATS[1]) |>mutate(o=6)) |>arrange(tech, ring, o)spokes <-expand_grid(tech =factor(samp, samp), category = CATS) |>left_join(ang, by="category") |>mutate(x =cos(ang), y =sin(ang))axlab <-expand_grid(tech =factor(samp, samp), category = CATS) |>left_join(ang, by="category") |>mutate(x =1.24*cos(ang), y =1.24*sin(ang), lab = CATS_SHORT[category])ggplot() +geom_path(data = ringsc, aes(x, y, group =interaction(tech, ring)), colour ="grey86", linewidth = .3) +geom_segment(data = spokes, aes(0, 0, xend = x, yend = y), colour ="grey86", linewidth = .3) +geom_polygon(data = polyc, aes(x, y, fill = series, colour = series), alpha = .3, linewidth = .8) +geom_text(data = axlab, aes(x, y, label = lab), family ="Archivo", size =2.5, colour ="#334155", lineheight=.85) +facet_wrap(~tech, ncol =4) +scale_fill_manual(values =c(`Shipped (sum of z)`="#94a3b8", `Target (winners' recipe)`=NZ_GREEN), name=NULL) +scale_colour_manual(values =c(`Shipped (sum of z)`="#64748b", `Target (winners' recipe)`=NZ_TEAL), name=NULL) +coord_equal(clip ="off") +xlim(-1.4, 1.4) +ylim(-1.35, 1.35) +labs(title ="Shipped radar vs a winners'-recipe target radar",subtitle ="Each facet self-scaled to its longest axis. Grey = shipped count-driven axis; green = magnitude- and direction-preserving target.") +theme_void(base_family ="Archivo") +theme(legend.position ="bottom", strip.text =element_text(family="Archivo", face="bold", colour=NZ_DARK, size=11),plot.title =element_text(family="Archivo", face="bold", colour=NZ_DARK, size=14),plot.subtitle =element_text(family="Archivo", colour="#475569", size=10.5))
Reading the radars. Each pentagon is one technology; the five spokes are the capability clusters; the two overlaid shapes are the shipped, count-driven axis (grey) and the winners’-recipe target (green). Each is scaled to its own longest spoke, so what we are comparing is the shape of the two profiles, not their absolute size. Wherever the green shape reaches out beyond the grey on a spoke, the corrected target places more weight on that cluster than the shipped axis does; wherever it falls short, less. The silhouettes are not subtle variations of each other — read Solar, for example, where the grey shape bulges toward Machinery while the green pulls firmly toward Electronics. That visible disagreement is the same story EDA 2 told with numbers, now in a single glance.
Code
hd <- rad |>mutate(tech =factor(tech, rev(TECHS)), category =factor(category, CATS),series =factor(series, c("Shipped (sum of z)","Target (winners' recipe)")))ggplot(hd, aes(category, tech, fill = share*100)) +geom_tile(colour ="white", linewidth = .6) +geom_text(aes(label =ifelse(share*100<1, "", sprintf("%.0f", share*100))),family ="Archivo", size =2.7, colour ="#0b1a12") +facet_wrap(~series) +scale_fill_gradient(low ="#f0f7f0", high = NZ_GREEN, na.value ="grey93", name ="% of need") +labs(x =NULL, y =NULL, title ="Cluster need across all 11 techs — shipped vs target",subtitle ="Shipped lights up Machinery (the largest-count cluster); the target redistributes toward where magnitude actually is.") +theme(axis.text.x =element_text(angle =20, hjust =1), legend.position ="bottom")
The heatmap widens the same comparison to all eleven technologies at once. The layout is the one from ml_replication‘s EDA 2 — rows are technologies, the five columns are clusters, each cell is that cluster’s share of the technology’s “need” — but here the right-hand panel is the corrected target rather than raw magnitude. Reading down the left panel, the Machinery column is lit up almost everywhere, the count-driven pattern we already diagnosed; reading the right panel, the brightness disperses toward wherever the competitive producers’ SHAP genuinely concentrates. Seeing all eleven together guards against cherry-picking: the redistribution is systematic, not a quirk of one or two technologies.
Code
flip |>mutate(tech =factor(tech, TECHS)) |>arrange(tech) |>transmute(Tech = tech, `Shipped top cluster`=`Shipped (sum of z)`,`Target top cluster`=`Target (winners' recipe)`,Flip =ifelse(`Shipped (sum of z)`!=`Target (winners' recipe)`, "⚠ flips", "")) |>kable(caption ="Top-priority cluster under the shipped axis vs the winners'-recipe target. Rebuilding the axis to preserve magnitude and direction moves the headline priority for most technologies.")
Top-priority cluster under the shipped axis vs the winners’-recipe target. Rebuilding the axis to preserve magnitude and direction moves the headline priority for most technologies.
tech
Tech
Shipped top cluster
Target top cluster
Flip
Solar
Solar
Chemicals
Electronics
⚠ flips
Wind
Wind
Machinery
Machinery
Batteries
Batteries
Machinery
Chemicals
⚠ flips
EVs
EVs
Machinery
Machinery
Nuclear
Nuclear
Metals
Chemicals
⚠ flips
Geothermal
Geothermal
Machinery
Machinery
Heat Pumps
Heat Pumps
Machinery
Machinery
Electrolyzers
Electrolyzers
Machinery
Machinery
Transmission
Transmission
Machinery
Electronics
⚠ flips
Magnets
Magnets
Metals
Metals
Biofuel
Biofuel
Machinery
Chemicals
⚠ flips
What the table says, and why it matters. For each technology it names the single most-important cluster under the shipped axis and under the corrected target, and flags where the two disagree. This is the bottom line of the section stated as plainly as possible: the “what should this country build first” signal a reader takes off the Atlas radar is not robust to how the axis is aggregated — for most technologies, preserving magnitude and direction moves the headline priority to a different cluster. For a document meant to guide industrial-policy attention, that is precisely the sort of instability worth fixing at the source.
6 The gap layer, briefly
The target radar answers what to build toward; a country’s gap answers how far it is and where it’s over/under-built. That is the country’s own signed SHAP row read against the target — and it is exactly where the country dimension (§3’s magnitude dispersion) and negative numbers (a surplus the country should rotate out of) do their real work.
This is now built → capability_gap.qmd. It overlays each of the six focal countries’ recent-3-year signed SHAP on the common target, per cluster: a gap radar (target frontier as the outer ring, the country’s fill as % reached), ranked build-priority bars, and a country × tech distance-to-frontier board. E.g. India · Solar — a machinery-heavy economy against an Electronics-led target — reads as a deep Electronics gap with a Machinery surplus.
7 Recommendation
Keep the common profile — it is justified. Countries rank-agree on the shape of each tech’s capability needs (median ρ ≈ 0.88); a single target radar per technology is a faithful summary. EDA 4’s dispersion is a scale fact that belongs to the gap layer, not a refutation of the target.
Build the target axis from the competitive-class signed SHAP, summed per cluster — magnitude-preserving and direction-checked — rather than the sum-of-z, which is a product count. This moves the headline cluster for most techs (5/11).
Do not sign the pooled mean. Pooled negativity (~48% of retained features) is base rate — the have-nots’ deficit — not “avoid.” Report a genuine avoid list from competitive-class-negative features instead; for these 11 techs it is essentially empty, which is itself a clean finding.
Put sign and the country dimension in a gap overlay, not in the target. That is where “this country has a surplus it should rotate out of” is a real, signed, country-specific statement.
Reproduce.scripts/ml/regen_shap.py regenerates the raw signed SHAP; scripts/ml/target_profile_prep.py reconstructs the RCA>1 class per country-year from the CICE trade files and writes shap_replication_country_rank.csv (§3) and shap_replication_by_class.csv (§4–§5). No model re-fit is needed for the class split — the label is deterministic from dv_<finalproduct>.
Source Code
---title: "A Common Technology Profile"subtitle: "Is a single 'standard capability radar' countries evolve toward defensible — and where do negative (avoid) weights belong? A follow-on to the SHAP replication."author: "Net Zero Industrial Policy Lab (NZIPL) · Johns Hopkins SAIS"date: todayformat: html: toc: true toc-depth: 3 toc-location: left toc-title: "Contents" number-sections: true self-contained: true page-layout: full fig-width: 10 fig-height: 5.5 code-fold: true code-tools: true theme: [cosmo, ../nzipl.scss] grid: sidebar-width: 270px body-width: 980px docx: toc: true number-sections: trueexecute-dir: projectexecute: echo: true warning: false message: false---```{r setup}#| code-fold: truelibrary(readr); library(dplyr); library(tidyr); library(stringr); library(forcats)library(ggplot2); library(patchwork); library(knitr); library(showtext); library(tibble)font_add_google("Archivo", "Archivo"); showtext_auto(); showtext_opts(dpi =200)NZ_DARK <-"#073309"; NZ_GREEN <-"#3cb54a"; NZ_TEAL <-"#0d9488"theme_nr <-function(...) theme_minimal(base_family ="Archivo", base_size =12) +theme(plot.title =element_text(size =14, face ="bold", colour = NZ_DARK),plot.subtitle =element_text(size =11, colour ="#475569"),panel.grid.minor =element_blank(), ...)theme_set(theme_nr())CAT_COL <-c(Chemicals="#2563eb", Electronics="#7c3aed", Metals="#ef4444",Machinery="#10b981", "Industrial Materials"="#f59e0b", Other="#6b7280", "Non-HS"="#94a3b8")CATS <-c("Chemicals","Electronics","Industrial Materials","Machinery","Metals")CATS_SHORT <-c(Chemicals="Chem", Electronics="Elec", "Industrial Materials"="Ind.\nMat.",Machinery="Mach", Metals="Met")TECHS <-c("Solar","Wind","Batteries","EVs","Nuclear","Geothermal","Heat Pumps","Electrolyzers","Transmission","Magnets","Biofuel")ml <- here::here("analysis","ml")feat <-read_csv(file.path(ml,"shap_replication_features.csv"), show_col_types =FALSE) |>mutate(retained =as.logical(retained))catt <-read_csv(file.path(ml,"shap_replication_by_category.csv"), show_col_types =FALSE)bycl <-read_csv(file.path(ml,"shap_replication_by_class.csv"), show_col_types =FALSE) |>mutate(retained =as.logical(retained))crank <-read_csv(file.path(ml,"shap_replication_country_rank.csv"), show_col_types =FALSE)ex <-read_csv(file.path(ml,"shap_replication_cy_examples.csv"), show_col_types =FALSE)disp <-read_csv(file.path(ml,"shap_replication_cy_dispersion.csv"), show_col_types =FALSE)gd <-read_csv(here::here("data","green_dict","green_dictionary.csv"), show_col_types =FALSE) |>distinct(code, name) |>mutate(code =as.character(code))hs_name <-function(hs) { nm <- gd$name[match(as.character(hs), gd$code)]; ifelse(is.na(nm), "", nm) }```::: {.callout-note appearance="minimal"}**Where this sits.** The companion `ml_replication.qmd` reproduced the shipped SHAP need-axis from source and showed it discards three things — **magnitude, sign, and the country dimension**. That note was *diagnostic*. This one is *constructive*: the co-directors want a single **standard capability radar per technology — the profile a country's industrial policy steers toward.** That is a legitimate object, but only if (a) a common profile actually *exists* across countries, and (b) we know where "**don't specialise here**" (negative) weights belong. Both questions are answered here from the same regenerated SHAP (`analysis/ml/`, produced by `scripts/ml/regen_shap.py` + `scripts/ml/target_profile_prep.py`), with the RCA>1 "competitive" class reconstructed per country-year from the CICE trade files — no model re-fit.:::# Two layers: the target and the gapThe audit and the co-directors are not actually in conflict — they are describing **two different objects** that the shipped pipeline collapses into one:|| **Target profile** | **Country gap** ||---|---|---|| Question | *What does technology X require?* | *How far is country C from it, and in which direction?* || Varies by country? | **No** — one profile per tech | **Yes** — one per country || Needs sign? | Only to say what's on- vs off-path | **Yes** — surplus vs deficit, build vs avoid || Natural unit | cluster shares of a canonical fingerprint | that country's own SHAP row vs the target |`ml_replication` faulted the shipped radar for being **country-invariant and unsigned**. But country-invariance is a *feature* of the target layer, not a bug — a "standard profile countries evolve toward" is by definition the same for everyone. The critique bites on the **gap** layer, where sign and the country dimension genuinely live. So the constructive question splits cleanly:1. **Is the target layer well-defined?** Do countries agree on *what the technology needs* strongly enough that a single profile is meaningful? (§2–§3)2. **How does sign enter?** Where is "don't specialise here", and does the shipped `|SHAP|` throw away something the target radar needs? (§4)3. **What does the corrected target radar look like, and how is it built?** (§5–§6)# Re-reading EDA 4 — what the dispersion numbers actually said`ml_replication`'s EDA 4 reported four alarming numbers about how the pooled `mean|SHAP|` hides country-year variation. They are easy to over-read, so here is one feature, concretely. Take Solar's **HS 830249 (mounting equipment)** — a retained radar feature — across all `r nrow(ex |> filter(tech=="Solar", feature=="s_830249"))` country-years:```{r eda4-worked, fig.height=3.9}#| code-fold: truew <- ex |>filter(tech=="Solar", feature=="s_830249")mw <-mean(w$abs_shap)stats <-tibble(Statistic =c("CV (spread ÷ mean)", "max ÷ mean", "% of country-years below ½ the mean","top-2% country-years' share of total"),Value =c(sprintf("%.2f", sd(w$abs_shap)/mw),sprintf("%.0f×", max(w$abs_shap)/mw),sprintf("%.0f%%", mean(w$abs_shap < mw/2)*100),sprintf("%.0f%%", sum(sort(w$abs_shap, decreasing=TRUE)[1:ceiling(.02*nrow(w))])/sum(w$abs_shap)*100)),`Plain meaning`=c("the importance varies more across countries than its own average","for one country-year it matters ~17× more than the reported mean","for a third of countries it is nearly inert","a handful of countries carry most of the total"))ggplot(w, aes(abs_shap +1e-6)) +geom_histogram(bins =46, fill = NZ_GREEN, colour ="white", linewidth = .15) +geom_vline(xintercept = mw, colour = NZ_DARK, linewidth = .7) +annotate("text", x = mw*1.15, y =Inf, vjust =1.6, hjust =0, family ="Archivo", size =3,colour = NZ_DARK, label ="the mean\nthe axis keeps") +scale_x_log10() +labs(x ="|SHAP| for HS 830249 in one country-year (log scale)", y ="country-years",title ="One retained feature, across every country-year",subtitle =paste0("Solar · mounting equipment. Importance is concentrated (peak in Cambodia 2021-24); ","the single mean sits low in a long right tail."))``````{r eda4-table}#| code-fold: truekable(stats, caption ="The four EDA-4 statistics for one feature, in plain language. Every one of them is a statement about how MAGNITUDE varies across countries.")```**Reading the histogram.** The horizontal axis (log scale) is this feature's `|SHAP|` in a *single* country-year; each bar counts how many country-years fall in that importance band; the vertical line is the one average the pipeline keeps. The mean sits low on a long right tail — dragged down by the many countries for which mounting-equipment capability barely moves the prediction, and pulled up only by the handful (here, Cambodia 2021–24) for which it is decisive. So the single reported number describes almost no individual country well. The table restates that in four ways.**The point that was missing.** Every one of these numbers measures how the *magnitude* of a feature's importance varies across countries. That matters enormously for a **country gap** ("how much does mounting-equipment capability matter *for Cambodia specifically*") — but it does **not**, on its own, sink a **target profile**. A standard profile does not require the magnitude to be equal across countries; it only requires the **shape** — which capabilities rank high relative to each other — to be shared. EDA 4 measured scale dispersion and left the shape question open. §3 asks it directly.# Does a common profile exist? Shape, not scaleFor each technology, take **each country's own importance vector** (its mean `|SHAP|` per feature over its years) and rank-correlate it against the pooled profile. If countries agree on the *ranking* of capabilities — even while differing wildly in magnitude — then a single target profile is a faithful summary of a shared shape.```{r rank-agree, fig.height=4.4}#| code-fold: truecr <- crank |>filter(!is.na(spearman_retained)) |>mutate(tech =factor(tech, TECHS))med <- cr |>group_by(tech) |>summarise(m =median(spearman_retained), .groups="drop")p1 <-ggplot(cr, aes(x =fct_rev(tech), y = spearman_retained)) +geom_hline(yintercept =c(0, .5, 1), linetype ="22", colour ="grey85") +geom_violin(fill = NZ_GREEN, alpha = .25, colour =NA, scale ="width") +geom_boxplot(width = .18, outlier.size = .5, fill ="white", colour = NZ_DARK, linewidth = .4) +coord_flip(ylim =c(0, 1)) +labs(x =NULL, y ="each country's rank-agreement with the pooled profile (Spearman ρ)",title ="Countries agree on the SHAPE of what each tech needs",subtitle =paste0("Per-country importance vectors vs the pooled profile, retained radar features. ","Median ρ = ", round(median(cr$spearman_retained),2), " overall."))p2 <-ggplot(cr, aes(frac_competitive, spearman_retained)) +geom_point(alpha = .25, size =1, colour = NZ_TEAL) +geom_smooth(method ="lm", se =FALSE, colour = NZ_DARK, linewidth = .7) +labs(x ="share of years the country is RCA-competitive", y ="rank-agreement (ρ)",title ="Have-nots share the starting line; winners specialise",subtitle ="Non-competitive countries hug the common profile; competitive ones deviate into niches.")p1 | p2```**Reading the two panels.** Let us take them one at a time. The left panel has one row per technology; each violin-and-box summarises, across that technology's many countries, a single score per country — how closely that country *ranks* the capabilities the same way the pooled profile does (Spearman's ρ, where 1 means an identical ordering and 0 means no relationship). The boxes cluster high and tight, around 0.8–0.95, so the reassuring message is that countries broadly agree on *which* capabilities a technology leans on, even though EDA 4 showed they differ enormously in *how much*. The right panel then asks a gentler follow-up: does that agreement depend on how advanced a country is? Each dot is one country; the horizontal axis is the share of years it was competitive (`RCA>1`), the vertical axis its rank-agreement. The line slopes *down* — at first glance surprising — because a country that is short on nearly everything trivially traces the common shape, while a country that has actually become competitive has bent its own profile toward a specialisation. So the shared profile is best pictured as the on-ramp everyone starts on, not a description of where the winners end up.```{r rank-summary}#| code-fold: truecrank |>filter(!is.na(spearman_retained)) |>mutate(tech =factor(tech, TECHS)) |>group_by(tech) |>summarise(`N countries`=n(), `median ρ`=round(median(spearman_retained),2),`10th pct ρ`=round(quantile(spearman_retained,.1),2),`% above 0.7`=round(mean(spearman_retained>.7)*100,0), .groups="drop") |>arrange(tech) |>kable(caption ="Per-tech rank agreement between each country's importance vector and the pooled profile. Even the 10th-percentile country is well above chance; a common profile is a defensible summary of a shared shape.")```**Read.** Median rank-agreement is **`r round(median(cr$spearman_retained),2)`** and never below ~0.8 at the tech level — countries overwhelmingly agree on *which* capabilities a technology leans on, differing mainly in *how much* (the EDA-4 scale story). So the co-directors' object is justified: **a single target profile is a real, shared shape.** The one nuance (right panel): countries that are *not yet* competitive agree with the profile **most** (they are uniformly "short" everywhere), while *competitive* producers deviate — each has specialised into a niche. The common profile is therefore best read as the **aspirational path a not-yet-competitive country climbs**, with successful producers branching into specialisations once on it.# Where negative weights belongThe intuition behind wanting sign is right: a target radar should distinguish "**build this**" from "**don't specialise here**", and the shipped `|SHAP|` cannot. But the naive fix — sign the *pooled* mean SHAP — is **wrong**, and the replication data shows exactly why.```{r sign-scatter, fig.height=4.6}#| code-fold: truerr <- bycl |>filter(retained, category %in% CATS)share_neg_pooled <-round(mean(rr$mean_shap_signed <0)*100, 0)share_neg_comp <-round(mean(rr$mean_shap_comp <0)*100, 0)ggplot(rr, aes(mean_shap_signed, mean_shap_comp, colour = category)) +geom_hline(yintercept =0, colour ="grey60") +geom_vline(xintercept =0, colour ="grey60") +geom_point(alpha = .7, size =1.7) +scale_colour_manual(values = CAT_COL, name =NULL) +annotate("rect", xmin =-Inf, xmax =0, ymin =0, ymax =Inf, alpha = .05, fill = NZ_GREEN) +annotate("text", x =min(rr$mean_shap_signed), y =max(rr$mean_shap_comp), hjust =0, vjust =1,family ="Archivo", size =3.1, colour = NZ_DARK,label ="pooled sign says 'negative'\nbut winners' sign says 'build'\n= a GAP, not an anti-target") +labs(x ="pooled mean SHAP (signed) — what a naive sign would use",y ="mean SHAP among RCA-competitive producers — the target direction",title ="Pooled negativity is a base-rate artifact, not 'avoid'",subtitle =paste0(share_neg_pooled, "% of retained features are pooled-negative; ", share_neg_comp, "% are negative among the winners.")) +theme(legend.position ="bottom")```**Reading the plot.** Each dot is one retained radar feature. Its horizontal position is the *pooled* mean signed SHAP — the number you would get by naively adding the sign back to the shipped axis, averaging over every country. Its vertical position is the mean SHAP among only the countries that are actually competitive (`RCA>1`) — the direction that matters if we want to describe a *winning* profile. The shaded band in the top-left is the crux: features that look **negative** when pooled over everyone but sit **positive** among the winners. A feature there is emphatically not something to avoid — competitive producers plainly benefit from it; it merely reads negative overall because most countries have not built it yet. That is the signature of a *gap*, not an anti-target — and it is why signing the pooled mean would mislead.Because `RCA>1` is rare (~5% base rate), the large non-competitive majority reads as "not enough of everything" — so **`r share_neg_pooled`% of retained features carry a net-negative *pooled* SHAP**. That is the deficit of the have-nots, i.e. the **gap**, not a signal that the capability is bad to hold. Split the sign by the target class and the artifact disappears:```{r sign-class-summary}#| code-fold: trueallf <- bycl |>filter(category %in% CATS)tibble(Group =c("All radar features", "Retained radar features"),`N`=c(nrow(allf), nrow(rr)),`% pooled-SHAP negative`=c(round(mean(allf$mean_shap_signed<0)*100,0), round(mean(rr$mean_shap_signed<0)*100,0)),`% negative among WINNERS`=c(round(mean(allf$mean_shap_comp<0)*100,0), round(mean(rr$mean_shap_comp<0)*100,0)),`% where winners < non-winners (disc<0)`=c(round(mean(allf$disc<0)*100,0), round(mean(rr$disc<0)*100,0))) |>kable(caption ="Sign under two readings. Among competitive producers, essentially no retained feature points 'away' — and none is systematically lower among winners than non-winners. The pooled negativity was base rate.")```Among the **`r nrow(rr)`** retained radar features, **`r sum(rr$mean_shap_comp<0)`** are negative among competitive producers, and **`r sum(rr$disc<0)`** are lower among winners than non-winners. In other words, for these 11 technologies there is **no meaningful "don't specialise here" among the capabilities that reach the radar** — the winning recipes are uniformly additive. Genuine anti-targets do exist, but they are rare and sit *below* the selection bar:```{r avoid-table}#| code-fold: trueavoid <- bycl |>filter(category %in% CATS, mean_shap_comp <0) |>arrange(mean_shap_comp) |>transmute(Tech = tech, HS =str_remove(feature, "^n?s_"),Product =hs_name(str_remove(feature, "^n?s_")), Cluster = category,`SHAP among winners`=round(mean_shap_comp, 5),Retained =ifelse(retained, "yes (on radar)", "no (below cut)"))if (nrow(avoid) ==0) {cat("_No feature — retained or not — has a negative SHAP among competitive producers._")} elsekable(avoid, caption ="The only genuine 'avoid' candidates across all 11 techs: features that push competitive producers' predicted competitiveness DOWN. All sit below the selection cut, so none reaches the shipped radar.")```**Takeaway on sign.** Sign belongs in the model, but its job is **directional confirmation**, not carving negative wedges out of the radar. Building the target from the **competitive-class signed SHAP** gives an axis that *is* signed (so it can flag an anti-target if one appears) and *is* magnitude-preserving — and it happens to be all-positive here. The place negative numbers do real work is the **gap layer**: a specific country with a *surplus* in a cluster it should be de-emphasising. That is a country-level reading, not a property of the tech's profile.# The target radar, built the right wayPutting §3 (a common profile exists) and §4 (sign belongs to the winners) together, we can finally state how the corrected radar is built. It is worth going slowly here, because the whole argument rests on this one construction.**What "competitive-class signed SHAP" means.** Recall from §4 that for every country-year the model produces a *signed* SHAP value per feature — positive if that capability pushed the country *toward* predicted competitiveness, negative if *away*. The model's target is binary: is the country RCA-competitive (`RCA>1`) in the technology, or not? That splits the country-years into a **competitive class** (the ~5% that are the actual "winners") and the rest. A feature's *competitive-class signed SHAP* is simply the average of its signed value **over the winners only** — in plain terms, *among the countries that actually became competitive, how much did this capability contribute, on net, to getting them there?* We average over the winners rather than over everyone on purpose: the pooled mean is dragged negative by the 95% who have not built the capability yet (the base-rate gap of §4), whereas the winners' average isolates the recipe itself.**Building the axis, in four steps.** With that single number in hand for each feature:1. **Assign** every retained feature to its capability cluster via its HS chapter — e.g. Solar's `854190` (semiconductor/LED components) → Electronics.2. **Sum** the winners' contributions within each cluster, clipping any negative to zero (a safeguard that, as §4 showed, never actually bites for retained features): $\text{target}_c = \sum_{j\in c} \max(\overline{\phi_j}^{\text{comp}},\,0)$. This is the **magnitude-preserving** step — a feature contributing 0.07 adds 0.07, one contributing 0.002 adds 0.002 — unlike the shipped z-score, which first squashes every retained feature onto ~0.5–0.85 so that summing them merely *counts* them.3. **Normalise** each cluster's total by the sum across all five clusters, giving shares that add to 100% — the shape the radar draws.4. **Plot** the five shares as a pentagon: five spokes 72° apart, one per cluster, the radius on each being that cluster's share. Each facet below is self-scaled to its own longest spoke, so we are comparing *shapes*, not absolute sizes.The axis is **direction-checked** because we used the *signed* SHAP and kept only the positive winners' contributions: it can only reflect capabilities that push winners *toward* competitiveness, and a genuine anti-target (negative even among winners) would be clipped out and surfaced separately, not silently inflate a cluster.**A worked read — Solar.** The Electronics features' winners'-SHAP sum is ≈ 0.21 against a five-cluster total of ≈ 0.34, so Electronics makes up ≈ **62%** of Solar's target — carried by a few heavy contributors (`854190` alone ≈ 0.071). The shipped sum-of-`z` axis instead crowns **Machinery**, which merely holds more product codes. That one contrast — magnitude vs count — is what the radars below make visible for every technology at once.```{r radar-build}#| code-fold: truetgt <- bycl |>filter(retained, category %in% CATS) |>group_by(tech, category) |>summarise(w =sum(pmax(mean_shap_comp, 0)), .groups="drop") |>group_by(tech) |>mutate(share = w/sum(w)) |>ungroup() |>transmute(tech, category, series ="Target (winners' recipe)", share)shp <- catt |>transmute(tech, category, series ="Shipped (sum of z)", share = pct_shipped_sum_z/100)rad <-bind_rows(tgt, shp) |>complete(tech, category = CATS, series, fill =list(share =0))# top cluster under eachflip <- rad |>group_by(tech, series) |>slice_max(share, n =1, with_ties =FALSE) |>select(tech, series, category) |>pivot_wider(names_from = series, values_from = category)``````{r radar-plot, fig.height=9.6, fig.width=10}#| code-fold: truesamp <- TECHS # all 11 technologiesang <-tibble(category = CATS, ang = pi/2-2*pi*(0:4)/5)poly <- rad |>filter(tech %in% samp) |>left_join(ang, by="category") |>group_by(tech) |>mutate(rr = share/max(share)) |>ungroup() |>mutate(x = rr*cos(ang), y = rr*sin(ang), o =match(category, CATS), tech =factor(tech, samp))polyc <-bind_rows(poly, poly |>filter(category==CATS[1]) |>mutate(o =6)) |>arrange(tech, series, o)rings <-expand_grid(tech =factor(samp, samp), ring =c(.33,.66,1), category = CATS) |>left_join(ang, by="category") |>mutate(x = ring*cos(ang), y = ring*sin(ang), o =match(category, CATS))ringsc <-bind_rows(rings, rings |>filter(category==CATS[1]) |>mutate(o=6)) |>arrange(tech, ring, o)spokes <-expand_grid(tech =factor(samp, samp), category = CATS) |>left_join(ang, by="category") |>mutate(x =cos(ang), y =sin(ang))axlab <-expand_grid(tech =factor(samp, samp), category = CATS) |>left_join(ang, by="category") |>mutate(x =1.24*cos(ang), y =1.24*sin(ang), lab = CATS_SHORT[category])ggplot() +geom_path(data = ringsc, aes(x, y, group =interaction(tech, ring)), colour ="grey86", linewidth = .3) +geom_segment(data = spokes, aes(0, 0, xend = x, yend = y), colour ="grey86", linewidth = .3) +geom_polygon(data = polyc, aes(x, y, fill = series, colour = series), alpha = .3, linewidth = .8) +geom_text(data = axlab, aes(x, y, label = lab), family ="Archivo", size =2.5, colour ="#334155", lineheight=.85) +facet_wrap(~tech, ncol =4) +scale_fill_manual(values =c(`Shipped (sum of z)`="#94a3b8", `Target (winners' recipe)`=NZ_GREEN), name=NULL) +scale_colour_manual(values =c(`Shipped (sum of z)`="#64748b", `Target (winners' recipe)`=NZ_TEAL), name=NULL) +coord_equal(clip ="off") +xlim(-1.4, 1.4) +ylim(-1.35, 1.35) +labs(title ="Shipped radar vs a winners'-recipe target radar",subtitle ="Each facet self-scaled to its longest axis. Grey = shipped count-driven axis; green = magnitude- and direction-preserving target.") +theme_void(base_family ="Archivo") +theme(legend.position ="bottom", strip.text =element_text(family="Archivo", face="bold", colour=NZ_DARK, size=11),plot.title =element_text(family="Archivo", face="bold", colour=NZ_DARK, size=14),plot.subtitle =element_text(family="Archivo", colour="#475569", size=10.5))```**Reading the radars.** Each pentagon is one technology; the five spokes are the capability clusters; the two overlaid shapes are the shipped, count-driven axis (grey) and the winners'-recipe target (green). Each is scaled to its own longest spoke, so what we are comparing is the *shape* of the two profiles, not their absolute size. Wherever the green shape reaches out beyond the grey on a spoke, the corrected target places more weight on that cluster than the shipped axis does; wherever it falls short, less. The silhouettes are not subtle variations of each other — read Solar, for example, where the grey shape bulges toward Machinery while the green pulls firmly toward Electronics. That visible disagreement is the same story EDA 2 told with numbers, now in a single glance.```{r radar-heat, fig.height=5.2}#| code-fold: truehd <- rad |>mutate(tech =factor(tech, rev(TECHS)), category =factor(category, CATS),series =factor(series, c("Shipped (sum of z)","Target (winners' recipe)")))ggplot(hd, aes(category, tech, fill = share*100)) +geom_tile(colour ="white", linewidth = .6) +geom_text(aes(label =ifelse(share*100<1, "", sprintf("%.0f", share*100))),family ="Archivo", size =2.7, colour ="#0b1a12") +facet_wrap(~series) +scale_fill_gradient(low ="#f0f7f0", high = NZ_GREEN, na.value ="grey93", name ="% of need") +labs(x =NULL, y =NULL, title ="Cluster need across all 11 techs — shipped vs target",subtitle ="Shipped lights up Machinery (the largest-count cluster); the target redistributes toward where magnitude actually is.") +theme(axis.text.x =element_text(angle =20, hjust =1), legend.position ="bottom")```The heatmap widens the same comparison to all eleven technologies at once. The layout is the one from `ml_replication`'s EDA 2 — rows are technologies, the five columns are clusters, each cell is that cluster's share of the technology's "need" — but here the right-hand panel is the *corrected target* rather than raw magnitude. Reading down the left panel, the Machinery column is lit up almost everywhere, the count-driven pattern we already diagnosed; reading the right panel, the brightness disperses toward wherever the competitive producers' SHAP genuinely concentrates. Seeing all eleven together guards against cherry-picking: the redistribution is systematic, not a quirk of one or two technologies.```{r flip-tab}#| code-fold: trueflip |>mutate(tech =factor(tech, TECHS)) |>arrange(tech) |>transmute(Tech = tech, `Shipped top cluster`=`Shipped (sum of z)`,`Target top cluster`=`Target (winners' recipe)`,Flip =ifelse(`Shipped (sum of z)`!=`Target (winners' recipe)`, "⚠ flips", "")) |>kable(caption ="Top-priority cluster under the shipped axis vs the winners'-recipe target. Rebuilding the axis to preserve magnitude and direction moves the headline priority for most technologies.")```**What the table says, and why it matters.** For each technology it names the single most-important cluster under the shipped axis and under the corrected target, and flags where the two disagree. This is the bottom line of the section stated as plainly as possible: the "what should this country build first" signal a reader takes off the Atlas radar is *not robust* to how the axis is aggregated — for most technologies, preserving magnitude and direction moves the headline priority to a different cluster. For a document meant to guide industrial-policy attention, that is precisely the sort of instability worth fixing at the source.```{r flip-closing}#| code-fold: true#| include: falseinvisible(NULL)```# The gap layer, brieflyThe target radar answers *what to build toward*; a country's **gap** answers *how far it is and where it's over/under-built*. That is the country's own signed SHAP row read against the target — and it is exactly where the country dimension (§3's magnitude dispersion) and negative numbers (a **surplus** the country should rotate out of) do their real work.::: {.callout-tip appearance="minimal"}**This is now built → `capability_gap.qmd`.** It overlays each of the six focal countries' recent-3-year signed SHAP on the common target, per cluster: a gap radar (target frontier as the outer ring, the country's fill as % reached), ranked build-priority bars, and a country × tech distance-to-frontier board. E.g. **India · Solar** — a machinery-heavy economy against an Electronics-led target — reads as a deep **Electronics** gap with a **Machinery** surplus.:::# Recommendation1. **Keep the common profile — it is justified.** Countries rank-agree on the shape of each tech's capability needs (median ρ ≈ `r round(median(cr$spearman_retained),2)`); a single target radar per technology is a faithful summary. EDA 4's dispersion is a *scale* fact that belongs to the gap layer, not a refutation of the target.2. **Build the target axis from the competitive-class signed SHAP**, summed per cluster — magnitude-preserving and direction-checked — rather than the sum-of-`z`, which is a product count. This moves the headline cluster for most techs (`r sum(flip[["Shipped (sum of z)"]] != flip[["Target (winners' recipe)"]])`/11).3. **Do not sign the pooled mean.** Pooled negativity (~`r share_neg_pooled`% of retained features) is base rate — the have-nots' deficit — not "avoid." Report a genuine **avoid list** from competitive-class-negative features instead; for these 11 techs it is essentially empty, which is itself a clean finding.4. **Put sign and the country dimension in a gap overlay**, not in the target. That is where "this country has a surplus it should rotate out of" is a real, signed, country-specific statement.::: {.callout-tip appearance="minimal"}**Reproduce.** `scripts/ml/regen_shap.py` regenerates the raw signed SHAP; `scripts/ml/target_profile_prep.py` reconstructs the RCA>1 class per country-year from the CICE trade files and writes `shap_replication_country_rank.csv` (§3) and `shap_replication_by_class.csv` (§4–§5). No model re-fit is needed for the class split — the label is deterministic from `dv_<finalproduct>`.:::
