MoE-XRay / Model analysis
Inspecting which experts a model uses for different tasks.
OLMoE routing traces, task-category comparisons and sanity checks, with analysis code and recorded results.
The problem
A model’s final answer does not show how its experts were used. MoE-XRay investigates whether routing patterns differ across code, maths, factual and story prompts.
The work
- Capture expert-routing traces and turn them into per-layer routing signatures.
- Compare similarity within and across task categories, with a load-balancing baseline.
- Record label-shuffle and seed-variation checks alongside the analysis results.
What the experiment establishes
The recorded experiment covers 80 traces from one OLMoE model. Routing differences do not establish better answer quality or generalise to every MoE model.