Precision where the question is, not everywhere
A question about biology and a question about a proof do not need more or less precision. They need precision in different places.
Every quantization scheme decides precision per fixed unit - the model, a layer, a channel, a token, an expert. The boundaries come from the architecture; a controller only picks how many bits each unit gets.
FoQLens asks the model itself which weights matter for this query, places those blocks on a map, and reads the rest coarsely. Mixture of Experts is the rigid special case of the same idea: experts with hard edges, fixed at training, opened by a router.
The field above is not decoration. It is the real weight map of Gemma 4 E2B from run E009 - 14 708 blocks of 64 output rows each, placed so that blocks lighting up together lie close, colored from the first layer to the last.
Coarse pixels are weights read at few bits; sharp ones are read at many. Memory is not a budget you set - it is what your lenses cost.
The mechanism
Three controls, each doing one thing: the glass - what everything outside the lenses keeps; focus_area - how large the lenses are; focus_strength - how far their centers rise above the glass. The rules below are what the field above runs; the full text is in docs/lens.md.
ladder ℓ₀ < ℓ₁ < … < ℓ_m the levels this model can be read at; ℓ₀ = ZERO glass G = ℓ_γ any rung 1 radius R_i = r_i · f / (1 − f) 2 ceiling κ = γ + ⌊g · (m − γ)⌋ g = 0: the lens is the glass; g = 1: the top rung 3 profile stops s_κ < … < s_(γ+1) where each ring ends, in radii 4 lift L_i(b) = max(0, 1 − ρ_i(b) / s_last) 5 combine L(b) = min(1, Σ L_i) or max L_i 6 level ρ* = s_last · (1 − L(b)) the ring that ρ* falls into; outside every lens: G 7 memory bits = Σ w_b · bits(b) / Σ w_b
Nothing here names a level: the ladder comes from the model and its storage. On Gemma 4 E2B one stored copy of residual slices is read at 2, 4, 6 or 8 bits, so the ladder is ZERO, D2, D4, D6, D8.
Where this stands: nothing has been tested yet
The bench has been reset. Not one hypothesis carries a verdict, in either direction, and the runs that produced verdicts have had them withdrawn rather than reworded.
Three things were wrong at once, and each alone is enough to void a measurement.
The questions. Four options, scored by which of A-D the model ranks highest. On only 31% of them is it right under every ordering of the options; on the mathematics subject, 3%, because those questions are answered by calculating and the format allows one token to do it in. More than half the set is a coin toss no layout can move.
The mask. It is the gradient of the model's language-model loss on the prompt - it shows which blocks the text lights up, not which blocks the answer needs. Nothing in it ever asked about the answer. Reordering the four options changes the text, so the mask moves with it: same question, same meaning, a different address.
The comparisons. Built on those two, so whichever way they came out, they were reading the coin toss.
What is not being claimed any more. Everything: that the query's lenses beat uniform quantization at the same memory, that quality holds at a third of the memory, that one topic's mask beats another's, and the negative readings just as much. Two configurations agreeing on a set answered by guessing means they guess alike - not that quality held.
What is being rebuilt, in order. First data the bench can trust: a corpus measured rather than chosen, where the model's answers come from knowledge, and a metric with nothing to lean on - the model reads a passage and writes the answer, scored the way SQuAD scores it. Then the mask, which has to be asked about the answer rather than about the text. Only then the hypotheses, from the first.
The runs, their code, their raw numbers and their preregistrations all stay in the repository. They are what re-testing will be measured against - not evidence, but a baseline to beat.