A retrieval improvement needs a strong baseline

Start with the decision

MultiplexRAG asks a specific question: should every query use the same retrieval method? Sparse search can be useful when exact terms matter. Dense search can connect related language. A hybrid can combine their rankings. A router adds another decision: choose one of those paths for the current query.

That extra decision is not automatically valuable. It introduces a learned component, features, a confidence threshold and another way to fail. The useful comparison is against the strongest fixed method under the same protocol, including a good hybrid baseline. Comparing only with a weak sparse or dense setup would make the router look better without answering the practical question.

Keep the result attached to its split

In the stored Phase 4 NFCorpus validation report, the routed system reaches MRR@10 of 0.5270 against 0.5110 for the fixed hybrid baseline, over 324 queries. MRR@10 summarizes how highly a relevant result appears within the first ten positions. Those numbers describe ranking quality on that particular evaluation.

The word validation matters. A validation set can influence model and configuration choices. Calling the same result a held-out test gain would give it a stronger meaning than the evidence supports. The Phase 4 preset is also distinct from the default integrated pipeline, so reproducing the default command is not automatically reproducing that result.

A fallback is part of the design

MultiplexRAG uses a confidence-based fallback. When the router is uncertain, it can return to a fixed retrieval strategy instead of insisting on a fragile choice. That makes the threshold part of the system being evaluated, rather than a detail to omit from the result.

A useful report keeps the feature set, label construction, confidence threshold and fallback visible. Ablations then help separate the contributions of those decisions. A higher final score alone cannot explain whether the useful change came from routing, richer features or a different fallback.

What the experiment does not answer

A retrieval metric does not establish that a complete generated answer is correct. It also says little about production latency, infrastructure cost or a new document collection. Those require their own measurements and failure analysis. A router that helps one dataset can add complexity without helping another.

Reproduction also needs more than a score pasted into a README. The report should identify the data split, selected configuration and actual comparison. Checking saved arithmetic is useful, but it is different from rerunning retrieval and observing the same result.

For a project case study, I want the reader to be able to find the baseline, split, population and configuration without reverse-engineering a headline. The MultiplexRAG repository links the implementation and stored reports. They support a bounded retrieval result and a clear next question: does the same decision remain useful on genuinely new queries under the intended operating constraints?