RESEARCH
Every number, with the setup that produced it.
We publish the questions, the answer model, the judge, the date and the code. We report answer accuracy and precision, not only how often the right lines came back.
Results
We publish the results after our current test round: all four benchmarks at once, with the weaker numbers next to the strong ones.
| Benchmark | What it tests |
|---|---|
| LongMemEval-S | 500 questions · six types |
| LoCoMo | 10 long conversations |
| HaluMem | QA · hallucination and omission rates too |
| BEAM | 128K to 10M tokens · run once, on frozen code |
How we measured
Answer modelQwen3-14B, open weights, the same for every benchmark
Two runs eachThe ceiling: only the gold evidence · Remiq: Remiq's context
JudgeQwen3-32B, open weights, with each benchmark's own judging prompt
What countsEach benchmark's own scoring
MeasuredDated next to the results
Test kitPublished with the results
The ceiling is what the answer model can do with perfect evidence, so Remiq's share of it shows what our memory keeps and what it loses. Where we studied a benchmark while building Remiq, we say so next to its number. Until a number is measured this way, it stays a placeholder.