One system for cheaper inference — and the proof that it works.
BTB Intelligence is an inference-optimization company. Our system is a stack of independent layers: each one removes a different source of serving cost, and a proof layer turns the speed-up into a result the customer can check. Layers can be deployed one at a time or together. The methods are not disclosed — a patent application is in preparation.
The system
Figures in the diagram: internal measurements by BTB Intelligence on NVIDIA B200, September 2026 — see the table and notes below.
The layers
BTB Micro — input efficiency
Lets the same GPU serve up to 2× more concurrent users at the same latency target — median 1.9–2.0×, best case 2.37× — measured on NVIDIA B200 against a tuned vLLM setup across five open models, with answer quality held at parity under blinded judging. No retraining; once deployed it runs inside the customer's own environment.
BTB CASCADE — a right-sized model for every request
Gives every request a model of the size it actually needs, so routine traffic stops paying for the largest model while hard requests still get it. First measurements are done; results will be published once verified.
Regime dispatcher
Recognises whether traffic is interactive or batch and chooses which combination of layers is active for it — fast first answers for chat, maximum throughput for batch jobs.
QUANT family — full-quality low-precision inference
Runs large language models at reduced numerical precision while meeting official accuracy targets. Llama 2 70B: 13,511 tokens/s on one B200 — ×3.23 against standard 16-bit serving on the same harness. Mixtral 8x7B: 24,254 tokens/s on one B200 — ×4.18 — interactive median latency 3.3 s → 1.5 s, and 0.0275 J per token, about half the energy of standard reduced-precision serving. On 8× B200 systems: 198,896 tokens/s (Mixtral 8x7B, with BTB Micro) and 107,384 tokens/s (Llama 2 70B).
Fewer sequential generation steps
On structured and code tasks the engine reaches the same answer in fewer sequential steps: ×1.35 faster, 25% less energy per token, output identical bit for bit.
Proof layer — every result verifiable
Every run is written to a sealed, append-only record that shows any later change; quality is certified against the baseline before a speed-up counts. The September 2026 B200 campaign is sealed in one manifest of 158 result files.
Measured on NVIDIA B200
| Workload and setup | Result | Compared with | Quality |
|---|---|---|---|
| Mixtral 8x7B · 8× B200 system · QUANT family + BTB Micro | 198,896 tokens/s | +54.4% over the best published 8× B200 result | Open-class: summarization accuracy targets met; one maths-reasoning target not met |
| Mixtral 8x7B · 8× B200 system · QUANT family alone | 193,233 tokens/s | +50.0% over the best published 8× B200 result | Open-class (as above) |
| Llama 2 70B · 8× B200 system · QUANT family | 107,384 tokens/s | +5.8% over NVIDIA's published 8× B200 result | Official accuracy targets met (Closed-style) |
| Llama 2 70B · per GPU, Closed-style version | 13,488 tokens/s | +6.3% over NVIDIA's published per-GPU figure on B200 | 100.9–102.2% of each official accuracy target |
| Llama 2 70B · 1× B200 · QUANT family vs standard 16-bit | 13,511 tokens/s | ×3.23 vs 4,185 tokens/s with standard 16-bit | Accuracy targets met |
| Mixtral 8x7B · 1× B200 · QUANT family vs standard 16-bit | 24,254 tokens/s | ×4.18 vs 5,796 tokens/s with standard 16-bit; interactive median latency 3.3 s → 1.5 s | Open-class (as above) |
| Mixtral 8x7B · energy per token | 0.0275 J/token | 0.0561 J/token with standard reduced-precision serving (about 2× less) | — |
| Structured and code tasks · fewer sequential steps · 1× B200 | ×1.35 faster | 25% less energy per token | Output identical bit for bit (verified on code tasks) |
| BTB Micro · five open models · 1× B200 | up to 2× concurrent users | Tuned vLLM at the same latency target (median 1.9–2.0×, best 2.37×) | Parity under blinded judging |
Internal measurements by BTB Intelligence, September 2026, on an industry-standard inference benchmark (Offline scenario, full official sample counts, quality gates). Not submitted for official review.
Result not verified by MLCommons Association. These results have not been through MLCommons review and may use measurement methodologies and/or workload implementations that differ from the specification for verified results.
“Best published 8× B200 result” means the highest published result for the same benchmark, scenario and system size as of September 2026; comparisons are system to system on the same GPU count. “NVIDIA's published 8× B200 result” is NVIDIA's own published result for the same benchmark, scenario and system size.
The gain on any given system depends on its hardware, models and traffic.
Proof layer — every result verifiable
Every run is written to a sealed, append-only record, and quality is certified against the baseline before a speed-up counts — any later change is visible. Our September 2026 B200 campaign is sealed in one manifest of 158 result files.
Talk to us about your inference
If you run LLM inference on NVIDIA GPUs and want to know what this could mean for your workload, write to us — we answer personally.
Email: info@btbintelligence.com
Contact usOur own models — in development
A second line of work, next to the inference system: our own language models and the platform that will deliver them.
BTB Blitz and BTB Tsar are BTB Intelligence's own language models, trained verifier-first: an answer earns training signal only if a machine can check it — mathematics is recomputed, code is executed, optimisation results are certified by solvers, facts are checked against sources. Training covers more than twenty separately verified domains, each with a sealed acceptance exam. Neither model is released yet. BTB Blitz · BTB Tsar
Optimum (in development) turns a real decision — a budget, a schedule, a plan — into a formal problem, solves it, and returns the answer together with the reason it is the best available.
Privacy by architecture — BTB Brain
BTB Brain, our app in private testing, keeps what you put in its vault — passwords, banking details, documents — encrypted with a key derived from your password that never leaves your device; we cannot read it. Your conversations and working memory are stored on our server in the EU (Finland), under your account, so they follow you across devices, and you can delete them at any time. We do not sell personal data.