Bottom line
GLM-5.1 has promising reported results, but the evidence answers two different questions. Z.ai’s April 7, 2026 launch post reports a 58.4 on SWE-Bench Pro and compares it with several other models. Lambda reports additional quality scores and self-hosted serving measurements. Those figures are useful for deciding whether to shortlist GLM-5.1; they do not establish how a particular hosted API will perform on your application. The published material does not provide enough evaluation detail to treat the scores as an independent, fully reproducible ranking.
As of November 9, 2026, Z.ai also has newer GLM models, including GLM-5.3. If you are selecting a model today rather than specifically evaluating GLM-5.1, compare the exact versions you can access—not just this model’s older benchmark results.
GLM-5.1 benchmark scores reported
| Benchmark | GLM-5.1 score reported by Lambda | GLM-5 reference shown by Lambda | What the evidence supports |
|---|---|---|---|
| Terminal-Bench 2 | 63.5 | 56.2 | A higher reported GLM-5.1 score on this row |
| AIME 25 | 95.3 | Not supplied | A GLM-5.1 score; no comparison with GLM-5 from this table |
| HLE | 31.0 | 30.5 | A higher reported GLM-5.1 score on this row |
| GPQA | 86.2 | Not supplied | A GLM-5.1 score; no comparison with GLM-5 from this table |
| SWE-Bench Pro | 58.4 | Not supplied | A GLM-5.1 score; no GLM-5 comparison from this table |
These are the values in Lambda’s table for GLM-5.1 FP8 and GLM-5 FP8 reference. Lambda’s table does not state score scales, benchmark versions, evaluation dates, or scoring protocols. Its dashes mean the comparison value is not provided—not that GLM-5 scored zero.
Z.ai’s own launch post also reports 58.4 on SWE-Bench Pro and shows a comparison table with GLM-5 at 55.1, GPT-5.4 at 57.7, Claude Opus 4.6 at 57.3, and Gemini 3.1 Pro at 54.2. This is a vendor-published comparison on the same named benchmark, but the post does not provide enough protocol detail to establish independent reproduction or generalize the result to every coding workload.
Selection takeaway: The results make GLM-5.1 worth testing for software-engineering work, especially if SWE-Bench Pro or terminal-oriented tasks resemble your workload. They do not prove it is the best choice for your agent, repository, or API provider. Treat broader parity claims as unconfirmed unless you have comparable results for the same benchmark version and evaluation setup.
Serving results: throughput is not a quality score
Lambda’s deployment page reports these GLM-5.1-FP8 results on 1× NVIDIA HGX B200, an eight-GPU B200 system. The test used 8,192 input tokens and 2,048 output tokens per request, 512 prompts, and 32 concurrent requests. Lambda describes this as a 4:1 input-to-output workload intended to approximate a coding-agent backend.
| Serving engine | Output generation | Per-user generation | Total throughput |
|---|---|---|---|
| SGLang | 1,345.44 tok/s | 42.04 tok/s/user | 6,727.19 tok/s |
| vLLM | 1,265.45 tok/s | 39.55 tok/s/user | 6,327.24 tok/s |
The same page reports mean/P99 latency in milliseconds:
| Engine | TTFT mean / P99 | TPOT mean / P99 | ITL mean / P99 |
|---|---|---|---|
| SGLang | 1,072.61 / 14,237.85 | 22.83 / 30.60 | 58.61 / 452.50 |
| vLLM | 1,316.86 / 14,853.08 | 24.34 / 30.26 | 57.79 / 530.82 |
Under this particular test, SGLang reports higher output generation, per-user generation, and total throughput than vLLM. That is a result for the stated hardware, FP8 model, workload, and concurrency—not evidence that SGLang will always be faster, or that a managed API will match these numbers. Lambda reports TTFT, TPOT, and ITL but does not define those abbreviations in the benchmark results.
Lambda provides a command to rerun its throughput workload with vllm bench serve: 8,192 random input tokens, 2,048 random output tokens, 512 prompts, and maximum concurrency of 32. This does not reproduce the quality-score table; Lambda provides no equivalent quality-evaluation command or full protocol on that page.
What to verify before choosing a production API
A documented starting point is Z.ai’s own API. Z.ai’s developer documentation lists GLM-5.1 and documents the standard chat-completions endpoint as https://api.z.ai/api/paas/v4/chat/completions, with Bearer-token authentication. The documentation excerpt available for this review does not show a GLM-5.1 request example with the exact model field value, so confirm the currently accepted identifier in the linked model and API documentation before sending requests. Z.ai’s GLM-5.1 model documentation lists function calling, and its API guide describes passing function definitions in tools and using tool_choice: "auto".
Thinking behavior can matter when evaluating coding agents. Z.ai’s documentation states that thinking is enabled by default for GLM-5.1, and that Preserved Thinking has different defaults on the standard API and Coding Plan endpoints: disabled by default on the standard API, enabled by default on the Coding Plan endpoint. The retrieved documentation excerpts do not establish the precise standard-API request field and placement for enabling Preserved Thinking, or the follow-up-message requirements. Consult the current thinking-mode guide rather than relying on unconfirmed request instructions, and verify which endpoint and plan you will use.
These are Z.ai channel facts, not proof of Token Forge Cloud’s GLM-5.1 availability or behavior. The Token Forge Cloud material reviewed here does not establish a GLM-5.1-specific model ID, endpoint, or request behavior. Confirm those details directly with the provider if you intend to use its service. Lambda’s B200 figures are self-hosted deployment results and should not be treated as Z.ai or Token Forge Cloud hosted-API performance.
Then run a small evaluation against your own workload:
- Use representative tasks. Include the coding, reasoning, or tool-use requests your application actually sends, with realistic context lengths.
- Fix the conditions. Keep prompts, tool definitions, output limits, concurrency, and success criteria consistent across candidate models.
- Measure quality and operations separately. Check task completion and tool-call correctness alongside latency, errors, and throughput at expected traffic levels.
- Calculate cost from verified terms. Use the provider’s current price and billing unit, plus your expected input, output, and concurrency—not the model’s benchmark score.
Verdict
GLM-5.1’s reported scores support shortlisting and testing, not a blanket production recommendation. The strongest direct evidence in the reviewed material is Z.ai’s vendor-published SWE-Bench Pro comparison and Lambda’s explicitly conditioned B200 serving test. For a real selection, match benchmark evidence to your use case, test the actual API channel you plan to ship, and verify its terms and performance independently.
Sources
GLM-5.1: Towards Long-Horizon Tasks
GLM-5.3: Frontier Coding with Emergent Cyber Capabilities
GLM-5.1 - Overview - Z.AI Developer Documentation
Chat Completion - Z.AI Developer Documentation
Function Calling - Z.AI Developer Documentation