Can you run GLM-4.6 locally?
Yes. Z.AI publishes GLM-4.6 weights and provides an SGLang launch example for local deployment. Its official repositories list the standard zai-org/GLM-4.6 checkpoint and the zai-org/GLM-4.6-FP8 checkpoint; both identify an MIT license. The FP8 checkpoint appears in Z.AI’s local-serving example.
A published FP8 serving reference from GPUStack uses 8 NVIDIA H100 SXM GPUs in one node and SGLang 0.5.3. This is a reference configuration, not a minimum requirement or a performance guarantee for another workload. SGLang’s GLM-4.6 cookbook says BF16 requires more VRAM than 8× H100 and lists FP8 as an option. The available sources do not give general minimums for VRAM, system RAM, or disk space.
Setup prerequisites and local-serving example
Z.AI recommends SGLang for local deployment and provides an FP8 launch example. Its reported example includes the checkpoint path zai-org/GLM-4.6-FP8 and tensor parallelism set to eight. The supplied documentation does not establish that the current lmsysorg/sglang:latest container matches the SGLang 0.5.3 hardware reference. Z.AI also warns that some newer SGLang versions or Docker images may cause errors or lower evaluation scores; it mentions a patch or a historical image for its evaluation setup, but the available evidence does not establish that image as the matching installation path for the FP8 reference configuration.
The following shows those model-specific launch settings only; it is not a complete, version-pinned, copy-and-run setup:
python3 -m sglang.launch_server \
--model-path zai-org/GLM-4.6-FP8 \
--tp 8
The general SGLang installation guide describes a Docker route and host prerequisites, but the available evidence does not confirm a single compatible container image, driver prerequisite, and startup command for this specific launch configuration. Confirm those details against the runtime version you select before treating the launch settings as a working deployment recipe. The available sources also do not establish universal checkpoint storage, system-memory, or per-GPU memory requirements.
Local GLM-4.6 or the Z.AI API?
| Option | When it may fit | What the documentation establishes |
|---|---|---|
| Local weights | You can provision and operate a multi-GPU inference environment. | Official weights are published, and Z.AI provides an SGLang FP8 launch example. A separate published FP8 reference uses 8× H100 SXM GPUs and SGLang 0.5.3. |
Hosted Z.AI API |
You prefer a hosted endpoint rather than deploying model weights. | Z.AI documents a chat-completions endpoint, Bearer-token authentication, and model ID glm-4.6. |
The Z.AI API overview lists text input and output, a 200K-token context length, and a 128K maximum output. Its example request sets max_tokens to 4096; that is the example’s requested value, not the listed maximum. These API figures do not establish the context or output limits available with a particular local runtime.
A basic hosted request looks like this:
curl -X POST "https://api.z.ai/api/paas/v4/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ZAI_API_KEY" \
-d '{
"model": "glm-4.6",
"messages": [{"role": "user", "content": "Summarize this text."}],
"max_tokens": 4096
}'
This is a hosted API call; it requires a Z.AI API key and network access to the endpoint. The documentation also shows streaming requests using "stream": true. Check current pricing and terms separately if cost or data handling affects your choice; the deployment sources cited here do not establish those details.
Bottom line
GLM-4.6 local deployment is supported by official weights and a provider-published SGLang launch example. A published FP8 reference used 8× H100 SXM GPUs and SGLang 0.5.3, but it is not a minimum requirement. The available information does not establish one complete, version-aligned installation-to-launch sequence for that reference. Verify runtime, image, and host-driver compatibility before deploying; if you prefer hosted access, Z.AI documents an API alternative.
Sources
Z.ai: GLM-4.6 model repository
Z.ai: GLM-4.6-FP8 model repository
Z.ai: GLM-4.6 evaluation and local deployment guide