1/
đ§”diving deeper into our new work on zkGPT: Proving LLM inference fast with Zero-Knowledge Proofs.
Why? Service providers might deploy a smaller/cheaper model than promised. ZK lets them prove correctness without revealing model parameters.
đ
2/
The Problem:
- LLMs = powerful but costly.
- Providers could cheat by running smaller models.
- Users canât verify which model was used.
ZK Proofs solve this, but current zkML systems choke on real LLMs:
- No support for transformer architectures.
- Huge proving times (minutesâhours).
3/
Previous work:
- ZKML (Eurosysâ24): General ML verification framework. Good for small models, but too slow for LLMs.
- Hao et al. (USENIX Securityâ24): Early zkLLM attempt, still pretty slow (thousands of seconds).
- Both suffer from massive nonlinear layer overhead + poor parallelization.
4/
Our contributions:
1. Efficient proofs for linear & nonlinear layers tailored to LLMs (e.g., GPT-2).
2. Constraint fusion â reduce overhead in nonlinear layers (like GeLU).
3. Circuit squeeze â boosts parallelism in proof generation.
4. Full-stack implementation optimized for transformer blocks.
5/
Results:
- Proves GPT-2 inference in <25 seconds.
- 279Ă faster than Hao et al. (USENIX'24).
- 185Ă faster than ZKML (Eurosys'24).
- Orders-of-magnitude less overhead than naive zk-transformer implementations.
6/
Why it matters:
- Enables practical zkLLM deployment â you can now verify an LLMâs output in seconds.
- Keeps model weights secret.
- Opens doors for privacy-preserving AI services with cryptographic auditability.
7/
Key insight:
Donât just naively compile an LLM into a circuit.
Exploit structure:
- Linear ops (MatMul, LayerNorm) â custom efficient constraints.
- Nonlinear ops (GELU) â fused constraints to slash complexity.
- Parallel-friendly layout to max out modern prover hardware.
23.89K
537
The content on this page is provided by third parties. Unless otherwise stated, OKX is not the author of the cited article(s) and does not claim any copyright in the materials. The content is provided for informational purposes only and does not represent the views of OKX. It is not intended to be an endorsement of any kind and should not be considered investment advice or a solicitation to buy or sell digital assets. To the extent generative AI is utilized to provide summaries or other information, such AI generated content may be inaccurate or inconsistent. Please read the linked article for more details and information. OKX is not responsible for content hosted on third party sites. Digital asset holdings, including stablecoins and NFTs, involve a high degree of risk and can fluctuate greatly. You should carefully consider whether trading or holding digital assets is suitable for you in light of your financial condition.