ml0x welcomes support for KVCacheEstimator through GitHub Sponsors. If you find this package useful, please consider supporting it.
When working with an Xcode project:
When working with a Swift Package Manager manifest:
Select a package version:
0.1.0
main
Size an LLM's KV cache from its config.json (Swift): GQA-aware, head_dim-aware, fp16/fp8/int4