KVCacheEstimator

Size an LLM's KV cache from its config.json (Swift): GQA-aware, head_dim-aware, fp16/fp8/int4

Written by theluckystrike.
In development for 3 hours, with 1 commit and 1 release.
MIT licensed
no stars
1 library
No executables
No plugins
No macros
No data race safety information availableWhat is data race safety and how is it tested?

Michael Lip welcomes support for KVCacheEstimator through GitHub Sponsors. If you find this package useful, please consider supporting it.

Compatibility

Full Build Results

This package currently has no compatibility information. We are currently processing a large build job backlog and it may take much longer than usual for compatibility information to appear.

You can see what builds the system is currently processing by checking the build system monitoring page.

Package ID	C9D31827-C398-474A-BE47-9FEB8F23DE4F
Score	50

Swift Package Index

When working with an Xcode project:

When working with a Swift Package Manager manifest:

Compatibility