In-memory caching library

Otter is designed to provide an excellent developer experience while maintaining blazing-fast performance. It aims to address the shortcomings of its predecessors and incorporates design principles from high-performance libraries in other languages (such as Caffeine).

Features¶

Performance-wise, Otter provides:

• High hit rate: Top-tier hit rates across all workload types via adaptive W-TinyLFU
• Blazing fast: Excellent throughput under high contention on most workload types
• Low memory overhead: Among the lowest memory overheads across all cache capacities
• Self-tuning: Automatic data structures configuration based on contention/parallelism and workload patterns

Otter also provides a highly configurable caching API, enabling any combination of these optional features:

• Eviction: Size-based eviction when a maximum is exceeded
• Expiration: Time-based expiration of entries (using Hierarchical Timing Wheel), measured since last access or last write
• Loading: Automatic loading of entries into the cache
• Refresh: Asynchronously refresh when the first stale request for an entry occurs
• Stats: Accumulation of cache access statistics