What is the memory sizing recommendation for using AIStor Cache?

AIStor Cache is an optional in-memory metadata caching feature that improves read performance by reducing disk I/O for object metadata lookups. Understanding the memory requirements helps you properly size your nodes.

Answer

How AIStor Cache Works

AIStor Cache uses BigCache^[1] (a high-performance, in-memory key-value (KV) store library written in Go) to cache xl.meta metadata for objects on each drive. When enabled:

1. Metadata reads check the cache first before accessing disk
2. Cache hits return metadata directly from RAM (microseconds vs milliseconds)
3. Cache misses read from disk and populate the cache for future requests
4. Cache is per-drive - each local drive gets its own cache allocation

Cache Flow:

What gets cached:

• Object metadata (xl.meta files)
• NOT object data itself

Benefits:

• Reduced disk I/O for metadata-heavy workloads
• Lower latency for object listings and HEAD operations
• Improved performance for small object workloads with frequent metadata access

Memory Allocation Formula

When AIStor Cache is enabled, memory is allocated as follows:

Cache Memory per Drive = (Total Cache Memory) / (Number of Local Drives)

Default behavior (automatic):^[2]

Total Cache Memory = Node Memory × 25%

Example with 256 GiB RAM and 16 drives:

Total Cache Memory = 256 GiB × 0.25 = 64 GiB
Cache per Drive = 64 GiB / 16 drives = 4 GiB per drive

Configuration

Enable cache with automatic sizing:

export MINIO_CACHE_ENABLE=on

Enable cache with custom size:

export MINIO_CACHE_ENABLE=on
export MINIO_CACHE_SIZE=32GiB

Note: If MINIO_CACHE_SIZE exceeds 25% of node memory, AIStor ignores the specified value and uses the automatic calculation instead.

Total Node Memory Calculation

To calculate total memory requirements per node, consider these components:

Concurrent connections formula:

memory_limit = Available Memory × 75%
max_concurrent_requests = memory_limit ÷ 2 MiB

Where 2 MiB is twice the default erasure block size (1 MiB).^[4][5]

Concurrent request capacity by RAM:

Memory Sizing Recommendations

Without AIStor Cache

With AIStor Cache Enabled

Add 25% more memory to support the cache allocation:

MinIO’s general recommendation: At least 256 GiB of RAM per host for production deployments.

When to Enable AIStor Cache

Enable cache when:

• Workload is metadata-heavy (frequent listings, HEAD requests)
• Many small objects with repeated access patterns
• Sufficient RAM available beyond concurrent connection needs
• Disk latency is a bottleneck for metadata operations

Keep cache disabled when:

• Memory is constrained
• Workload is primarily large sequential reads/writes
• Objects are accessed infrequently (cold storage)

Monitoring Cache Effectiveness

Use these v3 metrics^[6] to monitor cache performance:

# Cache hit rate
minio_system_drive_cache_hits / (minio_system_drive_cache_hits + minio_system_drive_cache_misses)

# Cache utilization
minio_system_drive_cache_used / minio_system_drive_cache_capacity

# Cache collisions (evictions due to hash conflicts)
rate(minio_system_drive_cache_collisions[5m])

Target metrics:

• Hit rate > 80% indicates effective caching
• Utilization near 100% may indicate cache is undersized
• High collision rate suggests cache size should be increased

Example: Sizing a Production Node

Scenario:

• 16 NVMe drives per node
• Expected 50,000 concurrent connections
• Want to enable AIStor Cache

Step 1: Calculate memory for concurrent connections

Required for connections = 50,000 × 2 MiB = 100 GiB
At 75% allocation: 100 GiB / 0.75 = 133 GiB minimum

Step 2: Add cache overhead (25%)

With cache: 133 GiB / 0.75 = 177 GiB

Step 3: Add base process allocation

Total: 177 GiB + 2 GiB = 179 GiB minimum

Recommendation: Use 256 GiB RAM for headroom and OS requirements.

Cache allocation at 256 GiB:

Total cache: 256 GiB × 0.25 = 64 GiB
Per drive: 64 GiB / 16 = 4 GiB per drive

Summary

For detailed capacity planning assistance, contact your MinIO account team.