Sailfish

Retention and rollups

Seven days of five-minute snapshots, ninety of hourly ones, then daily ones for good.

A snapshot every five minutes is 288 a day. sailfish:rollup runs daily at 03:00 and keeps that from growing with every collection:

GranularityKept forThen
Raw (every five minutes)7 daysMerged to one per hour
Hourly90 daysMerged to one per day
DailyFor good

How snapshots are merged#

An hour's snapshots are merged into the last of them. Counters add up, the digest quantiles (p95_us, p99_us) and the server's gauges keep their highest value, and the merged snapshot covers the seconds of all of them. Daily table sizes are kept as they are.

Charts and detectors read a window the same way whatever its granularity, as what moved over the seconds covered. A chart reaching back into daily snapshots draws daily buckets. Re-running the rollup is safe: nothing is counted twice.

Dropped tables, old digests, cache key groups, long transactions and settings#

Dropped tables and indexes are kept, history and all, so you can still see what an index was doing the week before someone dropped it. Statement digests not seen in 90 days are deleted with their samples. One that runs again later starts over with a baseline. Groups of cache keys not used in 90 days are deleted with their samples too, unless they still hold entries; their events are kept, without a link to the group. Long transactions are kept for 30 days after they ended. The committed and rolled-back counts are samples like any other, and are rolled up with them. Setting changes are kept for good: a server sees a few a month, and the one that matters may be a year old.

Changing the windows#

// config/sailfish.php
'retention' => [
    'raw_days' => 7,
    'hourly_days' => 90,
    'digest_days' => 90,
    'cache_key_days' => 90,
    'long_transaction_days' => 30,
],

Set any of them to null to keep that granularity, those digests, those groups or those transactions forever. The rollup's time is SAILFISH_ROLLUP_AT.

Install Sailfish today.

Checkout ends with your license key, and the installation guide takes it from there.