PSA: When Redis (on LRU noeviction) reaches memory limits, keys with (any) expiry are removed, and keys without are not

Posted . Visible to the public.

In production, you (or your ops team) should configure a maximum number of bytes that Redis can use to store data (maxmemory setting).
They may even want to set maxmemory-policy to noeviction to avoid keys being removed that you want to keep around (e.g. for Sidekiq queues!).

The following applies to the noeviction LRU approach.


When you ask Redis to store a value that would exceed this limit, it tries to clean up:

  • Keys for which an expiry value (TTL) was set (explicitly, or when written via SETEX) will be removed.
    If necessary, this happens even when that expiration timestamp is in the future.

  • Keys without a TTL (that were just written via SET) will never be removed unless you DEL them explicitly.

  • If there are no keys that can be removed, you will see this error:

    OOM command not allowed when used memory > 'maxmemory'
    

tl;dr

Generally speaking, when you write to Redis, use a TTL for anything that can be reproduced (like cached values).
If you store data that you want to keep for 1 year or so, you must keep in mind that this data may disappear sooner.

When storing data without any TTL, you need to make sure that data is eventually cleaned up by you.

Arne Hartherz
Last edit
Arne Hartherz
License
Source code in this card is licensed under the MIT License.
Posted by Arne Hartherz to makandra dev (2016-06-13 07:51)