Elasticsearch defaults to go into readonly mode when you run low on disk space (< 95%). You might then see an error like this when you try to write to elastic:
Elasticsearch::Transport::Transport::Errors::Forbidden:
[403] {"error":{"root_cause":[{"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}],"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"},"status":403}
Even after freeing up space and restarting elasticsearch, elastic did not recover. I had to explicitly disable readonly via the following REST call:
curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'
Posted by Daniel Straßner to makandra dev (2021-11-24 06:05)