Posted 10 days ago. Visible to the public.

Elasticsearch: recover from readonly mode

Elasticsearch defaults to go into readonly mode when you run low on disk space (< 95%). You might then see an error like this when you try to write to elastic:

Copy
Elasticsearch::Transport::Transport::Errors::Forbidden: [403] {"error":{"root_cause":[{"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}],"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"},"status":403}

Even after freeing up space and restarting elasticsearch, elastic did not recover. I had to explicitly disable readonly via the following REST call:

Copy
curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

By refactoring problematic code and creating automated tests, makandra can vastly improve the maintainability of your Rails application.

Owner of this card:

Avatar
Daniel Straßner
Last edit:
10 days ago
by Daniel Straßner
About this deck:
We are makandra and do test-driven, agile Ruby on Rails software development.
License for source code
Posted by Daniel Straßner to makandra dev
This website uses short-lived cookies to improve usability.
Accept or learn more