How to: Validate dynamic attributes / JSON in ActiveRecord
PostgreSQL and ActiveRecord have a good support for storing dynamic attributes (hashes) in columns of type JSONB
. But sometimes you are missing some kind of validation or lookup possibility (with plain attributes you can use Active Record's built-in validations and have your schema.rb
).
One approach about being more strict with dynamic attributes is to use JSON Schema validations. Here is an example, where a project has the dynamic attributes analytic_stats
, that we can use to store analytics from an external measurement tool.
- A good default is to start without schema versioning (this card already included a versioning for documentation purpose) and add it once you are not able to migrate the old data to a new schema.
- If you have different versions, you always need to test all different version if required. With
FactoryBot
this should be traits. - The JSON Schema files could also live somewhere in
config/some_folder
, to keep theapp/models
directory cleaner. - Check which version of JSON Schema you need. The gem
json-schema
uses draft-v4 as default, whereas the most recent version isdraft-08
. You might want to use
another gem if you need the newest version ofjson-schema
. - A good default is to use the
strict
option, which marks every attribute as required and do not allow additional attribute. If you have optional attributes, they have to be present in the dynamic attributes, but can benull
. - JSON Schema is very powerful, actually you can express if/else and case statements (a use case for this are different attributes based on an enum).
Gemfile
Copygem 'has_defaults' gem 'json-schema'
db/migrate/20190511155410_add_analytic_stats_to_projects.rb
Copyclass AddRepositoryStatsToProjects < ActiveRecord::Migration[5.2] class Project < ActiveRecord::Base end def up add_column(:projects, :analytic_stats, :jsonb) Project.reset_column_information Project.find_each do |project| project.analytic_stats = { version: 1, analytics_url: nil, clicks_this_week: 0, clicks_this_month: 0, clicks_this_year: 0, top_referrer: [], last_synchronized_at: nil, } project.save! end end def down remove_column(:projects, :analytic_stats, :jsonb) end end
app/models/project.rb
Copyclass Project < ApplicationRecord validate :validate_analytic_stats_against_json_schema has_defaults( analytic_stats: proc { { version: 1, analytics_url: nil, clicks_this_week: 0, clicks_this_month: 0, clicks_this_year: 0, top_referrer: [], last_synchronized_at: nil, } } ) def validate_analytic_stats_against_json_schema @schema ||= File.read(File.join(Rails.root, 'app', 'models', 'project', "analytic_stats_schema_v#{analytics_stats_version}.json")) analytic_stats_errors = JSON::Validator.fully_validate(@schema, analytic_stats, strict: true, validate_schema: true) analytic_stats_errors.each do |error| errors.add(:analytic_stats, error) end end def analytics_stats_version analytic_stats.fetch(:version) end end
app/models/project/analytic_stats_schema_v1.json
Copy{ "type": "object", "properties": { "version": { "type": ["number"] }, "analytics_url": { "type": ["string", "null"] }, "clicks_this_week": { "type": ["number"] }, "clicks_this_month": { "type": ["number"] }, "clicks_this_year": { "type": ["number"] }, "languages": { "type": "array", "items": { "type": "string" } }, "top_referrer": { "type": "array", "items": { "type": "string" } }, "last_synchronized_at": { "type": ["string", "null"], "format": "date-time" } } }
Your development team has a full backlog of feature requests, chores and refactoring coupled with deadlines? We are familiar with that. With our "DevOps as a Service" offering, we support developer teams with infrastructure and operations expertise.