How to: Validate dynamic attributes / JSON in ActiveRecord

PostgreSQL and ActiveRecord have a good support for storing dynamic attributes (hashes) in columns of type JSONB. But sometimes you are missing some kind of validation or lookup possibility (with plain attributes you can use Active Record's built-in validations and have your schema.rb).

One approach about being more strict with dynamic attributes is to use JSON Schema validations. Here is an example, where a project has the dynamic attributes analytic_stats, that we can use to store analytics from an external measurement tool.

• A good default is to start without schema versioning (this card already included a versioning for documentation purpose) and add it once you are not able to migrate the old data to a new schema.
• If you have different versions, you always need to test all different version if required. With FactoryBot this should be traits.
• The JSON Schema files could also live somewhere in config/some_folder, to keep the app/models directory cleaner.
• Check which version of JSON Schema you need. The gem json-schema uses draft-v4 as default, whereas the most recent version is draft-08. You might want to use
  another gem if you need the newest version of json-schema.
• A good default is to use the strict option, which marks every attribute as required and do not allow additional attribute. If you have optional attributes, they have to be present in the dynamic attributes, but can be null.
• JSON Schema is very powerful, actually you can express if/else and case statements (a use case for this are different attributes based on an enum).

Gemfile

Copy
gem 'has_defaults'
gem 'json-schema'

db/migrate/20190511155410_add_analytic_stats_to_projects.rb

Copy
class AddRepositoryStatsToProjects < ActiveRecord::Migration[5.2]
  class Project < ActiveRecord::Base
  end

  def up
    add_column(:projects, :analytic_stats, :jsonb)

    Project.reset_column_information
    Project.find_each do |project|
      project.analytic_stats = {
        version: 1,
        analytics_url: nil,
        clicks_this_week: 0,
        clicks_this_month: 0,
        clicks_this_year: 0,
        top_referrer: [],
        last_synchronized_at: nil,
      }

      project.save!
    end
  end

  def down
    remove_column(:projects, :analytic_stats, :jsonb)
  end
end

app/models/project.rb

Copy
class Project < ApplicationRecord
  validate :validate_analytic_stats_against_json_schema

  has_defaults(
    analytic_stats: proc {
      {
        version: 1,
        analytics_url: nil,
        clicks_this_week: 0,
        clicks_this_month: 0,
        clicks_this_year: 0,
        top_referrer: [],
        last_synchronized_at: nil,
      }
    }
  )

  def validate_analytic_stats_against_json_schema
    @schema ||= File.read(File.join(Rails.root, 'app', 'models', 'project', "analytic_stats_schema_v#{analytics_stats_version}.json"))
    analytic_stats_errors = JSON::Validator.fully_validate(@schema, analytic_stats, strict: true, validate_schema: true)

    analytic_stats_errors.each do |error|
      errors.add(:analytic_stats, error)
    end
  end

  def analytics_stats_version
    analytic_stats.fetch(:version)
  end

end

app/models/project/analytic_stats_schema_v1.json

Copy
{
  "type": "object",
  "properties": {
    "version": {
      "type": ["number"]
    },
    "analytics_url": {
      "type": ["string", "null"]
    },
    "clicks_this_week": {
      "type": ["number"]
    },
    "clicks_this_month": {
      "type": ["number"]
    },
    "clicks_this_year": {
      "type": ["number"]
    },
    "languages": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "top_referrer": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "last_synchronized_at": {
      "type": ["string", "null"],
      "format": "date-time"
    }
  }
}