jq is great when dealing with JSON data. Check out the official jq Manual Show archive.org snapshot .

Content

• Select only specific keys
• Modify keys
• Set default value
• Filter by key name
• Delete specific keys
• Overwrite/add nested key value

Select only specific keys

Command

jq '.[] | { Name, OriginString }'

Input

[
  {
    "Id": 1,
    "Name": "foobar.net",
    "OriginString": "/FOOBAR/foobar.net"
  },
  {
    "Id": 2,
    "Name": "barfoo.baz",
    "OriginString": "/FOOBAR/barfoo.baz"
  }
]

Output

{
  "Name": "foobar.net",
  "OriginString": "/FOOBAR/foobar.net"
}
{
  "Name": "barfoo.baz",
  "OriginString": "/FOOBAR/barfoo.baz"
}

Modify keys

Command

jq '.[] | { Name, OriginString, Number: .Id, Static: "Foobar" }'

Input

[
  {
    "Id": 1,
    "Name": "foobar.net",
    "OriginString": "/FOOBAR/foobar.net"
  },
  {
    "Id": 2,
    "Name": "barfoo.baz",
    "OriginString": "/FOOBAR/barfoo.baz"
  }
]

Output

{
  "Name": "foobar.net",
  "OriginString": "/FOOBAR/foobar.net",
  "Number": 1,
  "Static": "Foobar"
}
{
  "Name": "barfoo.baz",
  "OriginString": "/FOOBAR/barfoo.baz",
  "Number": 2,
  "Static": "Foobar"
}

Set default value

Command

jq '.[] | .Namo // "unkown", .Id'

Input

[
  {
    "Id": 1,
    "Name": "foobar.net",
    "OriginString": "/FOOBAR/foobar.net"
  },
  {
    "Id": 2,
    "Name": "barfoo.baz",
    "OriginString": "/FOOBAR/barfoo.baz"
  }
]

Output

"unkown"
1
"unkown"
2

Filter by key name

Command

jq 'to_entries | map(select(.key | contains("TIMESTAMP"))) | from_entries'

# Examples
sudo journalctl -o json -n 10 |  jq 'to_entries | map(select(.key | contains("TIMESTAMP"))) | from_entries'
sudo journalctl -o json -n 10 |  jq 'to_entries | map(select(.key | startswith("_"))) | from_entries'

Input

{
  "_SYSTEMD_CGROUP": "/system.slice/cron.service",
  "_HOSTNAME": "app01-prod",
  "_SELINUX_CONTEXT": "unconfined\n",
  "SYSLOG_FACILITY": "9",
  "SYSLOG_PID": "350655",
  "_PID": "350655",
  "__REALTIME_TIMESTAMP": "1708813921675267",
  "_AUDIT_LOGINUID": "21220",
  "_AUDIT_SESSION": "222968",
  "_SYSTEMD_INVOCATION_ID": "0985986d011a47e9ae98b5d2162a2216",
  "_SYSTEMD_UNIT": "cron.service",
  "_MACHINE_ID": "b892b6d3b3194583a79fb77327217d04",
  "_BOOT_ID": "db8f42320fb9450f8fbd499e6ba3d587",
  "PRIORITY": "6",
  "__MONOTONIC_TIMESTAMP": "6274590235728",
  "_TRANSPORT": "syslog",
  "_GID": "21220",
  "MESSAGE": "(foobar) LIST (foobar)",
  "_UID": "21220",
  "_SOURCE_REALTIME_TIMESTAMP": "1708813921674658",
  "SYSLOG_IDENTIFIER": "crontab",
  "_CAP_EFFECTIVE": "0",
  "_COMM": "crontab",
  "SYSLOG_TIMESTAMP": "Feb 24 23:32:01 ",
  "__CURSOR": "s=107a823f117a4bf2ab396889c1c2cd9f;i=4804a8;b=db8f42320fb9450f8fbd499e6ba3d587;m=5b4eab8dc50;t=612283ec9e803;x=86d130329516f86d",
  "_SYSTEMD_SLICE": "system.slice"
}
{
  "_UID": "21220",
  "_SOURCE_REALTIME_TIMESTAMP": "1708900321625410",
  "SYSLOG_FACILITY": "9",
  "__REALTIME_TIMESTAMP": "1708900321638757",
  "__MONOTONIC_TIMESTAMP": "6360990199217",
  "_HOSTNAME": "app01-prod",
  "__CURSOR": "s=107a823f117a4bf2ab396889c1c2cd9f;i=486868;b=db8f42320fb9450f8fbd499e6ba3d587;m=5c9088fadb1;t=6123c5ca0b965;x=951c6306e6999356",
  "_MACHINE_ID": "b892b6d3b3194583a79fb77327217d04",
  "MESSAGE": "(foobar) LIST (foobar)",
  "_BOOT_ID": "db8f42320fb9450f8fbd499e6ba3d587",
  "SYSLOG_PID": "1046516",
  "_GID": "21220",
  "_TRANSPORT": "syslog",
  "PRIORITY": "6",
  "_PID": "1046516",
  "SYSLOG_TIMESTAMP": "Feb 25 23:32:01 ",
  "SYSLOG_IDENTIFIER": "crontab"
}

Ouput

{
  "_SOURCE_REALTIME_TIMESTAMP": "1708813921674658",
  "__MONOTONIC_TIMESTAMP": "6274590235728",
  "SYSLOG_TIMESTAMP": "Feb 24 23:32:01 ",
  "__REALTIME_TIMESTAMP": "1708813921675267"
}
{
  "__REALTIME_TIMESTAMP": "1708900321638757",
  "_SOURCE_REALTIME_TIMESTAMP": "1708900321625410",
  "__MONOTONIC_TIMESTAMP": "6360990199217",
  "SYSLOG_TIMESTAMP": "Feb 25 23:32:01 "
}

Delete specific keys

Command

jq 'del(.rules[].created_at, .rules[].updated_at, .rules[].id, .rules[].name, .rules[]."@id", .rules[]."@type")'

Input

{
  "platform": "http_large",
  "rules": [
    {
      "@id": "/rules-engine/v1.1/policies/1234567/rules/98765432",
      "@type": "Rule",
      "id": "1337",
      "name": "",
      "description": "Block Client IP",
      "ordinal": 1,
      "created_at": "2024-03-13T16:43:45Z",
      "updated_at": "2024-03-13T16:43:45Z",
      "matches": [
        {
          "type": "match.request.client-ip-address.literal",
          "ordinal": 1,
          "result": "match",
          "value": [
            "127.0.0.1/32"
          ],
          "features": [
            {
              "type": "feature.access.deny-access",
              "ordinal": 1,
              "enabled": "true"
            }
          ]
        }
      ]
    }
  ]
}

Output

{
  "platform": "http_large",
  "rules": [
    {
      "description": "Block Client IP",
      "ordinal": 1,
      "matches": [
        {
          "type": "match.request.client-ip-address.literal",
          "ordinal": 1,
          "result": "match",
          "value": [
            "127.0.0.1/32"
          ],
          "features": [
            {
              "type": "feature.access.deny-access",
              "ordinal": 1,
              "enabled": "true"
            }
          ]
        }
      ]
    }
  ]
}

Overwrite/add nested key value

Rails Long Term Support

Rails LTS provides security patches for old versions of Ruby on Rails (2.3, 3.2, 4.2 and 5.2)

• Prevents you from data breaches and liability risks
• Upgrade at your own pace
• Works with modern Rubies

Read more Show archive.org snapshot

If the key exists it will overwrite it's value, if it doesn't exist it will add the key with the given value

Command

jq '.DistributionConfig.Origins.Items[].OriginAccessControlId=""'
# Example: aws cloudfront get-distribution-config --id "$cloudfront_distribution_ID" | jq '.DistributionConfig.Origins.Items[].OriginAccessControlId=""'

Input

This is only part of a AWS Cloudfront config

{
    "ETag": "BBTM0OSB52L2YY",
    "DistributionConfig": {
        "Origins": {
            "Quantity": 1,
            "Items": [
                {
                    "Id": "name-of-my-s3-origin",
                    "DomainName": "bucket-name-example.s3.eu-central-1.amazonaws.com",
                    "OriginAccessControlId": "BQLJS0DMIOW6VJ"
                }
            ]
        }
    }
}

Output

{
  "ETag": "BBTM0OSB52L2YY",
  "DistributionConfig": {
    "Origins": {
      "Quantity": 1,
      "Items": [
        {
          "Id": "name-of-my-s3-origin",
          "DomainName": "bucket-name-example.s3.eu-central-1.amazonaws.com",
          "OriginAccessControlId": ""
        }
      ]
    }
  }
}