Useful jq commands

jq is great when dealing with JSON data. Check out the official jq Manual Show archive.org snapshot .

Content

• Select only specific keys
• Modify keys
• Set default value
• Filter by key name
• Delete specific keys
• Overwrite/add nested key value

Select only specific keys

Command

jq '.[] | { Name, OriginString }'

Input

[
  {
    "Id": 1,
    "Name": "foobar.net",
    "OriginString": "/FOOBAR/foobar.net"
  },
  {
    "Id": 2,
    "Name": "barfoo.baz",
    "OriginString": "/FOOBAR/barfoo.baz"
  }
]

Output

{
  "Name": "foobar.net",
  "OriginString": "/FOOBAR/foobar.net"
}
{
  "Name": "barfoo.baz",
  "OriginString": "/FOOBAR/barfoo.baz"
}

Modify keys

Command

jq '.[] | { Name, OriginString, Number: .Id, Static: "Foobar" }'

Input

[
  {
    "Id": 1,
    "Name": "foobar.net",
    "OriginString": "/FOOBAR/foobar.net"
  },
  {
    "Id": 2,
    "Name": "barfoo.baz",
    "OriginString": "/FOOBAR/barfoo.baz"
  }
]

Output

{
  "Name": "foobar.net",
  "OriginString": "/FOOBAR/foobar.net",
  "Number": 1,
  "Static": "Foobar"
}
{
  "Name": "barfoo.baz",
  "OriginString": "/FOOBAR/barfoo.baz",
  "Number": 2,
  "Static": "Foobar"
}

Set default value

Command

jq '.[] | .Namo // "unkown", .Id'

Input

[
  {
    "Id": 1,
    "Name": "foobar.net",
    "OriginString": "/FOOBAR/foobar.net"
  },
  {
    "Id": 2,
    "Name": "barfoo.baz",
    "OriginString": "/FOOBAR/barfoo.baz"
  }
]

Output

"unkown"
1
"unkown"
2

Filter by key name

Command

jq 'to_entries | map(select(.key | contains("TIMESTAMP"))) | from_entries'

# Examples
sudo journalctl -o json -n 10 |  jq 'to_entries | map(select(.key | contains("TIMESTAMP"))) | from_entries'
sudo journalctl -o json -n 10 |  jq 'to_entries | map(select(.key | startswith("_"))) | from_entries'

Input

{
  "_SYSTEMD_CGROUP": "/system.slice/cron.service",
  "_HOSTNAME": "app01-prod",
  "_SELINUX_CONTEXT": "unconfined\n",
  "SYSLOG_FACILITY": "9",
  "SYSLOG_PID": "350655",
  "_PID": "350655",
  "__REALTIME_TIMESTAMP": "1708813921675267",
  "_AUDIT_LOGINUID": "21220",
  "_AUDIT_SESSION": "222968",
  "_SYSTEMD_INVOCATION_ID": "0985986d011a47e9ae98b5d2162a2216",
  "_SYSTEMD_UNIT": "cron.service",
  "_MACHINE_ID": "b892b6d3b3194583a79fb77327217d04",
  "_BOOT_ID": "db8f42320fb9450f8fbd499e6ba3d587",
  "PRIORITY": "6",
  "__MONOTONIC_TIMESTAMP": "6274590235728",
  "_TRANSPORT": "syslog",
  "_GID": "21220",
  "MESSAGE": "(foobar) LIST (foobar)",
  "_UID": "21220",
  "_SOURCE_REALTIME_TIMESTAMP": "1708813921674658",
  "SYSLOG_IDENTIFIER": "crontab",
  "_CAP_EFFECTIVE": "0",
  "_COMM": "crontab",
  "SYSLOG_TIMESTAMP": "Feb 24 23:32:01 ",
  "__CURSOR": "s=107a823f117a4bf2ab396889c1c2cd9f;i=4804a8;b=db8f42320fb9450f8fbd499e6ba3d587;m=5b4eab8dc50;t=612283ec9e803;x=86d130329516f86d",
  "_SYSTEMD_SLICE": "system.slice"
}
{
  "_UID": "21220",
  "_SOURCE_REALTIME_TIMESTAMP": "1708900321625410",
  "SYSLOG_FACILITY": "9",
  "__REALTIME_TIMESTAMP": "1708900321638757",
  "__MONOTONIC_TIMESTAMP": "6360990199217",
  "_HOSTNAME": "app01-prod",
  "__CURSOR": "s=107a823f117a4bf2ab396889c1c2cd9f;i=486868;b=db8f42320fb9450f8fbd499e6ba3d587;m=5c9088fadb1;t=6123c5ca0b965;x=951c6306e6999356",
  "_MACHINE_ID": "b892b6d3b3194583a79fb77327217d04",
  "MESSAGE": "(foobar) LIST (foobar)",
  "_BOOT_ID": "db8f42320fb9450f8fbd499e6ba3d587",
  "SYSLOG_PID": "1046516",
  "_GID": "21220",
  "_TRANSPORT": "syslog",
  "PRIORITY": "6",
  "_PID": "1046516",
  "SYSLOG_TIMESTAMP": "Feb 25 23:32:01 ",
  "SYSLOG_IDENTIFIER": "crontab"
}

Ouput

{
  "_SOURCE_REALTIME_TIMESTAMP": "1708813921674658",
  "__MONOTONIC_TIMESTAMP": "6274590235728",
  "SYSLOG_TIMESTAMP": "Feb 24 23:32:01 ",
  "__REALTIME_TIMESTAMP": "1708813921675267"
}
{
  "__REALTIME_TIMESTAMP": "1708900321638757",
  "_SOURCE_REALTIME_TIMESTAMP": "1708900321625410",
  "__MONOTONIC_TIMESTAMP": "6360990199217",
  "SYSLOG_TIMESTAMP": "Feb 25 23:32:01 "
}

Delete specific keys

Command

jq 'del(.rules[].created_at, .rules[].updated_at, .rules[].id, .rules[].name, .rules[]."@id", .rules[]."@type")'

Input

{
  "platform": "http_large",
  "rules": [
    {
      "@id": "/rules-engine/v1.1/policies/1234567/rules/98765432",
      "@type": "Rule",
      "id": "1337",
      "name": "",
      "description": "Block Client IP",
      "ordinal": 1,
      "created_at": "2024-03-13T16:43:45Z",
      "updated_at": "2024-03-13T16:43:45Z",
      "matches": [
        {
          "type": "match.request.client-ip-address.literal",
          "ordinal": 1,
          "result": "match",
          "value": [
            "127.0.0.1/32"
          ],
          "features": [
            {
              "type": "feature.access.deny-access",
              "ordinal": 1,
              "enabled": "true"
            }
          ]
        }
      ]
    }
  ]
}

Output

{
  "platform": "http_large",
  "rules": [
    {
      "description": "Block Client IP",
      "ordinal": 1,
      "matches": [
        {
          "type": "match.request.client-ip-address.literal",
          "ordinal": 1,
          "result": "match",
          "value": [
            "127.0.0.1/32"
          ],
          "features": [
            {
              "type": "feature.access.deny-access",
              "ordinal": 1,
              "enabled": "true"
            }
          ]
        }
      ]
    }
  ]
}

Overwrite/add nested key value

If the key exists it will overwrite it's value, if it doesn't exist it will add the key with the given value

Command

jq '.DistributionConfig.Origins.Items[].OriginAccessControlId=""'
# Example: aws cloudfront get-distribution-config --id "$cloudfront_distribution_ID" | jq '.DistributionConfig.Origins.Items[].OriginAccessControlId=""'

Input

This is only part of a AWS Cloudfront config

{
    "ETag": "BBTM0OSB52L2YY",
    "DistributionConfig": {
        "Origins": {
            "Quantity": 1,
            "Items": [
                {
                    "Id": "name-of-my-s3-origin",
                    "DomainName": "bucket-name-example.s3.eu-central-1.amazonaws.com",
                    "OriginAccessControlId": "BQLJS0DMIOW6VJ"
                }
            ]
        }
    }
}

Output

{
  "ETag": "BBTM0OSB52L2YY",
  "DistributionConfig": {
    "Origins": {
      "Quantity": 1,
      "Items": [
        {
          "Id": "name-of-my-s3-origin",
          "DomainName": "bucket-name-example.s3.eu-central-1.amazonaws.com",
          "OriginAccessControlId": ""
        }
      ]
    }
  }
}