jq is great when dealing with JSON data. Check out the official
jq Manual
Show archive.org snapshot
.
Content
- Select only specific keys
- Modify keys
- Set default value
- Filter by key name
- Delete specific keys
- Overwrite/add nested key value
Select only specific keys
Command
jq '.[] | { Name, OriginString }'
Input
[
{
"Id": 1,
"Name": "foobar.net",
"OriginString": "/FOOBAR/foobar.net"
},
{
"Id": 2,
"Name": "barfoo.baz",
"OriginString": "/FOOBAR/barfoo.baz"
}
]
Output
{
"Name": "foobar.net",
"OriginString": "/FOOBAR/foobar.net"
}
{
"Name": "barfoo.baz",
"OriginString": "/FOOBAR/barfoo.baz"
}
Modify keys
Command
jq '.[] | { Name, OriginString, Number: .Id, Static: "Foobar" }'
Input
[
{
"Id": 1,
"Name": "foobar.net",
"OriginString": "/FOOBAR/foobar.net"
},
{
"Id": 2,
"Name": "barfoo.baz",
"OriginString": "/FOOBAR/barfoo.baz"
}
]
Output
{
"Name": "foobar.net",
"OriginString": "/FOOBAR/foobar.net",
"Number": 1,
"Static": "Foobar"
}
{
"Name": "barfoo.baz",
"OriginString": "/FOOBAR/barfoo.baz",
"Number": 2,
"Static": "Foobar"
}
Set default value
Command
jq '.[] | .Namo // "unkown", .Id'
Input
[
{
"Id": 1,
"Name": "foobar.net",
"OriginString": "/FOOBAR/foobar.net"
},
{
"Id": 2,
"Name": "barfoo.baz",
"OriginString": "/FOOBAR/barfoo.baz"
}
]
Output
"unkown"
1
"unkown"
2
Filter by key name
Command
jq 'to_entries | map(select(.key | contains("TIMESTAMP"))) | from_entries'
# Examples
sudo journalctl -o json -n 10 | jq 'to_entries | map(select(.key | contains("TIMESTAMP"))) | from_entries'
sudo journalctl -o json -n 10 | jq 'to_entries | map(select(.key | startswith("_"))) | from_entries'
Input
{
"_SYSTEMD_CGROUP": "/system.slice/cron.service",
"_HOSTNAME": "app01-prod",
"_SELINUX_CONTEXT": "unconfined\n",
"SYSLOG_FACILITY": "9",
"SYSLOG_PID": "350655",
"_PID": "350655",
"__REALTIME_TIMESTAMP": "1708813921675267",
"_AUDIT_LOGINUID": "21220",
"_AUDIT_SESSION": "222968",
"_SYSTEMD_INVOCATION_ID": "0985986d011a47e9ae98b5d2162a2216",
"_SYSTEMD_UNIT": "cron.service",
"_MACHINE_ID": "b892b6d3b3194583a79fb77327217d04",
"_BOOT_ID": "db8f42320fb9450f8fbd499e6ba3d587",
"PRIORITY": "6",
"__MONOTONIC_TIMESTAMP": "6274590235728",
"_TRANSPORT": "syslog",
"_GID": "21220",
"MESSAGE": "(foobar) LIST (foobar)",
"_UID": "21220",
"_SOURCE_REALTIME_TIMESTAMP": "1708813921674658",
"SYSLOG_IDENTIFIER": "crontab",
"_CAP_EFFECTIVE": "0",
"_COMM": "crontab",
"SYSLOG_TIMESTAMP": "Feb 24 23:32:01 ",
"__CURSOR": "s=107a823f117a4bf2ab396889c1c2cd9f;i=4804a8;b=db8f42320fb9450f8fbd499e6ba3d587;m=5b4eab8dc50;t=612283ec9e803;x=86d130329516f86d",
"_SYSTEMD_SLICE": "system.slice"
}
{
"_UID": "21220",
"_SOURCE_REALTIME_TIMESTAMP": "1708900321625410",
"SYSLOG_FACILITY": "9",
"__REALTIME_TIMESTAMP": "1708900321638757",
"__MONOTONIC_TIMESTAMP": "6360990199217",
"_HOSTNAME": "app01-prod",
"__CURSOR": "s=107a823f117a4bf2ab396889c1c2cd9f;i=486868;b=db8f42320fb9450f8fbd499e6ba3d587;m=5c9088fadb1;t=6123c5ca0b965;x=951c6306e6999356",
"_MACHINE_ID": "b892b6d3b3194583a79fb77327217d04",
"MESSAGE": "(foobar) LIST (foobar)",
"_BOOT_ID": "db8f42320fb9450f8fbd499e6ba3d587",
"SYSLOG_PID": "1046516",
"_GID": "21220",
"_TRANSPORT": "syslog",
"PRIORITY": "6",
"_PID": "1046516",
"SYSLOG_TIMESTAMP": "Feb 25 23:32:01 ",
"SYSLOG_IDENTIFIER": "crontab"
}
Ouput
{
"_SOURCE_REALTIME_TIMESTAMP": "1708813921674658",
"__MONOTONIC_TIMESTAMP": "6274590235728",
"SYSLOG_TIMESTAMP": "Feb 24 23:32:01 ",
"__REALTIME_TIMESTAMP": "1708813921675267"
}
{
"__REALTIME_TIMESTAMP": "1708900321638757",
"_SOURCE_REALTIME_TIMESTAMP": "1708900321625410",
"__MONOTONIC_TIMESTAMP": "6360990199217",
"SYSLOG_TIMESTAMP": "Feb 25 23:32:01 "
}
Delete specific keys
Command
jq 'del(.rules[].created_at, .rules[].updated_at, .rules[].id, .rules[].name, .rules[]."@id", .rules[]."@type")'
Input
{
"platform": "http_large",
"rules": [
{
"@id": "/rules-engine/v1.1/policies/1234567/rules/98765432",
"@type": "Rule",
"id": "1337",
"name": "",
"description": "Block Client IP",
"ordinal": 1,
"created_at": "2024-03-13T16:43:45Z",
"updated_at": "2024-03-13T16:43:45Z",
"matches": [
{
"type": "match.request.client-ip-address.literal",
"ordinal": 1,
"result": "match",
"value": [
"127.0.0.1/32"
],
"features": [
{
"type": "feature.access.deny-access",
"ordinal": 1,
"enabled": "true"
}
]
}
]
}
]
}
Output
{
"platform": "http_large",
"rules": [
{
"description": "Block Client IP",
"ordinal": 1,
"matches": [
{
"type": "match.request.client-ip-address.literal",
"ordinal": 1,
"result": "match",
"value": [
"127.0.0.1/32"
],
"features": [
{
"type": "feature.access.deny-access",
"ordinal": 1,
"enabled": "true"
}
]
}
]
}
]
}
Overwrite/add nested key value
If the key exists it will overwrite it's value, if it doesn't exist it will add the key with the given value
Command
jq '.DistributionConfig.Origins.Items[].OriginAccessControlId=""'
# Example: aws cloudfront get-distribution-config --id "$cloudfront_distribution_ID" | jq '.DistributionConfig.Origins.Items[].OriginAccessControlId=""'
Input
This is only part of a AWS Cloudfront config
{
"ETag": "BBTM0OSB52L2YY",
"DistributionConfig": {
"Origins": {
"Quantity": 1,
"Items": [
{
"Id": "name-of-my-s3-origin",
"DomainName": "bucket-name-example.s3.eu-central-1.amazonaws.com",
"OriginAccessControlId": "BQLJS0DMIOW6VJ"
}
]
}
}
}
Output
{
"ETag": "BBTM0OSB52L2YY",
"DistributionConfig": {
"Origins": {
"Quantity": 1,
"Items": [
{
"Id": "name-of-my-s3-origin",
"DomainName": "bucket-name-example.s3.eu-central-1.amazonaws.com",
"OriginAccessControlId": ""
}
]
}
}
}
