Skip to content
Velloreth
Developer Tools 6 min read

jq: Working With JSON on the Command Line

jq filters, transforms and formats JSON in a shell pipeline. Learn the filters, the operators, and the patterns you will use daily.

By

What jq does

jq reads JSON, applies a filter, and writes JSON. The filter is a small program, and the same filter is applied to every value in the input.

echo '{"name":"alice","age":30}' | jq '.name'
# "alice"

The dot is the identity filter. It returns the input unchanged.

echo '{"a":1}' | jq '.'
# {
#   "a": 1
# }

A dot followed by a name selects a key.

echo '{"name":"alice"}' | jq '.name'
# "alice"

A dot followed by a number selects an array index.

echo '[10,20,30]' | jq '.[1]'
# 20

The flags that matter

FlagMeaning
-rraw output, no quotes on a string
-ccompact output, one line
-eset the exit code from the result
-nuse null as the input
-sslurp the whole input into an array
-Ssort the keys
-jjoin the output without a newline

-r is the one you will use most. A value that goes into a shell variable or a URL should not have quotes.

TOKEN=$(curl -s https://api.example.com/token | jq -r '.access_token')

-c is for a pipeline. One line per value is easier to pass to another tool.

curl -s https://api.example.com/users | jq -c '.[]'

-e sets the exit code. It is 0 if the last output was neither false nor null, and 1 otherwise, which lets a script branch.

if jq -e '.error' response.json > /dev/null; then
  echo "the response has an error"
fi

Selecting and composing

The pipe composes filters, passing the output of one to the next.

echo '{"user":{"name":"alice"}}' | jq '.user.name'
# "alice"

The comma produces multiple outputs.

echo '{"a":1,"b":2}' | jq '.a, .b'
# 1
# 2

An object constructor builds a new object.

echo '{"first":"alice","last":"smith"}' | jq '{name: (.first + " " + .last)}'
# {
#   "name": "alice smith"
# }

An array constructor builds a new array.

echo '{"a":1,"b":2}' | jq '[.a, .b]'
# [1, 2]

Working with arrays

.[] iterates over the elements.

echo '[1,2,3]' | jq '.[]'
# 1
# 2
# 3

map applies a filter to every element.

echo '[1,2,3]' | jq 'map(. * 2)'
# [2, 4, 6]

select keeps the elements that match.

echo '[{"age":25},{"age":40}]' | jq 'map(select(.age > 30))'
# [{"age": 40}]

length counts, and sort_by orders.

echo '[3,1,2]' | jq 'length'
# 3
echo '[{"n":3},{"n":1}]' | jq 'sort_by(.n)'

group_by groups, and unique removes duplicates.

echo '[{"t":"a"},{"t":"b"},{"t":"a"}]' | jq 'group_by(.t) | map({t: .[0].t, n: length})'

Handling a missing key

A missing key returns null.

echo '{}' | jq '.name'
# null

// provides a default.

echo '{}' | jq '.name // "unknown"'
# "unknown"

? suppresses an error.

echo '{"a":1}' | jq '.a.b?'
# null

has tests for a key.

echo '{"a":1}' | jq 'has("a")'
# true

type reports the type, which is useful when the shape is not what you expected.

echo '[]' | jq 'type'
# "array"

Common patterns

Extract a field from every object in an array.

curl -s https://api.example.com/users | jq -r '.[].email'

Filter and reshape.

curl -s https://api.example.com/users \
  | jq '[.[] | select(.active) | {id, email}]'

Count the elements.

curl -s https://api.example.com/users | jq 'length'

Sum a field.

echo '[{"n":1},{"n":2}]' | jq 'map(.n) | add'
# 3

Build a CSV.

curl -s https://api.example.com/users \
  | jq -r '.[] | [.id, .email] | @csv'

Read a value into a shell variable.

VERSION=$(jq -r '.version' package.json)

Update a value and write it back.

jq '.version = "2.0.0"' package.json > tmp.json && mv tmp.json package.json

Merge two objects.

jq -s '.[0] * .[1]' a.json b.json

Slurp a stream of objects into an array.

jq -s '.' <(echo '{"a":1}'; echo '{"b":2}')

Why jq is worth learning

jq is a small language, and the investment is repaid quickly because the alternative is a script in a general-purpose language that does the same thing with more code. A filter that extracts a field from an API response is one line, and the same filter composes with curl, with a loop, and with another jq invocation. The second reason is that jq is present in most environments where JSON is processed: a container image, a CI runner, and a developer’s machine. The third is that a jq filter is a precise description of a transformation, which makes it reviewable in a way that a bespoke script is not. The cost is the syntax, which is terse and unfamiliar at first, and the payoff is a tool that turns a multi-line script into a one-line command.

A jq workflow

A jq filter is easiest to build incrementally, because each step is a small transformation that can be checked on its own. The first step is to look at the shape of the data, which is a jq with the identity filter and a compact flag, or a jq that reports the type of the top-level value. The second is to select the part that matters, which is a path such as a key or an index. The third is to iterate, which is a map over an array or a stream over the elements. The fourth is to reshape, which is an object or array constructor. The fifth is to format, which is a raw flag for a value that goes into a variable or a CSV conversion for a table. Building the filter in those steps means a mistake is visible at the step where it was introduced, rather than as a wrong result at the end of a long pipeline.

Common mistakes

Forgetting -r. The value has quotes, which breaks a URL or a comparison.

Not quoting the filter. The shell interprets the special characters. Use single quotes.

Assuming a key exists. A missing key is null, and a null in a string concatenation produces an error or an unexpected value. Use // or ?.

Using .[] on an object and expecting an array. It iterates the values, which is often what you want, but the result is a stream rather than an array. Wrap it in [...] to collect.

A filter that produces a stream where an array is expected. map returns an array; .[] returns a stream.

Editing a file in place with a redirect to the same file. The shell truncates the file before jq reads it. Write to a temporary file and move it.

Ignoring the exit code. A jq filter that produces null exits 0 by default. Use -e when the result is a condition.

A quotable summary

jq applies a filter to JSON and writes JSON, where the dot is the identity, a dot and a name selects a key, and a pipe composes filters. Use -r for a raw value, -c for compact output, and -e to set the exit code from the result. map applies a filter to every element, select keeps the matching ones, and // provides a default for a missing key. Quote the filter in single quotes, and write to a temporary file when editing a file in place.

For the shell around this, see linux-command-line-basics and for the search tools, grep-and-find-guide. For the broader context, json-vs-yaml-vs-toml and api-error-responses-guide.

Frequently asked questions

What is jq used for?
jq is a command-line processor for JSON. It reads JSON, applies a filter to select or transform it, and writes JSON. It is used to extract a field from an API response, to reshape data, to filter an array, and to format JSON for reading.
What does the dot mean in jq?
The dot is the identity filter: it returns the input unchanged. A dot followed by a name selects that key, so .name returns the value of the name key. A dot followed by a number selects an array index, so .[0] returns the first element.
How do I get a value without quotes?
Use the -r flag for raw output. Without it, a string is printed as a JSON string with quotes. With -r, the string is printed as plain text, which is what you want when the value is used in a shell variable or a URL.
How do I filter an array in jq?
Use map(select(condition)). The map applies the filter to every element, and select keeps the elements for which the condition is true. For example, map(select(.age > 30)) keeps the objects whose age is greater than 30.
How do I handle a missing key?
A missing key returns null, and the // operator provides a default. So .name // "unknown" returns the name if it exists and the string otherwise. The ? operator suppresses an error, so .a.b? returns null instead of failing when a is not an object.
How do I use jq in a shell script?
Use -e so the exit code reflects the result, which lets a script branch on it. Use -r for a raw value that goes into a variable. Use -c for compact output that is easier to pass to another tool. Quote the filter in single quotes so the shell does not interpret it.