JSON command-line operations

Preparing JSON files from command-line using Python base library.

Python has a library to process JSON by default. See [1].

Using Python on command-line, simple scripts can be written for preparing JSON configurations (e.g., with a Makefile).

Stringify

Want to stringify a JSON file in order to embedded it into another JSON content as a string?

cat embedded.json | python -c 'import json,sys; print(json.dumps(sys.stdin.read()))'
#                                                          ^^^^^ dump (1)

Minify (a little bit)

Want to minify a JSON file in order to remove newlines and multiple spaces?

cat embedded.json | python -c 'import json,sys; print(json.dumps(json.loads(sys.stdin.read())))'
#                                                                     ^^^^^ parse (1)
#                                                          ^^^^^            dump  (2)

Minify & Stringify

cat embedded.json | python -c 'import json,sys; print(json.dumps(json.dumps(json.loads(sys.stdin.read()))))'
#                                                                                ^^^^^ parse (1)
#                                                                     ^^^^^            dump  (2)
#                                                          ^^^^^^                      dump  (3)

Script it!

Multiple lines of text are easy to assign to a variable in bash:

MY_VARIABLE="This content
is on
multiple lines."
echo "$MY_VARIABLE"

...but it is not convenient to use it for assigning JSON content because of all the " and '. Another way is to use cat for the assignment:

MY_VARIABLE=$(cat <<EOF
This content is on multiple lines
and has " and ' in it.
EOF
)
echo "$MY_VARIABLE"

Note1: There is a difference between echo $MY_VAR and echo "$MY_VAR" [2].

Note2: For variable replacement in text file, use envsubst [3]:

LOOP_IDS="i0 i1 i2 i3"
for IDX in $LOOP_IDS; do
    LOOP_VAR=$(IDX=$IDX envsubst < myTemplateFile.json)

    echo "$LOOP_VAR"
done

In the JSON template file, $IDX (or ${IDX}) will be replaced by i0, i1, i2, and i3.

All together:

LOOP_IDS="i0 i1 i2 i3"
for IDX in $LOOP_IDS; do
    # The magic lines!
    JSON_FULL=$(IDX=$IDX envsubst < myTemplateFile.json)
    JSON_MINI=$(echo "$JSON_FULL" | python -c 'import json,sys; print(json.dumps(json.dumps(json.loads(sys.stdin.read()))))')

LOOP_VAR=$(cat <<EOF
                   {
                        "id":"$IDX"
                        "json":$JSON_MINI
                   },
EOF
)

    echo "$LOOP_VAR"
done

Note for later

Why not using something less human- but more parser-friendly than JSON? Have a look at http://msgpack.org