简介

在前面两篇文章中,我们分别介绍了快速读取 JSON 值的库gjson和快速设置 JSON 值的库sjson。今天我们介绍它们的作者tidwall的一个基于gjsonsjson的非常实用的命令行工具jj。它是使用 Go 编写的快速读取和设置 JSON 值的命令行程序。

快速使用

Mac 上可以直接使用brew install tidwall/jj/jj安装。其他系统可以通过下载编译好的可执行程序,下载地址为https://github.com/tidwall/jj/releases

我选择使用go get安装:

1
$ go get github.com/tidwall/jj/cmd/jj

上面命令执行完成之后,编译生成的jj程序会放在$GOPATH/bin目录中,我习惯把$GOPATH/bin加入系统可执行目录$PATH中,故可以直接使用。

简单的读取和设置(我的环境为 Win10 + Git Bash):

1
2
3
4
5
$ echo '{"name":{"first":"li","last":"dj"}}' | jj name.last
dj

$ echo '{"name":{"first":"li","last":"dj"}}' | jj -v dajun name.last
{"name":{"first":"li","last":"dajun"}}

通过键路径来指定读取/设置的位置,上面第一个命令读取字段name.last,返回dj

-v选项指定设置的值。第二个命令将字段name.last设置为dajun,输出设置之后的 JSON 串。键路径在前两篇文章中有详细的介绍,不熟悉的可以回去看一下。

读取和设置

实际上读取和设置的语法和形式与我们前面介绍gjsonsjson提到的基本一样,只不过是在命令行上完成的而已。

读取不存在的字段,返回null

1
2
$ echo '{"name":{"first":"li","last":"dj"}}' | jj name.middle
null

读取一个对象类型的字段,返回该对象的 JSON 表示:

1
2
$ echo '{"name":{"first":"li","last":"dj"}}' | jj name
{"first":"li","last":"dj"}

使用索引(从 0 开始)读取数组的元素,非法的索引将返回空:

1
2
3
4
$ echo '{"fruits":["apple","orange","banana"]}' | jj fruits.1
orange

$ echo '{"fruits":["apple","orange","banana"]}' | jj fruits.3

使用索引设置数组的元素,下面命令将数组fruits的第二个元素设置为pear

1
2
$ echo '{"fruits":["apple","orange","banana"]}' | jj -v pear fruits.1
{"fruits":["apple","pear","banana"]}

使用-1或数组长度作为索引,可以在数组后添加一个元素。如果索引超过了数组长度,则会多一定数量的null

1
2
3
4
5
6
7
8
$ echo '{"fruits":["apple","orange","banana"]}' | jj -v strawberry fruits.-1
{"fruits":["apple","orange","banana","strawberry"]}

$ echo '{"fruits":["apple","orange","banana"]}' | jj -v grape fruits.3
{"fruits":["apple","orange","banana","grape"]}

$ echo '{"fruits":["apple","orange","banana"]}' | jj -v watermelon fruits.5
{"fruits":["apple","orange","banana",null,null,"watermelon"]}

使用选项-D删除指定键路径上的元素,如果对应元素不存在,则无效果:

1
2
3
4
5
6
7
8
$ echo '{"name":"dj","age":18}' | jj -D age
{"name":"dj"}

$ echo '{"fruits":["apple","orange","banana"]}' | jj -D fruits.2
{"fruits":["apple","orange"]}

$ echo '{"fruits":["apple","orange","banana"]}' | jj -D fruits.5
{"fruits":["apple","orange","banana"]}

第 1 个命令删除字段age;第 2 个命令删除数组fruits的第 2 个元素;第 3 个命令删除数组fruits的第 5 个元素,由于数组长度只有 3,故无效果。

文件

jj支持从文件中读取 JSON 串和将结果写到文件中。使用选项-i指定输入文件,选项-o指定输出文件。下面将从文件fruits.txt中读取 JSON 串,取数组的第 2 个元素,写到out.txt中:

1
$ jj -i fruits.txt -o out.txt fruits.1

fruits.txt的文件内容如下:

1
{"fruits":["apple","orange","banana"]}

执行命令,输出文件的内容为:

1
orange

格式化

jj支持将输出的 JSON 串进行一定的格式化。选项-u移除所有的空白符,节省存储空间。选项-p美化格式,便于阅读。

1
2
3
4
5
6
7
8
$ echo '{"name":{"first": "li", "last":"dj"}, "age":18}' | jj -u name
{"first":"li","last":"dj"}

$ echo '{"name":{"first": "li", "last":"dj"}, "age":18}' | jj -p name
{
  "first": "li",
  "last": "dj"
}

性能

与另一个 JSON 的命令行工具jq相比,jj是其性能的 10 倍以上。因为jj不会验证 JSON 串的有效性,并且它只关心键路径指定的值,一旦该值处理完成就停止。这里有性能对比:https://github.com/tidwall/jj#performance

用途

jj一个很方便的用途在于日志处理,当前很多日志库都支持 JSON 的格式,例如前面我们介绍的logrus。我们可以使用jj在这些日志中找到相应的信息。我们先用logrus生成 20 条玩家登陆和下线的日志:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
package main

import "github.com/sirupsen/logrus"

func main() {
  logrus.SetFormatter(&logrus.JSONFormatter{})

  for i := 1; i <= 10; i++ {
    logrus.WithFields(logrus.Fields{
      "userid": i,
    }).Info("login")
    logrus.WithFields(logrus.Fields{
      "userid": i,
    }).Info("logoff")
  }
}

生成日志存储在log.txt文件中:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":1}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":1}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":2}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":2}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":3}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":3}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":4}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":4}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":5}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":5}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":6}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":6}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":7}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":7}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":8}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":8}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":9}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":9}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":10}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":10}

由于每一行都是一个单独的 JSON 串,我们可以使用jj支持的 JSON 行特性,使用..路径标识这些行。..使得jj将这些行看成数组的元素。我们可以读取这些数组元素。

获取数组长度,返回 20:

1
2
$ jj -i log.txt ..#
20

只读取每一行中的userid信息:

1
2
$ jj -i log.txt ..#.userid
[1,1,2,2,3,3,4,4,5,5,6,6,7,7,8,8,9,9,10,10]

只读取每一行中的msg信息:

1
2
$ jj -i log.txt ..#.msg
["login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff"]

更复杂一点的,如果我们要查看所有userid=1的日志:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
$ jj -i log.txt ..#\(userid=1\)# -p
[  
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 1
  },
  {
    "level": "info",
    "msg": "logoff",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 1
  }
]

上面的命令注意两点,()是 shell 中的特殊字符,需要\转义。命令中我们使用-p选项使结果更易读。

如果我们只需要查找第一条符合条件的日志,则可以去掉最右侧的#

1
2
3
4
5
6
7
$ jj -i log.txt ..#\(userid=1\) -p
{
  "level": "info",
  "msg": "login",
  "time": "2020-03-26T23:36:04+08:00",
  "userid": 1
}

如果要查看所有的登录信息:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
$ jj -i log.txt ..#\(msg="login"\)# -p
[
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 1
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 2
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 3
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 4
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 5
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 6
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 7
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 8
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 9
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 10
  }
]

总结

jj是一个非常使用的 JSON 命令行工具,性能超赞。执行jj -h去看看其他选项吧。

大家如果发现好玩、好用的 Go 语言库,欢迎到 Go 每日一库 GitHub 上提交 issue😄

参考

  1. jj GitHub:https://github.com/tidwall/jj
  2. Go 每日一库 GitHub:https://github.com/darjun/go-daily-lib

我的博客:https://darjun.github.io

欢迎关注我的微信公众号【GoUpUp】,共同学习,一起进步~