docker 启动失败：Failed to start Docker Application Container Engine

docker 的错误日志

突然出现，并不是刚安装出现，唯一能得到的信息是：Failed to start Docker Application Container Engine，但是并没有明确的提示

[root@gitlab ~]# systemctl start docker
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
[root@gitlab ~]# systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Fri 2023-03-03 10:01:03 CST; 16s ago
     Docs: https://docs.docker.com
  Process: 14244 ExecStart=/usr/bin/dockerd -H fd:// --containerd /var/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
 Main PID: 14244 (code=exited, status=1/FAILURE)

Mar 03 10:01:01 gitlab.shandianji.net systemd[1]: Failed to start Docker Application Container Engine.
Mar 03 10:01:01 gitlab.shandianji.net systemd[1]: Unit docker.service entered failed state.
Mar 03 10:01:01 gitlab.shandianji.net systemd[1]: docker.service failed.
Mar 03 10:01:03 gitlab.shandianji.net systemd[1]: docker.service holdoff time over, scheduling restart.
Mar 03 10:01:03 gitlab.shandianji.net systemd[1]: Stopped Docker Application Container Engine.
Mar 03 10:01:03 gitlab.shandianji.net systemd[1]: start request repeated too quickly for docker.service
Mar 03 10:01:03 gitlab.shandianji.net systemd[1]: Failed to start Docker Application Container Engine.
Mar 03 10:01:03 gitlab.shandianji.net systemd[1]: Unit docker.service entered failed state.
Mar 03 10:01:03 gitlab.shandianji.net systemd[1]: docker.service failed.
[root@gitlab ~]# journalctl -xe
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit run-14395.scope has finished starting up.
-- 
-- The start-up result is done.
Mar 03 10:01:18 gitlab.shandianji.net kubelet[14387]: I0303 10:01:18.939036   14387 server.go:446] "Kubelet version" kubeletMar 03 10:01:18 gitlab.shandianji.net kubelet[14387]: I0303 10:01:18.939217   14387 server.go:874] "Client rotation is on, wMar 03 10:01:18 gitlab.shandianji.net kubelet[14387]: I0303 10:01:18.940432   14387 certificate_store.go:130] Loading cert/kMar 03 10:01:18 gitlab.shandianji.net kubelet[14387]: I0303 10:01:18.941004   14387 dynamic_cafile_content.go:156] "StartingMar 03 10:01:23 gitlab.shandianji.net kubelet[14387]: I0303 10:01:23.964360   14387 server.go:693] "--cgroups-per-qos enableMar 03 10:01:23 gitlab.shandianji.net kubelet[14387]: E0303 10:01:23.964501   14387 server.go:302] "Failed to run kubelet" eMar 03 10:01:23 gitlab.shandianji.net systemd[1]: kubelet.service: main process exited, code=exited, status=1/FAILURE
Mar 03 10:01:23 gitlab.shandianji.net systemd[1]: Unit kubelet.service entered failed state.
Mar 03 10:01:23 gitlab.shandianji.net systemd[1]: kubelet.service failed.
Mar 03 10:01:34 gitlab.shandianji.net systemd[1]: kubelet.service holdoff time over, scheduling restart.
Mar 03 10:01:34 gitlab.shandianji.net systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit kubelet.service has finished shutting down.
Mar 03 10:01:34 gitlab.shandianji.net systemd[1]: Started kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit kubelet.service has finished starting up.
-- 
-- The start-up result is done.
Mar 03 10:01:34 gitlab.shandianji.net systemd[1]: Started Kubernetes systemd probe.
-- Subject: Unit run-14491.scope has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit run-14491.scope has finished starting up.
-- 
-- The start-up result is done.
Mar 03 10:01:34 gitlab.shandianji.net kubelet[14483]: I0303 10:01:34.187253   14483 server.go:446] "Kubelet version" kubeletMar 03 10:01:34 gitlab.shandianji.net kubelet[14483]: I0303 10:01:34.187436   14483 server.go:874] "Client rotation is on, wMar 03 10:01:34 gitlab.shandianji.net kubelet[14483]: I0303 10:01:34.188495   14483 certificate_store.go:130] Loading cert/kMar 03 10:01:34 gitlab.shandianji.net kubelet[14483]: I0303 10:01:34.189092   14483 dynamic_cafile_content.go:156] "Starting

解决办法

网上修改 daemon.json，或者 mv daemon.json daemon.conf, 再或者加参数，启动还是一样。

于是在启动 docker 的时候查看系统的日志

tail -200f /var/log/messages

Mar  3 10:14:09 gitlab systemd: Starting Docker Application Container Engine...
Mar  3 10:14:09 gitlab dockerd: unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid character '"' after array element
Mar  3 10:14:09 gitlab systemd: docker.service: main process exited, code=exited, status=1/FAILURE
Mar  3 10:14:09 gitlab systemd: Failed to start Docker Application Container Engine.
Mar  3 10:14:09 gitlab systemd: Unit docker.service entered failed state.
Mar  3 10:14:09 gitlab systemd: docker.service failed.
Mar  3 10:14:12 gitlab systemd: docker.service holdoff time over, scheduling restart.
Mar  3 10:14:12 gitlab systemd: Stopped Docker Application Container Engine.
Mar  3 10:14:12 gitlab systemd: Closed Docker Socket for the API.
Mar  3 10:14:12 gitlab systemd: Stopping Docker Socket for the API.
Mar  3 10:14:12 gitlab systemd: Starting Docker Socket for the API.
Mar  3 10:14:12 gitlab systemd: Listening on Docker Socket for the API.
Mar  3 10:14:12 gitlab systemd: start request repeated too quickly for docker.service
Mar  3 10:14:12 gitlab systemd: Failed to start Docker Application Container Engine.
Mar  3 10:14:12 gitlab systemd: Unit docker.socket entered failed state.
Mar  3 10:14:12 gitlab systemd: Unit docker.service entered failed state.
Mar  3 10:14:12 gitlab systemd: docker.service failed.

得到以上跟 docker 启动失败相关的日志有用一句：

dockerd: unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid character '"' after array element

这一看，应该语法有错误，空格呀、逗号呀，什么的

vim /etc/docker/daemon.json

咋一看确实少了一个逗号，齐了怪了，都没改过，重启后突然就这样。不过这台是公司内网服务器，可能被其他同事修改过。