远程连接docker daemon,Docker Remote API

不知道大家有没有遇到这样一种情形:每次构建好了镜像,push到私有仓库后。你还要ssh到服务器,进行pull,每次登录服务器的过程非常的痛苦。如果服务器ip没有映射域名,那记录ip也是一个痛苦的过程。博主,开始关注到docker remote api,它可以让你在本地就可以完成docker的所有操作,于是博主开发了基于etcd配置的docker管理工具,还蛮好用的。下面是我docker部分配置的一些经历,分享给大家,希望对大家有帮助。

更新列表

  • 2017/04/27:添加 /etc/docker/daemon.json 文件配置过程(version 在 1.12 之后可用),文档地址

服务端搭建

首先,我们需要通过系统的包管理器安装docker

当我们的docker安装好之后,运行sudo docker ps查看是否运行成功。

$ sudo groupadd docker # 创建docker组
$ sudo usermod -aG docker whoami # 将当前用户添加到docker组
重启docker服务,注销登录,再次登录,这样就可以免去每次输入sudo的烦恼了。

ok,我们安装好之后,docker宿主程序默认是通过非网络的Unix套接字运行,是只能够进行本地通信(/var/run/docker.sock),是不能够直接远程连接docker的。需要修改其配置:

测试环境

ubuntu: 有host1,ubuntu系统,其配置文件路径在/etc/default/docker

$ sudo vi /etc/default/docker
  DOCKER_OPTS="-H tcp://0.0.0.0:2375"
$ sudo service docker restart

# ubuntu docker的其它操作方式
$ sudo service docker start
$ sudo service docker stop

这里监听了tcp2375端口,现在我们就可以通过别的主机host2访问到这台主机的docker了(但是host1现在不能本地访问了)。推荐DOCKER_OPTS写成:

DOCKER_OPTS="-H unix:///var/run/docker.sock -H 0.0.0.0:2375"

当然了这是不安全的,任何人都能够访问该端口,推荐测试用。

centos: 有host1,centos系统,其配置文件路径在/etc/sysconfig/docker

$ sudo vi /etc/sysconfig/docker
  other_args="-H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock"
$ sudo service docker restart

# centos docker的其它操作方式
$ sudo service docker start
$ sudo service docker stop

$ /bin/systemctl start  docker.service

注:同样这里也是不安全的。

daemon.json:1.12版本后, 用户可以自行创建配置文件 /etc/docker/daemon.json,该文件不区分系统,是通用的,推荐使用。具体参考:官方文档。不知道版本的可以通过 $ dockerd version 查看。

首先,你需要创建 /etc/docker/daemon.json 文件,文件内容如下:

{
  "hosts": [
    "tcp://0.0.0.0:2375",
    "unix:///var/run/docker.sock"
  ]
}

然后,通过 dockerd 启动守护进程:

$ dockerd
WARN[0000] [!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]
INFO[0000] libcontainerd: new containerd process, pid: 3952
WARN[0000] containerd: low RLIMIT_NOFILE changing to max  current=1024 max=4096
INFO[0001] [graphdriver] using prior storage driver "aufs"
INFO[0001] Graph migration to content-addressability took 0.00 seconds
WARN[0001] Your kernel does not support cgroup blkio weight
WARN[0001] Your kernel does not support cgroup blkio weight_device
INFO[0001] Loading containers: start.
......................INFO[0001] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address

INFO[0001] Loading containers: done.
INFO[0001] Daemon has completed initialization
INFO[0001] Docker daemon                                 commit=7392c3b graphdriver=aufs version=1.12.5
INFO[0001] API listen on /var/run/docker.sock
INFO[0001] API listen on [::]:2375

这里你可以看到警告 WARN[0000],在不使用 tlsverify 验证的情况下,一定要注意 IP 的绑定!!!

更多 dockerd 命令的使用方法请参考:https://docs.docker.com/engine/reference/commandline/dockerd/

可以看到,上面所说的均没有采取任何验证方式。仅供测试使用或内部使用。千万不要暴露到公网。

线上环境,安全环境

这里介绍,通过自签名证书安全认证构建HTTPS encypted socket。上面测试环境用的2375端口,docker推荐2376作为安全端口。当然我们可以随意设置端口,哈哈。

证书的生成: 详细信息,移步官网:Protect the Docker daemon socket。其原理是通过指定tlsverify标志并将Docker的tlscacert标志指向受信任的CA证书来启用TLS。在守护进程模式下,它只允许来自由该CA签名的证书认证的客户端的连接。 在客户端模式下,它将只连接到具有由该CA签名的证书的服务器。

警告:使用TLS和管理CA是一个高深的主题。在生产环境中使用OpenSSL,x509和TLS之前,请熟悉OpenSSL,x509和TLS。
警告:这些TLS命令将只在Linux上生成一组有效的证书。 macOS附带的OpenSSL版本与Docker所需的证书不兼容。

首先,生成CA公钥和私钥:

$ openssl genrsa -aes256 -out ca-key.pem 4096       # 生成CA私钥
Generating RSA private key, 4096 bit long modulus
............................................................................................................................................................................................++
........++
e is 65537 (0x10001)
Enter pass phrase for ca-key.pem:
Verifying - Enter pass phrase for ca-key.pem:

$ openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem     #生成CA公钥,也就是证书
Enter pass phrase for ca-key.pem:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:Queensland
Locality Name (eg, city) []:Brisbane
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Docker Inc
Organizational Unit Name (eg, section) []:Sales
Common Name (e.g. server FQDN or YOUR name) []:$HOST
Email Address []:Sven@home.org.au

现在我们有了CA,就可以创建服务器私钥和证书请求文件了,请确保Common Name (i.e., server FQDN or YOUR name)匹配你将要连接的docker主机。

注意,使用你docker宿主机的DNS name替换下面的$HOST

$ openssl genrsa -out server-key.pem 4096       # 生成服务器私钥
Generating RSA private key, 4096 bit long modulus
.....................................................................++
.................................................................................................++
e is 65537 (0x10001)
$ openssl req -subj "/CN=$HOST" -sha256 -new -key server-key.pem -out server.csr  # 用私钥生成证书请求文件

现在,我们可以用CA来签署证书了。这里我们可以填写IP地址或则DNS name,如,我们需要允许10.10.10.20127.0.0.1连接:

$ echo subjectAltName = IP:10.10.10.20,IP:127.0.0.1 > extfile.cnf

$ openssl x509 -req -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem \
  -CAcreateserial -out server-cert.pem -extfile extfile.cnf
Signature ok
subject=/CN=your.host.com
Getting CA Private Key
Enter pass phrase for ca-key.pem:

客户端证书:

$ openssl genrsa -out key.pem 4096      # 客户端私钥
Generating RSA private key, 4096 bit long modulus
.........................................................++
................++
e is 65537 (0x10001)
$ openssl req -subj '/CN=client' -new -key key.pem -out client.csr      # 客户端证书请求文件

CA为客户端签署证书文件:

$ openssl x509 -req -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem \
  -CAcreateserial -out cert.pem -extfile extfile.cnf
Signature ok
subject=/CN=client
Getting CA Private Key
Enter pass phrase for ca-key.pem:

删除证书请求文件:

$ rm -v client.csr server.csr

默认的私钥权限太开放了,为了更加的安全,我们需要更改证书的权限,删除写入权限,限制阅读权限(只有你能查看):

$ chmod -v 0400 ca-key.pem key.pem server-key.pem

证书文件删除其写入权限:

$ chmod -v 0444 ca.pem server-cert.pem cert.pem

证书的部署:

# ubuntu
$ sudo vi /etc/default/docker
    如:DOCKER_OPTS="-D --tlsverify=true --tlscert=/var/docker/server-cert.pem --tlskey=/var/docker/server-key.pem --tlscacert=/var/docker/ca.pem
    -H tcp://0.0.0.0:2376"
$ sudo service docker restart


# centos
$ sudo vi /etc/sysconfig/docker
    如:OPTIONS='--selinux-enabled --log-driver=journald --tlsverify=true'
    DOCKER_CERT_PATH=/etc/docker
$ sudo service docker restart


# daemon.json
$ sudo vi /etc/docker/daemon.json
{
  "tlsverify": true,
  "tlscert": "/var/docker/server-cert.pem",
  "tlskey": "/var/docker/server-key.pem",
  "tlscacert": "/var/docker/ca.pem",
  "hosts": [
    "tcp://0.0.0.0:2376",
    "unix:///var/run/docker.sock"
  ]
}

$ dockerd
INFO[0000] libcontainerd: new containerd process, pid: 4823
WARN[0000] containerd: low RLIMIT_NOFILE changing to max  current=1024 max=4096
INFO[0001] [graphdriver] using prior storage driver "aufs"
INFO[0001] Graph migration to content-addressability took 0.00 seconds
WARN[0001] Your kernel does not support cgroup blkio weight
WARN[0001] Your kernel does not support cgroup blkio weight_device
INFO[0001] Loading containers: start.
......................INFO[0001] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address

INFO[0001] Loading containers: done.
INFO[0001] Daemon has completed initialization
INFO[0001] Docker daemon                                 commit=7392c3b graphdriver=aufs version=1.12.5
INFO[0001] API listen on /var/run/docker.sock
INFO[0001] API listen on [::]:2376

ok,可以看到没有警告 WARN

警告:这些证书的保存非常重要,关系着你的服务器的安全,请妥善保管。

客户端连接

普通连接:

$ docker -H tcp://127.0.0.1:2375 ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                      NAMES
3ed7b8f338ad        mongo:3.2           "/entrypoint.sh mo..."   11 days ago         Up 3 hours          0.0.0.0:27017->27017/tcp   eidb

TLS连接:

$ docker --tlsverify --tlscacert=~/docker/ca.pem \
  --tlscert=~/docker/cert.pem \
  --tlskey=~/docker/key.pem \
  -H=192.168.99.100:2376 version
  
Client:
 Version:      1.13.0-rc1
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   75fd88b
 Built:        Fri Nov 11 22:32:34 2016
 OS/Arch:      darwin/amd64

Server:
 Version:             1.13.0-rc1
 API version:         1.25
 Minimum API version: 1.12
 Go version:          go1.7.3
 Git commit:          75fd88b
 Built:               Fri Nov 11 22:32:34 2016
 OS/Arch:             linux/amd64
 Experimental:        false
 
# curl 连接测试
$ curl https://192.168.99.100:2376/images/json \
  --cert ~/.docker/cert.pem \
  --key ~/.docker/key.pem \
  --cacert ~/.docker/ca.pem

为了不每次都指定证书,我们可以指定默认连接:

$ mkdir -pv ~/.docker
$ cp -v ~/{ca,cert,key}.pem ~/.docker
$ export DOCKER_HOST=tcp://192.168.99.100:2376 DOCKER_TLS_VERIFY=1     # 这里只是临时指定,永久请写入*profile里面

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                      NAMES
3ed7b8f338ad        mongo:3.2           "/entrypoint.sh mo..."   11 days ago         Up 3 hours          0.0.0.0:27017->27017/tcp   eidb

当然也可以指定证书文件路径:

$ export DOCKER_CERT_PATH="~/docker"

我的mac下的~/.bash_profile如下:

export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_API_VERSION=""
export DOCKER_CERT_PATH="/Users/chen/.docker/machine/machines/default"

特别说明以下,DOCKER_API_VERSION可以不设置,默认是docker/client/client.go中的const DefaultVersion string版本。

其它模式

如果你不想有完全的双向认证,你可以通过混合标志来运行各种其他模式的Docker。

宿主机 模式:

  • tlsverify, tlscacert, tlscert, tlskey set: Authenticate clients
  • tls, tlscert, tlskey: Do not authenticate clients

客户度 模式:

  • tls: Authenticate server based on public/default CA pool
  • tlsverify, tlscacert: Authenticate server based on given CA
  • tls, tlscert, tlskey: Authenticate with client certificate, do not authenticate server based on given CA
  • tlsverify, tlscacert, tlscert, tlskey: Authenticate with client certificate and authenticate server based on given CA

附上:

docker remote apihttps://github.com/docker/docker/tree/master/client

etcd clientv3https://github.com/coreos/etcd/tree/master/clientv3

本文链接:参与评论 »

--EOF--

专题「Docker 相关技术」的其它文章 »

Comments