不同主机上的Docker容器之间通信

2019-02-17 78次浏览 发表评论

网上找到一些解决方案:

  1. 使用openvswitch 搭建 xvlan协议隧道.
  2. 将多个物理机的容器组到一个物理网络,这需要在每台机器上创建自己的网桥br0,然后将docker默认网桥绑定到br0。
  3. 使用docker的swarm集群。
  4. 使用docker的overlay网络。

这里实现方案4,使用docker的overlay网络。

准备环境(Ubuntu 18.04.2 LTS)

  • 物理机server-1------192.168.0.10------运行docker容器地址:10.10.0.2/16
  • 物理机server-2------192.168.0.20------运行docker容器地址:10.10.1.2/16
  • 物理机server-3------192.168.0.30------运行docker容器地址:10.10.2.2/16

1、安装并配置consul

server-1、server-2、server-3 三台物理机全部安装consul:

wget https://releases.hashicorp.com/consul/1.4.2/consul_1.4.2_linux_amd64.zip
unzip consul_1.4.2_linux_amd64.zip
chmod +x consul
mv consul /usr/bin/

这里将server-1作为键值存储服务器,其他两个作为客户端:

server-1 / 192.168.0.10:
root@server-1:~# nohup consul agent -server -bootstrap -data-dir /var/lib/consul -bind=192.168.0.10 &> /var/log/consul.log &

server-2 / 192.168.0.20:
root@server-2:~# nohup consul agent -data-dir /var/lib/consul -bind=192.168.0.20 &> /var/log/consul.log &
root@server-2:~# consul join 192.168.0.10 # 加入到consul群集

server-3 / 192.168.0.30:
root@server-3:~# nohup consul agent -data-dir /var/lib/consul -bind=192.168.0.30 &> /var/log/consul.log &
root@server-3:~# consul join 192.168.0.10

查看群集内成员:

root@server-1:~# consul members list
Node Address Status Type Build Protocol DC Segment
server-1 192.168.0.10:8301 alive server 1.4.2 2 dc1
server-2 192.168.0.20:8301 alive client 1.4.2 2 dc1
server-3 192.168.0.30:8301 alive client 1.4.2 2 dc1

2、配置docker启动参数

为了重启的时候能找到consul的服务端,在三台机器上操作:

vim /lib/systemd/system/docker.service
[Service]
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --cluster-store=consul://localhost:8500 --cluster-advertise=ens33:2375 

其中cluster-store的主机指定为localhost即可,cluster-advertise的ip可以指定为本机的网卡名,修改好之后需要重启docker服务:

systemctl daemon-reload
systemctl restart docker

3、创建overlay网络

在server-1上执行:

docker network create -d overlay --gateway=10.10.0.1 --subnet=10.10.0.0/16 --attachable=true dknet

创建一个类型为 overlay 的网络 dknet,创建好之后其他两个节点会自动同步网络信息。

root@server-1:~# docker network ls
NETWORK ID NAME DRIVER SCOPE
83e7e5433a4b bridge bridge local
0200c8c4e84f dknet overlay global
b986c512c5ce host host local
ee7d6b44478d none null local

4、创建容器并测试

分别在三台物理机上执行:

server-1:
docker run -it --net=dknet --ip=10.10.0.2 --name=web ubuntu:latest bash

server-2:
docker run -it --net=dknet --ip=10.10.1.2 --name=db ubuntu:latest bash

server-3:
docker run -it --net=dknet --ip=10.10.2.2 --name=app ubuntu:latest bash

进入容器后先安装命令工具:

root@3539858a25e2:/# apt update && apt install -y net-tools iputils-ping

server-1 上查看ip,并且ping其他两台机器上的容器IP地址:

root@3539858a25e2:/# ifconfig 
eth0: flags=4163 mtu 1450
inet 10.10.0.2 netmask 255.255.0.0 broadcast 10.10.255.255
ether 02:42:0a:0a:00:02 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163 mtu 1500
inet 172.18.0.2 netmask 255.255.0.0 broadcast 172.18.255.255
ether 02:42:ac:12:00:02 txqueuelen 0 (Ethernet)
RX packets 5442 bytes 16213018 (16.2 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 5441 bytes 505135 (505.1 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 8 bytes 890 (890.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8 bytes 890 (890.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0


root@3539858a25e2:/# ping 10.10.1.2 -c 3
PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data.
64 bytes from 10.10.1.2: icmp_seq=1 ttl=64 time=1.10 ms
64 bytes from 10.10.1.2: icmp_seq=2 ttl=64 time=0.367 ms
64 bytes from 10.10.1.2: icmp_seq=3 ttl=64 time=0.401 ms
--- 10.10.1.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 0.367/0.623/1.103/0.340 ms

root@3539858a25e2:/# ping 10.10.2.2 -c 3
PING 10.10.2.2 (10.10.2.2) 56(84) bytes of data.
64 bytes from 10.10.2.2: icmp_seq=1 ttl=64 time=0.451 ms
64 bytes from 10.10.2.2: icmp_seq=2 ttl=64 time=0.409 ms
64 bytes from 10.10.2.2: icmp_seq=3 ttl=64 time=0.921 ms

其他两台机器测试方法相同,至此配置完毕!

查看创建的overlay网络dknet:

root@server-1:~# docker network inspect dknet
[
{
"Name": "dknet",
"Id": "0200c8c4e84ff9a2912552b019e8c90122ffca5066c7b118df8ec5350cb6378c",
"Created": "2019-02-17T07:45:31.429607539Z",
"Scope": "global",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "10.10.0.0/16",
"Gateway": "10.10.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"3539858a25e22837fa6649facf4ff1b2ff9581b4dbcdd21541037976a09660da": {
"Name": "web",
"EndpointID": "cde41697a54feb968024e0ffb057e370eff18bdfc7723633d7babb30e342fd93",
"MacAddress": "02:42:0a:0a:00:02",
"IPv4Address": "10.10.0.2/16",
"IPv6Address": ""
},
"ep-9e1670750656ae673948019e7ab08223a1f10d5d1cd8b0c9c4f678d636cae607": {
"Name": "app",
"EndpointID": "9e1670750656ae673948019e7ab08223a1f10d5d1cd8b0c9c4f678d636cae607",
"MacAddress": "02:42:0a:0a:02:02",
"IPv4Address": "10.10.2.2/16",
"IPv6Address": ""
},
"ep-f8a0a2d27b81fc292e75fb15cd8c0d920f81c2178e7eb4bdfa3ad40df3310d78": {
"Name": "db",
"EndpointID": "f8a0a2d27b81fc292e75fb15cd8c0d920f81c2178e7eb4bdfa3ad40df3310d78",
"MacAddress": "02:42:0a:0a:01:02",
"IPv4Address": "10.10.1.2/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}
]

评论

随便说点,聊聊天~