一个系统运维的打怪升级之路

不同主机上的Docker容器之间通信

2019-02-17
257次查阅
2019/5/6

网上找到一些解决方案:

  1. 使用openvswitch 搭建 xvlan协议隧道.
  2. 将多个物理机的容器组到一个物理网络,这需要在每台机器上创建自己的网桥br0,然后将docker默认网桥绑定到br0。
  3. 使用docker的swarm集群。
  4. 使用docker的overlay网络。

这里实现方案4,使用docker的overlay网络。

准备环境(Ubuntu 18.04.2 LTS)

  • 物理机server-1------192.168.0.10------运行docker容器地址:10.10.0.2/16
  • 物理机server-2------192.168.0.20------运行docker容器地址:10.10.1.2/16
  • 物理机server-3------192.168.0.30------运行docker容器地址:10.10.2.2/16

1、安装并配置consul

server-1、server-2、server-3 三台物理机全部安装consul:

wget https://releases.hashicorp.com/consul/1.4.2/consul_1.4.2_linux_amd64.zip
unzip consul_1.4.2_linux_amd64.zip
chmod +x consul
mv consul /usr/bin/

这里将server-1作为键值存储服务器,其他两个作为客户端:

server-1 / 192.168.0.10:
root@server-1:~# nohup consul agent -server -bootstrap -data-dir /var/lib/consul -bind=192.168.0.10 &> /var/log/consul.log &

server-2 / 192.168.0.20:
root@server-2:~# nohup consul agent -data-dir /var/lib/consul -bind=192.168.0.20 &> /var/log/consul.log &
root@server-2:~# consul join 192.168.0.10      # 加入到consul群集

server-3 / 192.168.0.30:
root@server-3:~# nohup consul agent -data-dir /var/lib/consul -bind=192.168.0.30 &> /var/log/consul.log &
root@server-3:~# consul join 192.168.0.10

查看群集内成员:

root@server-1:~# consul members list
 Node      Address            Status  Type    Build  Protocol  DC   Segment
 server-1  192.168.0.10:8301  alive   server  1.4.2  2         dc1  
 server-2  192.168.0.20:8301  alive   client  1.4.2  2         dc1  
 server-3  192.168.0.30:8301  alive   client  1.4.2  2         dc1  

2、配置docker启动参数

为了重启的时候能找到consul的服务端,在三台机器上操作:

vim /lib/systemd/system/docker.service
[Service]
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --cluster-store=consul://localhost:8500 --cluster-advertise=ens33:2375 

其中cluster-store的主机指定为localhost即可,cluster-advertise的ip可以指定为本机的网卡名,修改好之后需要重启docker服务:

systemctl daemon-reload
systemctl restart docker

3、创建overlay网络

在server-1上执行:

docker network create -d overlay --gateway=10.10.0.1 --subnet=10.10.0.0/16 --attachable=true dknet

创建一个类型为 overlay 的网络 dknet,创建好之后其他两个节点会自动同步网络信息。

root@server-1:~# docker network ls

NETWORK ID          NAME                DRIVER              SCOPE

83e7e5433a4b        bridge              bridge              local

0200c8c4e84f        dknet               overlay             global

b986c512c5ce        host                host                local

ee7d6b44478d        none                null                local

4、创建容器并测试

分别在三台物理机上执行:

server-1:
docker run -it --net=dknet --ip=10.10.0.2 --name=web ubuntu:latest bash

server-2:
docker run -it --net=dknet --ip=10.10.1.2 --name=db ubuntu:latest  bash

server-3:
docker run -it --net=dknet --ip=10.10.2.2 --name=app ubuntu:latest  bash

进入容器后先安装命令工具:

root@3539858a25e2:/# apt update && apt install -y net-tools iputils-ping

server-1 上查看ip,并且ping其他两台机器上的容器IP地址:

root@3539858a25e2:/# ifconfig 
 eth0: flags=4163  mtu 1450
         inet 10.10.0.2  netmask 255.255.0.0  broadcast 10.10.255.255
         ether 02:42:0a:0a:00:02  txqueuelen 0  (Ethernet)
         RX packets 0  bytes 0 (0.0 B)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 0  bytes 0 (0.0 B)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 eth1: flags=4163  mtu 1500
         inet 172.18.0.2  netmask 255.255.0.0  broadcast 172.18.255.255
         ether 02:42:ac:12:00:02  txqueuelen 0  (Ethernet)
         RX packets 5442  bytes 16213018 (16.2 MB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 5441  bytes 505135 (505.1 KB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 lo: flags=73  mtu 65536
         inet 127.0.0.1  netmask 255.0.0.0
         loop  txqueuelen 1000  (Local Loopback)
         RX packets 8  bytes 890 (890.0 B)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 8  bytes 890 (890.0 B)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


root@3539858a25e2:/# ping 10.10.1.2 -c 3 
 PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data.
 64 bytes from 10.10.1.2: icmp_seq=1 ttl=64 time=1.10 ms
 64 bytes from 10.10.1.2: icmp_seq=2 ttl=64 time=0.367 ms
 64 bytes from 10.10.1.2: icmp_seq=3 ttl=64 time=0.401 ms
 --- 10.10.1.2 ping statistics ---
 3 packets transmitted, 3 received, 0% packet loss, time 2004ms
 rtt min/avg/max/mdev = 0.367/0.623/1.103/0.340 ms

root@3539858a25e2:/# ping 10.10.2.2 -c 3 
 PING 10.10.2.2 (10.10.2.2) 56(84) bytes of data.
 64 bytes from 10.10.2.2: icmp_seq=1 ttl=64 time=0.451 ms
 64 bytes from 10.10.2.2: icmp_seq=2 ttl=64 time=0.409 ms
 64 bytes from 10.10.2.2: icmp_seq=3 ttl=64 time=0.921 ms

其他两台机器测试方法相同,至此配置完毕!

查看创建的overlay网络dknet:

root@server-1:~# docker network inspect dknet
 [
     {
         "Name": "dknet",
         "Id": "0200c8c4e84ff9a2912552b019e8c90122ffca5066c7b118df8ec5350cb6378c",
         "Created": "2019-02-17T07:45:31.429607539Z",
         "Scope": "global",
         "Driver": "overlay",
         "EnableIPv6": false,
         "IPAM": {
             "Driver": "default",
             "Options": {},
             "Config": [
                 {
                     "Subnet": "10.10.0.0/16",
                     "Gateway": "10.10.0.1"
                 }
             ]
         },
         "Internal": false,
         "Attachable": true,
         "Ingress": false,
         "ConfigFrom": {
             "Network": ""
         },
         "ConfigOnly": false,
         "Containers": {
             "3539858a25e22837fa6649facf4ff1b2ff9581b4dbcdd21541037976a09660da": {
                 "Name": "web",
                 "EndpointID": "cde41697a54feb968024e0ffb057e370eff18bdfc7723633d7babb30e342fd93",
                 "MacAddress": "02:42:0a:0a:00:02",
                 "IPv4Address": "10.10.0.2/16",
                 "IPv6Address": ""
             },
             "ep-9e1670750656ae673948019e7ab08223a1f10d5d1cd8b0c9c4f678d636cae607": {
                 "Name": "app",
                 "EndpointID": "9e1670750656ae673948019e7ab08223a1f10d5d1cd8b0c9c4f678d636cae607",
                 "MacAddress": "02:42:0a:0a:02:02",
                 "IPv4Address": "10.10.2.2/16",
                 "IPv6Address": ""
             },
             "ep-f8a0a2d27b81fc292e75fb15cd8c0d920f81c2178e7eb4bdfa3ad40df3310d78": {
                 "Name": "db",
                 "EndpointID": "f8a0a2d27b81fc292e75fb15cd8c0d920f81c2178e7eb4bdfa3ad40df3310d78",
                 "MacAddress": "02:42:0a:0a:01:02",
                 "IPv4Address": "10.10.1.2/16",
                 "IPv6Address": ""
             }
         },
         "Options": {},
         "Labels": {}
     }
 ]

评论

想说点什么?