V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
just4id
V2EX  ›  Docker

基于 docker swarm 搭建跨主机的 mongodb 副本集遇到的坑,请大牛赐教

  •  
  •   just4id · 2019-11-18 14:26:01 +08:00 · 6064 次点击
    这是一个创建于 1836 天前的主题,其中的信息可能已经有所发展或是发生改变。
    问题如下:
    创建了两个 vm,主机名分别是: 'RedHat'和'gen8-arch-linux',用 docker stack 命令在两个 docker node 上起了两个 mongodb 的容器,并已加入到 docker overlay 的网络。但进入到其中任何一个 mongodb 的容器,初始化 replica set 时,连通不了另一个容器里的 mongodb 服务,故组 rs 不成功。经本人查看两个容器的 overlay 网络 IP,网络应该是通的,详情如下:
    
    管理结点:
    [jacky@gen8-arch-linux ~]$ cat mongo.yml
    version: '3.4'
    
    services:
    
      primary:
        image: mongo:3.4
        hostname: "{{.Service.Name}}"
        command: mongod --port 27017 --bind_ip=0.0.0.0 --replSet mongo --dbpath /data/db
        networks:
          - es
        volumes:
          - /home/jacky/data/mongo-data:/data/db
        deploy:
          restart_policy:
            condition: on-failure
          replicas: 1
          placement:
            constraints:
              - node.hostname==gen8-arch-linux
          endpoint_mode: dnsrr
    
      secondary:
        image: mongo:3.4
        hostname: "{{.Service.Name}}"
        command: mongod --port 27017 --bind_ip=0.0.0.0 --replSet mongo --dbpath /data/db
        networks:
          - es
        volumes:
          - /home/oracle/data/mongo-data:/data/db
        deploy:
          restart_policy:
            condition: on-failure
          replicas: 1
          placement:
            constraints:
              - node.hostname==RedHat
          endpoint_mode: dnsrr
    
    networks:
      es:
        external: true
    [jacky@gen8-arch-linux ~]$ docker stack deploy -c mongo.yml mongo
    Creating service mongo_secondary
    Creating service mongo_primary
    [jacky@gen8-arch-linux ~]$ docker node ls
    ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
    vt82wfc13t831n2wcltm3d3lz     RedHat              Ready               Active                                  18.06.0-ce
    xazp9k3n4axfb2n8emdrxl8ba *   gen8-arch-linux     Ready               Active              Leader              19.03.2-ce
    [jacky@gen8-arch-linux ~]$ docker service ls
    ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
    mjfaxe0h7bms        mongo_primary       replicated          1/1                 mongo:3.4
    bcs50kfp6hd2        mongo_secondary     replicated          1/1                 mongo:3.4
    [jacky@gen8-arch-linux ~]$ docker network ls | grep es
    se7u7jxai61l        es                  overlay             swarm
    [jacky@gen8-arch-linux ~]$ docker ps -a | grep mongo
    97d200f88726        mongo:3.4           "docker-entrypoint.s…"   8 minutes ago       Up 8 minutes        27017/tcp                mongo_primary.1.699xh09shmdlyu7dpd43yzsy8
    [jacky@gen8-arch-linux ~]$ docker inspect 97 | grep v4
                            "IPv4Address": "10.0.0.4"
    
    工作结点:
    [oracle@RedHat ~]$ docker ps -a | grep 3.4
    227dac4e143b        mongo:3.4                           "docker-entrypoint.s…"   12 minutes ago      Up 11 minutes             27017/tcp           mongo_secondary.1.mk3jiksavbmh5qch1yx4p2mht
    [oracle@RedHat ~]$ docker inspect 227 | grep v4
                            "IPv4Address": "10.0.0.2"
    
    最后在管理结点上的 mongodb 容器组 replica set:
    [jacky@gen8-arch-linux ~]$ docker exec -it 97 /bin/bash
    root@mongo_primary:/# mongo
    MongoDB shell version v3.4.23
    connecting to: mongodb://127.0.0.1:27017
    MongoDB server version: 3.4.23
    Welcome to the MongoDB shell.
    For interactive help, type "help".
    For more comprehensive documentation, see
            http://docs.mongodb.org/
    Questions? Try the support group
            http://groups.google.com/group/mongodb-user
    Server has startup warnings:
    2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten]
    2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
    2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
    2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten]
    2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
    2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
    2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten]
    > exit
    bye
    root@mongo_primary:/# mongo
    MongoDB shell version v3.4.23
    connecting to: mongodb://127.0.0.1:27017
    MongoDB server version: 3.4.23
    Server has startup warnings:
    2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten]
    2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
    2019-11-18T05:59:18.088+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
    2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten]
    2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
    2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
    2019-11-18T05:59:20.480+0000 I CONTROL  [initandlisten]
    > config={_id:"mongo",members:[{_id:1,host:"mongo_primary:27017"},{_id:2,host:"mongo_secondary:27017"}]}
    {
            "_id" : "mongo",
            "members" : [
                    {
                            "_id" : 1,
                            "host" : "mongo_primary:27017"
                    },
                    {
                            "_id" : 2,
                            "host" : "mongo_secondary:27017"
                    }
            ]
    }
    > rs.initiate(config)
    {
            "ok" : 0,
            "errmsg" : "replSetInitiate quorum check failed because not all proposed set members responded affirmatively: mongo_secondary:27017 failed with Couldn't get a connection within the time limit",
            "code" : 74,
            "codeName" : "NodeNotFound"
    }
    > config={_id:"mongo",members:[{_id:1,host:"10.0.0.4:27017"},{_id:2,host:"10.0.0.2:27017"}]}
    {
            "_id" : "mongo",
            "members" : [
                    {
                            "_id" : 1,
                            "host" : "10.0.0.4:27017"
                    },
                    {
                            "_id" : 2,
                            "host" : "10.0.0.2:27017"
                    }
            ]
    }
    > rs.initiate(config)
    {
            "ok" : 0,
            "errmsg" : "replSetInitiate quorum check failed because not all proposed set members responded affirmatively: 10.0.0.2:27017 failed with Couldn't get a connection within the time limit",
            "code" : 74,
            "codeName" : "NodeNotFound"
    }
    
    总结:
    在管理结点上的 mongodb 容器上用主机名或 IP 地址访问另一个工作结点上的 mongodb 容器的服务,不通。
    
    请大牛们赐教,小白多谢了!
    
    2 条回复    2019-11-29 21:20:54 +08:00
    hopingtop
        1
    hopingtop  
       2019-11-29 16:28:58 +08:00
    @just4id
    overlay 如果容器之间相互 ping 能通
    你试试 telnet 27017 这个端口
    是不是差一个端口映射? -p 27017:27017
    just4id
        2
    just4id  
    OP
       2019-11-29 21:20:54 +08:00 via iPhone
    @hopingtop 谢谢回复,问题已解决,是防火墙的问题,关闭防火墙或开放端口就能解决问题。
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   5437 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 24ms · UTC 07:30 · PVG 15:30 · LAX 23:30 · JFK 02:30
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.