一个系统运维的打怪升级之路

docker仓库–Harbor搭建配置

2019-03-06
686次查阅
2019/5/6

Harbor是Vmvare中国团队开发的开源registry仓库,相比docker官方拥有更丰富的权限权利和完善的架构设计,适用大规模docker集群部署提供仓库服务。 下载地址:https://github.com/goharbor/harbor/releases

安装环境

  • Ubuntu 18.04.2 LTS
  • docker 18.06.1-ce
  • docker-compose version 1.17.1
  • harbor 1.7.4

安装docker和docker-compose

shell> apt update
shell> apt install docker.io
shell> apt-get install docker-compose

安装 Harbor

# 下载离线安装包
shell> wget https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.4.tgz

# 解压软件包
shell> tar zxvf harbor-offline-installer-v1.7.4.tgz && cd harbor

查看目录结构:

shell> tree -L 2
 .
 ├── common
 │   ├── config
 │   └── templates
 ├── docker-compose.chartmuseum.yml
 ├── docker-compose.clair.yml
 ├── docker-compose.notary.yml
 ├── docker-compose.yml
 ├── harbor.cfg
 ├── harbor.v1.7.4.tar.gz
 ├── install.sh
 ├── LICENSE
 ├── open_source_license
 └── prepare

修改配置文件harbor.conf :

## Configuration file of Harbor

#This attribute is for migrator to detect the version of the .cfg file, DO NOT MODIFY!
_version = 1.7.0
#The IP address or hostname to access admin UI and registry service.
#DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
#DO NOT comment out this line, modify the value of "hostname" directly, or the installation will fail.
hostname = 192.168.0.251        # 设置访问地址,可以使用ip、主机名,不可以设置为127.0.0.1或localhost

#The protocol for accessing the UI and token/notification service, by default it is http.
#It can be set to https if ssl is enabled on nginx.
ui_url_protocol = http          # 设置访问协议,默认http,若设为https则nginx ssl需要设置on

#Maximum number of job workers in job service  
max_job_workers = 10 

#Determine whether or not to generate certificate for the registry's token.
#If the value is on, the prepare script creates new root cert and private key 
#for generating token to access the registry. If the value is off the default key/cert will be used.
#This flag also controls the creation of the notary signer's cert.
customize_crt = on

#The path of cert and key files for nginx, they are applied only the protocol is set to https
ssl_cert = /data/harbor-data/cert/server.crt
ssl_cert_key = /data/harbor-data/cert/server.key

#The path of secretkey storage
secretkey_path = /data/harbor-data          # 若没有此目录则需要手动建立

#Admiral's url, comment this attribute, or set its value to NA when Harbor is standalone
admiral_url = NA

#Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated.
log_rotate_count = 50
#Log files are rotated only if they grow bigger than log_rotate_size bytes. If size is followed by k, the size is assumed to be in kilobytes. 
#If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100G 
#are all valid.
log_rotate_size = 200M

#Config http proxy for Clair, e.g. http://my.proxy.com:3128
#Clair doesn't need to connect to harbor internal components via http proxy.
http_proxy =
https_proxy =
no_proxy = 127.0.0.1,localhost,core,registry

#NOTES: The properties between BEGIN INITIAL PROPERTIES and END INITIAL PROPERTIES
#only take effect in the first boot, the subsequent changes of these properties 
#should be performed on web ui

#************************BEGIN INITIAL PROPERTIES************************

#Email account settings for sending out password resetting emails.

#Email server uses the given username and password to authenticate on TLS connections to host and act as identity.
#Identity left blank to act as username.
email_identity =                    # 邮件设置,发送重置密码邮件时使用

email_server = smtp.mydomain.com
email_server_port = 25
email_username = test@mydomain.com
email_password = pass
email_from = test <sunkai@mydomain.com>
email_ssl = false
email_insecure = false

##The initial password of Harbor admin, only works for the first time when Harbor starts. 
#It has no effect after the first launch of Harbor.
#Change the admin password from UI after launching Harbor.
harbor_admin_password = Testpass123456      # 启动Harbor后,管理员UI登录的密码,默认是Harbor12345

##By default the auth mode is db_auth, i.e. the credentials are stored in a local database.
#Set it to ldap_auth if you want to verify a user's credentials against an LDAP server.
auth_mode = db_auth         # 认证方式,这里支持多种认证方式,如LADP、本次存储、数据库认证。默认是db_auth,mysql数据库认证

#The url for an ldap endpoint.
ldap_url = ldaps://ldap.mydomain.com

#A user's DN who has the permission to search the LDAP/AD server. 
#If your LDAP/AD server does not support anonymous search, you should configure this DN and ldap_search_pwd.
#ldap_searchdn = uid=searchuser,ou=people,dc=mydomain,dc=com

#the password of the ldap_searchdn
#ldap_search_pwd = password

#The base DN from which to look up a user in LDAP/AD
ldap_basedn = ou=people,dc=mydomain,dc=com

#Search filter for LDAP/AD, make sure the syntax of the filter is correct.
#ldap_filter = (objectClass=person)

# The attribute used in a search to match a user, it could be uid, cn, email, sAMAccountName or other attributes depending on your LDAP/AD  
ldap_uid = uid 

#the scope to search for users, 0-LDAP_SCOPE_BASE, 1-LDAP_SCOPE_ONELEVEL, 2-LDAP_SCOPE_SUBTREE
ldap_scope = 2 

#Timeout (in seconds)  when connecting to an LDAP Server. The default value (and most reasonable) is 5 seconds.
ldap_timeout = 5

#Verify certificate from LDAP server
ldap_verify_cert = true

#The base dn from which to lookup a group in LDAP/AD
ldap_group_basedn = ou=group,dc=mydomain,dc=com

#filter to search LDAP/AD group
ldap_group_filter = objectclass=group

#The attribute used to name a LDAP/AD group, it could be cn, name
ldap_group_gid = cn

#The scope to search for ldap groups. 0-LDAP_SCOPE_BASE, 1-LDAP_SCOPE_ONELEVEL, 2-LDAP_SCOPE_SUBTREE
ldap_group_scope = 2

#Turn on or off the self-registration feature
self_registration = on        # 是否开启自注册

#The expiration time (in minute) of token created by token service, default is 30 minutes
token_expiration = 30         # Token有效时间,默认30分钟

#The flag to control what users have permission to create projects
#The default value "everyone" allows everyone to creates a project. 
#Set to "adminonly" so that only admin user can create project.
project_creation_restriction = everyone    # 用户创建项目权限控制,默认是everyone(所有人),也可以设置为adminonly(只能管理员)

#************************END INITIAL PROPERTIES************************

#######Harbor DB configuration section#######

#The address of the Harbor database. Only need to change when using external db.
db_host = postgresql

#The password for the root user of Harbor DB. Change this before any production use.
db_password = root123

#The port of Harbor database host
db_port = 5432

#The user name of Harbor database
db_user = postgres

##### End of Harbor DB configuration#######

##########Redis server configuration.############

#Redis connection address
redis_host = redis

#Redis connection port
redis_port = 6379

#Redis connection password
redis_password = 

#Redis connection db index
#db_index 1,2,3 is for registry, jobservice and chartmuseum. 
#db_index 0 is for UI, it's unchangeable
redis_db_index = 1,2,3

########## End of Redis server configuration ############

##########Clair DB configuration############

#Clair DB host address. Only change it when using an exteral DB.
clair_db_host = postgresql
#The password of the Clair's postgres database. Only effective when Harbor is deployed with Clair.
#Please update it before deployment. Subsequent update will cause Clair's API server and Harbor unable to access Clair's database.
clair_db_password = root123
#Clair DB connect port
clair_db_port = 5432
#Clair DB username
clair_db_username = postgres
#Clair default database
clair_db = postgres

#The interval of clair updaters, the unit is hour, set to 0 to disable the updaters.
clair_updaters_interval = 12

##########End of Clair DB configuration############

#The following attributes only need to be set when auth mode is uaa_auth
uaa_endpoint = uaa.mydomain.org
uaa_clientid = id
uaa_clientsecret = secret
uaa_verify_cert = true
uaa_ca_cert = /path/to/ca.pem


### Harbor Storage settings ###
#Please be aware that the following storage settings will be applied to both docker registry and helm chart repository.
#registry_storage_provider can be: filesystem, s3, gcs, azure, etc.
registry_storage_provider_name = filesystem
#registry_storage_provider_config is a comma separated "key: value" pairs, e.g. "key1: value, key2: value2".
#To avoid duplicated configurations, both docker registry and chart repository follow the same storage configuration specifications of docker registry.
#Refer to https://docs.docker.com/registry/configuration/#storage for all available configuration.
registry_storage_provider_config =
#registry_custom_ca_bundle is the path to the custom root ca certificate, which will be injected into the truststore
#of registry's and chart repository's containers.  This is usually needed when the user hosts a internal storage with self signed certificate.
registry_custom_ca_bundle = 

#If reload_config=true, all settings which present in harbor.cfg take effect after prepare and restart harbor, it overwrites exsiting settings.
#reload_config=true
#Regular expression to match skipped environment variables
#skip_reload_env_pattern=(^EMAIL.*)|(^LDAP.*)

编辑 docker-compose.yml 文件(主要是自定义目录路径):

version: '2'
services:
  log:
    image: goharbor/harbor-log:v1.7.4
    container_name: harbor-log 
    restart: always
    dns_search: .
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - DAC_OVERRIDE
      - SETGID
      - SETUID
    volumes:
      - /var/log/harbor/:/var/log/docker/:z
      - ./common/config/log/:/etc/logrotate.d/:z
    ports:
      - 127.0.0.1:1514:10514
    networks:
      - harbor
  registry:
    image: goharbor/registry-photon:v2.6.2-v1.7.4
    container_name: registry
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /data/harbor-data/registry:/storage:z
      - ./common/config/registry/:/etc/registry/:z
      - ./common/config/custom-ca-bundle.crt:/harbor_cust_cert/custom-ca-bundle.crt:z
    networks:
      - harbor
    dns_search: .
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "registry"
  registryctl:
    image: goharbor/harbor-registryctl:v1.7.4
    container_name: registryctl
    env_file:
      - ./common/config/registryctl/env
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /data/harbor-data/registry:/storage:z
      - ./common/config/registry/:/etc/registry/:z
      - ./common/config/registryctl/config.yml:/etc/registryctl/config.yml:z
    networks:
      - harbor
    dns_search: .
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "registryctl"
  postgresql:
    image: goharbor/harbor-db:v1.7.4
    container_name: harbor-db
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - DAC_OVERRIDE
      - SETGID
      - SETUID
    volumes:
      - /data/harbor-data/database:/var/lib/postgresql/data:z
    networks:
      - harbor
    dns_search: .
    env_file:
      - ./common/config/db/env
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "postgresql"
  adminserver:
    image: goharbor/harbor-adminserver:v1.7.4
    container_name: harbor-adminserver
    env_file:
      - ./common/config/adminserver/env
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /data/harbor-data/config/:/etc/adminserver/config/:z
      - /data/harbor-data/secretkey:/etc/adminserver/key:z
      - /data/harbor-data/:/data/:z
    networks:
      - harbor
    dns_search: .
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "adminserver"
  core:
    image: goharbor/harbor-core:v1.7.4
    container_name: harbor-core
    env_file:
      - ./common/config/core/env
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - SETGID
      - SETUID
    volumes:
      - ./common/config/core/app.conf:/etc/core/app.conf:z
      - ./common/config/core/private_key.pem:/etc/core/private_key.pem:z
      - ./common/config/core/certificates/:/etc/core/certificates/:z
      - /data/harbor-data/secretkey:/etc/core/key:z
      - /data/harbor-data/ca_download/:/etc/core/ca/:z
      - /data/harbor-data/psc/:/etc/core/token/:z
      - /data/harbor-data/:/data/:z
    networks:
      - harbor
    dns_search: .
    depends_on:
      - log
      - adminserver
      - registry
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "core"
  portal:
    image: goharbor/harbor-portal:v1.7.4
    container_name: harbor-portal
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
      - NET_BIND_SERVICE
    networks:
      - harbor
    dns_search: .
    depends_on:
      - log
      - core
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "portal"

  jobservice:
    image: goharbor/harbor-jobservice:v1.7.4
    container_name: harbor-jobservice
    env_file:
      - ./common/config/jobservice/env
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /data/harbor-data/job_logs:/var/log/jobs:z
      - ./common/config/jobservice/config.yml:/etc/jobservice/config.yml:z
    networks:
      - harbor
    dns_search: .
    depends_on:
      - redis
      - core
      - adminserver
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "jobservice"
  redis:
    image: goharbor/redis-photon:v1.7.4
    container_name: redis
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    volumes:
      - /data/harbor-data/redis:/var/lib/redis
    networks:
      - harbor
    dns_search: .
    depends_on:
      - log
    logging:
      driver: "syslog"
      options:
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "redis"
  proxy:
    image: goharbor/nginx-photon:v1.7.4
    container_name: nginx
    restart: always
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
      - NET_BIND_SERVICE
    volumes:
      - ./common/config/nginx:/etc/nginx:z
    networks:
      - harbor
    dns_search: .
    ports:
      - 80:80
      - 443:443
      - 4443:4443
    depends_on:
      - postgresql
      - registry
      - core
      - portal
      - log
    logging:
      driver: "syslog"
      options:  
        syslog-address: "tcp://127.0.0.1:1514"
        tag: "proxy"
networks:
  harbor:
    external: false

开始安装harbor:

shell> ./install.sh

Harbor依赖的镜像及启动服务如下:

sehll> docker images
REPOSITORY                      TAG                 IMAGE ID            CREATED             SIZE
goharbor/chartmuseum-photon     v0.8.1-v1.7.4       7e2272c02339        45 hours ago        113MB
goharbor/harbor-migrator        v1.7.4              968c31d07d2f        45 hours ago        678MB
goharbor/redis-photon           v1.7.4              611d1ead0a28        45 hours ago        99.7MB
goharbor/clair-photon           v2.0.7-v1.7.4       01090529ab14        45 hours ago        165MB
goharbor/notary-server-photon   v0.6.1-v1.7.4       737518b1b943        45 hours ago        135MB
goharbor/notary-signer-photon   v0.6.1-v1.7.4       495dc3326120        45 hours ago        132MB
goharbor/harbor-registryctl     v1.7.4              723aed7bbf8d        45 hours ago        102MB
goharbor/registry-photon        v2.6.2-v1.7.4       f4743bd7b0d9        45 hours ago        86.7MB
goharbor/nginx-photon           v1.7.4              dda34e6afafe        45 hours ago        35.9MB
goharbor/harbor-log             v1.7.4              bf4916eef530        45 hours ago        81.4MB
goharbor/harbor-jobservice      v1.7.4              1b6a0445ae9c        45 hours ago        84.1MB
goharbor/harbor-core            v1.7.4              e603b8750d26        45 hours ago        95.6MB
goharbor/harbor-portal          v1.7.4              2ca1d845cafa        45 hours ago        40.6MB
goharbor/harbor-adminserver     v1.7.4              5706c65d65dc        45 hours ago        72.3MB
goharbor/harbor-db              v1.7.4              08d163f732f3        45 hours ago        136MB

shell> docker ps
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS                 PORTS                                                              NAMES
a9d80f1900fe        goharbor/nginx-photon:v1.7.4             "nginx -g 'daemon of…"   2 hours ago         Up 2 hours (healthy)   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp   nginx
473e0991509f        goharbor/harbor-portal:v1.7.4            "nginx -g 'daemon of…"   2 hours ago         Up 2 hours (healthy)   80/tcp                                                             harbor-portal
30ea93434bdb        goharbor/harbor-jobservice:v1.7.4        "/harbor/start.sh"       2 hours ago         Up 2 hours                                                                                harbor-jobservice
8e7f44aec4b0        goharbor/harbor-core:v1.7.4              "/harbor/start.sh"       2 hours ago         Up 2 hours (healthy)                                                                      harbor-core
5ed5305ab1c3        goharbor/redis-photon:v1.7.4             "docker-entrypoint.s…"   2 hours ago         Up 2 hours             6379/tcp                                                           redis
6240ce80d3b9        goharbor/harbor-registryctl:v1.7.4       "/harbor/start.sh"       2 hours ago         Up 2 hours (healthy)                                                                      registryctl
e7d3baa5f786        goharbor/harbor-adminserver:v1.7.4       "/harbor/start.sh"       2 hours ago         Up 2 hours (healthy)                                                                      harbor-adminserver
34c9169ccfbf        goharbor/registry-photon:v2.6.2-v1.7.4   "/entrypoint.sh /etc…"   2 hours ago         Up 2 hours (healthy)   5000/tcp                                                           registry
6a3921e88626        goharbor/harbor-db:v1.7.4                "/entrypoint.sh post…"   2 hours ago         Up 2 hours (healthy)   5432/tcp                                                           harbor-db
722f2bf4353a        goharbor/harbor-log:v1.7.4               "/bin/sh -c /usr/loc…"   2 hours ago         Up 2 hours (healthy)   127.0.0.1:1514->10514/tcp      

安装好之后可以用浏览器访问 http://192.168.0.251 来访问harbor仓库。

常用操作命令:

docke-compose ps             # 查看容器。
docker-compose up -d         # 后台启动,如果容器不存在根据镜像自动创建。
docker-compose down -v       # 停止容器并删除容器。
docker-compose start         # 启动容器,容器不存在就无法启动,不会自动创建镜像。
docker-compose stop          # 停止容器。
docker-compose logs          # 查看日志(harbor日志存放 /var/logs/harbor)

配置https

创建存放证书目录:
mkdir /data/harbor-data/cer && cd /data/harbor-data/cer

创建自己的CA证书(不使用第三方权威机构的CA来认证,自己充当CA的角色):

openssl genrsa -out ca.key 2048   # 生成根证书私钥(无加密)

生成自签名证书(使用已有私钥ca.key自行签发根证书):

openssl req -x509 -new -nodes -key ca.key -days 10000 -out ca.crt -subj "/CN=Harbor-ca"

req     产生证书签发申请命令
-x509   签发X.509格式证书命令。X.509是最通用的一种签名证书格式。
-new    生成证书请求
-key     指定私钥文件
-nodes   表示私钥不加密
-out    输出
-subj    指定用户信息
-days    有效期

生成服务器端私钥和CSR签名请求:

openssl req -newkey rsa:4096 -nodes -sha256 -keyout server.key -out server.csr     # 一路回车

签发服务器证书:

echo subjectAltName = IP:192.168.0.251 > extfile.cnf
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 365 -extfile extfile.cnf -out server.crt

x509           签发X.509格式证书命令。
-req           表示证书输入请求。
-days          表示有效天数
-extensions    表示按OpenSSL配置文件v3_req项添加扩展。
-CA            表示CA证书,这里为ca.crt
-CAkey         表示CA证书密钥,这里为ca.key
-CAcreateserial表示创建CA证书序列号
-extfile      指定文件 

修改harbor配置文件harbor.cfg:

ui_url_protocol = https

ssl_cert = /data/harbor-data/cert/server.crt
ssl_cert_key = /data/harbor-data/cert/server.key

仓库操作

上传镜像到仓库:

# 打标签
docker tag ubuntu:latest 192.168.0.251/test/ubuntu:latest

# 推送到仓库
docker push 192.168.0.251/test/ubuntu:latest

# 从仓库拉取
docker pull 192.168.0.251/test/ubuntu:latest

可能会遇到的问题:

docker login的时候报错:

The push refers to repository [192.168.0.251/test/ubuntu]
4b7d93055d87: Preparing 
663e8522d78b: Preparing 
283fb404ea94: Preparing 
bebe7ce6215a: Preparing 
denied: requested access to the resource is denied

解决办法:

apt install dbus-x11
apt install gnupg2 pass
gpg2 --full-generate-key

docker pull的时候报错:

Trying to pull repository 192.168.0.251/test/ubuntu … 
Get https://192.168.0.251/v1/_ping: x509: certificate signed by unknown authority

解决方法:

1、CentOS:

编辑 /etc/sysconfig/docker,找到:

OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false'

修改为:

OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false --insecure-registry 192.168.0.251'

2、Ubuntu:

编辑/lib/systemd/system/docker.service,找到:

ExecStart=/usr/bin/dockerd -H fd://

修改为:

ExecStart=/usr/bin/dockerd -H fd:// --insecure-registry=192.168.0.251

评论

想说点什么?