Kubernetes环境GitLab部署与应用

发布于2022-09-18,最后编辑于2024-05-24,全文约1758字,阅读时间约4分钟。

Kubernetes Helm GitLab GitLab CE Source Control CI/CD DevOps PostgreSQL MinIO

GitLab Dashboard
GitLab Dashboard
更新记录
  • 2022-09-24

    • GitLab版本由v15.3.3更新至v15.4.0
    • GitLab Helm Chart版本由v6.3.3更新至v6.4.0
  • 2022-10-01

    • GitLab版本由v15.4.0更新至v15.4.1
    • GitLab Helm Chart版本由v6.4.0更新至v6.4.1
  • 2022-10-14

    • GitLab版本由v15.4.1更新至v15.4.2
    • GitLab Helm Chart版本由v6.4.1更新至v6.4.2
  • 2022-10-20

    • GitLab版本由v15.5.3更新至v15.6.2
    • GitLab Helm Chart版本由v6.5.5更新至v6.6.2
  • 2022-12-30

    • GitLab版本由v15.6.2更新至v15.7.0
    • GitLab Helm Chart版本由v6.6.2更新至v6.7.0
  • 2023-01-07

    • GitLab版本由v15.7.0更新至v15.7.1
    • GitLab Helm Chart版本由v6.7.0更新至v6.7.1
  • 2023-01-10

    • GitLab版本由v15.7.1更新至v15.7.2
    • GitLab Helm Chart版本由v6.7.1更新至v6.7.2
  • 2023-01-22

    • GitLab版本由v15.7.2更新至v15.7.5
    • GitLab Helm Chart版本由v6.7.2更新至v6.7.5
  • 2023-01-28

    • GitLab版本由v15.7.5更新至v15.8.0
    • GitLab Helm Chart版本由v6.7.5更新至v6.8.0
  • 2023-02-01

    • GitLab版本由v15.8.0更新至v15.8.1
    • GitLab Helm Chart版本由v6.8.0更新至v6.8.1
  • 2023-02-18

    • GitLab版本由v15.8.1更新至v15.8.3
    • GitLab Helm Chart版本由v6.8.1更新至v6.8.3
  • 2023-03-03

    • GitLab版本由v15.8.3更新至v15.9.2
    • GitLab Helm Chart版本由v6.8.3更新至v6.9.2
  • 2023-03-22

    • GitLab版本由v15.9.2更新至v15.9.3
    • GitLab Helm Chart版本由v6.9.2更新至v6.9.3
  • 2023-03-27

    • GitLab版本由v15.9.3更新至v15.10.0
    • GitLab Helm Chart版本由v6.9.3更新至v6.10.0
  • 2023-04-14

    • GitLab版本由v15.10.0更新至v15.10.2
    • GitLab Helm Chart版本由v6.10.0更新至v6.10.2
  • 2023-05-03

    • GitLab版本由v15.10.2更新至v15.11.1
    • GitLab Helm Chart版本由v6.10.2更新至v6.11.1
  • 2023-06-02

    • GitLab版本由v15.11.1更新至v16.0.1
    • GitLab Helm Chart版本由v6.11.1更新至v7.0.1
  • 2023-06-27

    • GitLab版本由v16.0.1更新至v16.1.0
    • GitLab Helm Chart版本由v7.0.1更新至v7.1.0
  • 2023-07-24

    • GitLab版本由v16.1.0更新至v16.2.0
    • GitLab Helm Chart版本由v7.1.0更新至v7.2.0
  • 2023-08-08

    • GitLab版本由v16.2.3更新至v16.2.4
    • GitLab Helm Chart版本由v7.2.3更新至v7.2.4
  • 2023-09-01

    • GitLab版本由v16.2.4更新至v16.3.1
    • GitLab Helm Chart版本由v7.2.4更新至v7.3.1
  • 2024-01-01

    • GitLab版本由v16.3.1更新至v16.7.0
    • GitLab Helm Chart版本由v7.3.1更新至v7.7.0
  • 2024-03-31

    • GitLab版本由v16.7.0更新至v16.10.1
    • GitLab Helm Chart版本由v7.7.0更新至v7.10.1

概述

本文用于整理基于Kubernetes环境的GitLab部署与应用,实现Source Control、CI/CD等功能,作为后续演练项目的前置环境准备。

随着各相关组件版本的更新,笔者将在验证通过后对本文进行补充和更新,请参考更新记录

本次演练环境为Kubernetes集群环境,环境配置可参考笔者另一篇笔记《Kubernetes集群部署笔记》。

本次演练使用Traefik作为Ingress Controller实现,环境配置可参考笔者另一篇笔记《Kubernetes环境Traefik部署与应用》。

本次演练使用外部数据库实例模式部署GitLab服务,关于PostgreSQL数据库实例的部署,可参考笔者另一篇笔记《Kubernetes环境PostgreSQL部署与应用》。

本次演练使用外部对象存储服务模式部署GitLab服务,关于MinIO对象存储服务的部署,可参考笔者另一篇笔记《Kubernetes环境MinIO部署与应用》。

组件版本

配置过程

准备工作

  • 添加Helm仓库

    添加用于安装GitLab的Helm仓库。

    1helm repo add gitlab https://charts.gitlab.io/
    2helm repo update
    
  • 创建命名空间

    本次演练中将GitLab安装至apps-gitlab命名空间,可根据需要替换。

    1kubectl create namespace apps-gitlab
    
  • 创建TLS证书Secret

    从已准备好的证书key文件和crt文件创建Secret

    1kubectl create secret tls local-choral-io-tls -n apps-gitlab \
    2  --key=local.choral.io.key --cert=local.choral.io.crt
    
  • 创建Traefik EntryPoint

    创建一个新的Traefik EntryPoint,用于提供对GitLab Shell SSH协议的访问。

    首先,导出当前Traefik部署的配置文件。

    1helm get values --output yaml --namespace kube-system traefik > helm-traefik.yaml
    

    添加新的参数,更新Traefik部署。

    1# ports.git-ssh.expose=false  禁用公开访问 稍后会手动创建用于访问该端口的负载均衡器
    2# ports.git-ssh.port=8022     指定绑定端口
    3# ports.git-ssh.protocol=TCP  指定绑定协议
    4helm upgrade --install --namespace kube-system \
    5  --values helm-traefik.yaml \
    6  --set ports.git-ssh.expose=false \
    7  --set ports.git-ssh.port=8022 \
    8  --set ports.git-ssh.protocol=TCP \
    9  traefik traefik/traefik
    
  • 创建负载均衡器

    创建一个新的LoadBalancer类型的Service,用于提供对2280443端口的访问。

     1cat <<EOF | kubectl apply -f - > /dev/null
     2apiVersion: v1
     3kind: Service
     4metadata:
     5  name: traefik-git
     6  namespace: kube-system
     7  labels:
     8    app.kubernetes.io/instance: traefik-kube-system
     9    app.kubernetes.io/name: traefik
    10spec:
    11  type: LoadBalancer
    12  selector:
    13    app.kubernetes.io/instance: traefik-kube-system
    14    app.kubernetes.io/name: traefik
    15  ports:
    16  - name: ssh
    17    port: 22
    18    protocol: TCP
    19    targetPort: 8022
    20  - name: web
    21    port: 80
    22    protocol: TCP
    23    targetPort: web
    24  - name: websecure
    25    port: 443
    26    protocol: TCP
    27    targetPort: websecure
    28EOF
    

安装GitLab

  • 创建PostgreSQL密码Secret

    1kubectl create secret -n apps-gitlab generic gitlab-postgresql-secret \
    2  --from-literal=postgresql-password=37Z8FeRZlkYuBtMWKtLsiLPz
    
  • 创建MinIO认证凭据Secret

    创建gitlab-minio.yaml

    1provider: AWS
    2region: cn-north-1
    3aws_access_key_id: TL6JVVW85A9L4MFI4985
    4aws_secret_access_key: ILnRAe8cuEJUmbCxTAPOodM3Rhu5gvD4ulZJskEL
    5aws_signature_version: 4
    6host: minio.local.choral.io
    7endpoint: "https://minio.local.choral.io"
    8path_style: true
    

    创建gitlab-minio.config

    1[default]
    2host_base = minio.local.choral.io
    3host_bucket = minio.local.choral.io
    4use_https = True
    5signature_v2 = False
    6access_key = TL6JVVW85A9L4MFI4985
    7secret_key = ILnRAe8cuEJUmbCxTAPOodM3Rhu5gvD4ulZJskEL
    8bucket_location = cn-north-1
    9multipart_chunk_size_mb = 128
    

    创建包含上述两个文件的Secret对象。

    1kubectl create secret -n apps-gitlab generic gitlab-minio-secret \
    2  --from-file=connection=gitlab-minio.yaml \
    3  --from-file=config=gitlab-minio.config
    
  • 创建所需的Buckets

    1mc mb choral-local/gitlab-{artifacts,backups,backups-tmp,caches,dependency-proxy,gitlfs,packages,uploads}
    
    1Bucket created successfully `choral-local/gitlab-artifacts`.
    2Bucket created successfully `choral-local/gitlab-backups`.
    3Bucket created successfully `choral-local/gitlab-backups-tmp`.
    4Bucket created successfully `choral-local/gitlab-caches`.
    5Bucket created successfully `choral-local/gitlab-dependency-proxy`.
    6Bucket created successfully `choral-local/gitlab-gitlfs`.
    7Bucket created successfully `choral-local/gitlab-packages`.
    8Bucket created successfully `choral-local/gitlab-uploads`.
    
  • 创建helm-gitlab.yaml

      1registry:
      2  enabled: false
      3postgresql:
      4  install: false
      5certmanager:
      6  install: false
      7prometheus:
      8  install: false
      9nginx-ingress:
     10  enabled: false
     11global:
     12  edition: ce
     13  time_zone: Asia/Shanghai
     14  kas:
     15    enabled: false
     16  minio:
     17    enabled: false
     18  hosts:
     19    https: true
     20    domain: local.choral.io
     21    gitlab:
     22      name: code.local.choral.io
     23  ingress:
     24    enabled: false
     25  psql:
     26    host: postgresql.data-postgresql
     27    database: gitlab
     28    username: gitlab
     29    password:
     30      secret: gitlab-postgresql-secret
     31      key: postgresql-password
     32  appConfig:
     33    lfs:
     34      bucket: gitlab-gitlfs
     35      connection:
     36        secret: gitlab-minio-secret
     37        key: connection
     38    artifacts:
     39      bucket: gitlab-artifacts
     40      connection:
     41        secret: gitlab-minio-secret
     42        key: connection
     43    uploads:
     44      bucket: gitlab-uploads
     45      connection:
     46        secret: gitlab-minio-secret
     47        key: connection
     48    packages:
     49      bucket: gitlab-packages
     50      connection:
     51        secret: gitlab-minio-secret
     52        key: connection
     53    dependencyProxy:
     54      enabled: true
     55      bucket: gitlab-dependency-proxy
     56      connection:
     57        secret: gitlab-minio-secret
     58        key: connection
     59    backups:
     60      bucket: gitlab-backups
     61      tmpBucket: gitlab-backups-tmp
     62    defaultProjectsFeatures:
     63      issues: true
     64      mergeRequests: true
     65      wiki: true
     66      snippets: true
     67      builds: true
     68      containerRegistry: false
     69gitlab:
     70  webservice:
     71    registry:
     72      enabled: false
     73    resources:
     74      requests:
     75        cpu: 150m
     76  sidekiq:
     77    registry:
     78      enabled: false
     79    resources:
     80      requests:
     81        cpu: 200m
     82  toolbox:
     83    backups:
     84      objectStorage:
     85        config:
     86          secret: gitlab-minio-secret
     87          key: config
     88  gitaly:
     89    persistence:
     90      size: 20Gi
     91gitlab-runner:
     92  runners:
     93    privileged: true
     94    config: |
     95      [[runners]]
     96        [runners.kubernetes]
     97          image = "debian:bullseye"
     98          privileged = true
     99          image_pull_secrets = []
    100        [runners.cache]
    101          Type = "s3"
    102          Path = "runners"
    103          Shared = true
    104          [runners.cache.s3]
    105            ServerAddress = "minio.local.choral.io"
    106            AccessKey = "TL6JVVW85A9L4MFI4985"
    107            SecretKey = "ILnRAe8cuEJUmbCxTAPOodM3Rhu5gvD4ulZJskEL"
    108            BucketName = "gitlab-caches"
    109            BucketLocation = "cn-north-1"
    110            Insecure = false      
    111  gitlabUrl: http://gitlab-webservice-default:8181
    
  • 安装GitLab

    使用上一步中创建的配置文件安装GitLab。

    1helm upgrade --install gitlab --namespace apps-gitlab \
    2  --values helm-gitlab.yaml \
    3  gitlab/gitlab
    
  • 配置Ingress入口规则

    创建用于提供http和https协议访问Ingress Route对象。

     1cat <<EOF | kubectl apply -f - > /dev/null
     2apiVersion: traefik.io/v1alpha1
     3kind: IngressRoute
     4metadata:
     5  name: gitlab-http-web
     6  namespace: apps-gitlab
     7spec:
     8  entryPoints:
     9    - websecure
    10  routes:
    11    - match: Host(\`code.local.choral.io\`)
    12      kind: Rule
    13      services:
    14        - name: gitlab-webservice-default
    15          kind: Service
    16          port: 8181
    17  tls:
    18    secretName: local-choral-io-tls
    19EOF
    

    创建用于提供SSH协议访问的Ingress Route TCP对象。

     1cat <<EOF | kubectl apply -f - > /dev/null
     2apiVersion: traefik.io/v1alpha1
     3kind: IngressRouteTCP
     4metadata:
     5  name: gitlab-ssh-shell
     6  namespace: apps-gitlab
     7spec:
     8  entryPoints:
     9    - git-ssh
    10  routes:
    11    - match: HostSNI(\`*\`)
    12      services:
    13        - name: gitlab-gitlab-shell
    14          port: 22
    15EOF
    
  • 获取root用户初始密码

    1kubectl get secret --namespace apps-gitlab gitlab-gitlab-initial-root-password \
    2  -o jsonpath="{.data.password}" | base64 -d
    

    获取密码之后可以使用root用户名和密码登录GitLab(本次演练中访问地址为https://code.local.choral.io/)。

参考资料