Kubernetes环境GitLab部署与应用

概述

本文用于整理基于Kubernetes环境的GitLab部署与应用，实现Source Control、CI/CD等功能，作为后续演练项目的前置环境准备。

随着各相关组件版本的更新，笔者将在验证通过后对本文进行补充和更新，请参考更新记录。

本次演练环境为Kubernetes集群环境，环境配置可参考笔者另一篇笔记《Kubernetes集群部署笔记》。

本次演练使用Traefik作为Ingress Controller实现，环境配置可参考笔者另一篇笔记《Kubernetes环境Traefik部署与应用》。

本次演练使用外部数据库实例模式部署GitLab服务，关于PostgreSQL数据库实例的部署，可参考笔者另一篇笔记《Kubernetes环境PostgreSQL部署与应用》。

本次演练使用外部对象存储服务模式部署GitLab服务，关于MinIO对象存储服务的部署，可参考笔者另一篇笔记《Kubernetes环境MinIO部署与应用》。

组件版本

• GitLab v18.0.1

• GitLab Helm Chart v9.0.1

配置过程

准备工作

• 添加Helm仓库

  添加用于安装GitLab的Helm仓库。

  helm repo add gitlab https://charts.gitlab.io/
  helm repo update gitlab

• 创建命名空间

  本次演练中将GitLab安装至apps-gitlab命名空间，可根据需要替换。

  kubectl create namespace apps-gitlab

• 创建TLS证书Secret

  从已准备好的证书key文件和crt文件创建Secret。

  kubectl create secret tls local-choral-io-tls -n apps-gitlab \
    --key=local.choral.io.key --cert=local.choral.io.crt

• 创建Traefik EntryPoint

  创建一个新的Traefik EntryPoint，用于提供对GitLab Shell SSH协议的访问。

  首先，导出当前Traefik部署的配置文件。

  helm get values --output yaml --namespace kube-system traefik > helm-traefik.yaml

  添加新的参数，更新Traefik部署。

  # ports.git-ssh.expose.default=false  禁用公开访问 稍后会手动创建用于访问该端口的负载均衡器
  # ports.git-ssh.port=8022             指定绑定端口
  # ports.git-ssh.protocol=TCP          指定绑定协议
  helm upgrade --install --namespace kube-system \
    --values helm-traefik.yaml \
    --set ports.git-ssh.expose.default=false \
    --set ports.git-ssh.port=8022 \
    --set ports.git-ssh.protocol=TCP \
    traefik traefik/traefik

• 创建负载均衡器

  创建一个新的LoadBalancer类型的Service，用于提供对22、80和443端口的访问。

  cat <<EOF | kubectl apply -f - > /dev/null
  apiVersion: v1
  kind: Service
  metadata:
    name: traefik-git
    namespace: kube-system
    labels:
      app.kubernetes.io/instance: traefik-kube-system
      app.kubernetes.io/name: traefik
  spec:
    type: LoadBalancer
    selector:
      app.kubernetes.io/instance: traefik-kube-system
      app.kubernetes.io/name: traefik
    ports:
    - name: ssh
      port: 22
      protocol: TCP
      targetPort: 8022
    - name: web
      port: 80
      protocol: TCP
      targetPort: web
    - name: websecure
      port: 443
      protocol: TCP
      targetPort: websecure
  EOF

安装GitLab

• 创建PostgreSQL密码Secret

  kubectl create secret -n apps-gitlab generic gitlab-postgresql-secret \
    --from-literal=postgresql-password=37Z8FeRZlkYuBtMWKtLsiLPz

• 创建MinIO认证凭据Secret

  创建gitlab-minio.yaml。

  provider: AWS
  region: cn-north-1
  aws_access_key_id: TL6JVVW85A9L4MFI4985
  aws_secret_access_key: ILnRAe8cuEJUmbCxTAPOodM3Rhu5gvD4ulZJskEL
  aws_signature_version: 4
  host: minio.local.choral.io
  endpoint: "https://minio.local.choral.io"
  path_style: true

  创建gitlab-minio.config。

  [default]
  host_base = minio.local.choral.io
  host_bucket = minio.local.choral.io
  use_https = True
  signature_v2 = False
  access_key = TL6JVVW85A9L4MFI4985
  secret_key = ILnRAe8cuEJUmbCxTAPOodM3Rhu5gvD4ulZJskEL
  bucket_location = cn-north-1
  multipart_chunk_size_mb = 128

  创建包含上述两个文件的Secret对象。

  kubectl create secret -n apps-gitlab generic gitlab-minio-secret \
    --from-file=connection=gitlab-minio.yaml \
    --from-file=config=gitlab-minio.config

• 创建所需的Buckets

  mc mb choral-local/gitlab-{artifacts,backups,backups-tmp,caches,dependency-proxy,gitlfs,packages,uploads}

  Bucket created successfully `choral-local/gitlab-artifacts`.
  Bucket created successfully `choral-local/gitlab-backups`.
  Bucket created successfully `choral-local/gitlab-backups-tmp`.
  Bucket created successfully `choral-local/gitlab-caches`.
  Bucket created successfully `choral-local/gitlab-dependency-proxy`.
  Bucket created successfully `choral-local/gitlab-gitlfs`.
  Bucket created successfully `choral-local/gitlab-packages`.
  Bucket created successfully `choral-local/gitlab-uploads`.

• 创建helm-gitlab.yaml

  registry:
    enabled: false
  postgresql:
    install: false
  certmanager:
    install: false
  prometheus:
    install: false
  nginx-ingress:
    enabled: false
  global:
    edition: ce
    time_zone: Asia/Shanghai
    kas:
      enabled: false
    minio:
      enabled: false
    hosts:
      https: true
      domain: local.choral.io
      gitlab:
        name: code.local.choral.io
    ingress:
      enabled: false
    psql:
      host: postgresql.data-postgresql
      database: gitlab
      username: gitlab
      password:
        secret: gitlab-postgresql-secret
        key: postgresql-password
    appConfig:
      lfs:
        bucket: gitlab-gitlfs
        connection:
          secret: gitlab-minio-secret
          key: connection
      artifacts:
        bucket: gitlab-artifacts
        connection:
          secret: gitlab-minio-secret
          key: connection
      uploads:
        bucket: gitlab-uploads
        connection:
          secret: gitlab-minio-secret
          key: connection
      packages:
        bucket: gitlab-packages
        connection:
          secret: gitlab-minio-secret
          key: connection
      dependencyProxy:
        enabled: true
        bucket: gitlab-dependency-proxy
        connection:
          secret: gitlab-minio-secret
          key: connection
      backups:
        bucket: gitlab-backups
        tmpBucket: gitlab-backups-tmp
      defaultProjectsFeatures:
        issues: true
        mergeRequests: true
        wiki: true
        snippets: true
        builds: true
        containerRegistry: false
  gitlab:
    webservice:
      registry:
        enabled: false
      resources:
        requests:
          cpu: 150m
    sidekiq:
      registry:
        enabled: false
      resources:
        requests:
          cpu: 200m
    toolbox:
      backups:
        objectStorage:
          config:
            secret: gitlab-minio-secret
            key: config
    gitaly:
      persistence:
        size: 20Gi
  gitlab-runner:
    runners:
      privileged: true
      config: |
        [[runners]]
          [runners.kubernetes]
            image = "debian:bullseye"
            privileged = true
            image_pull_secrets = []
          [runners.cache]
            Type = "s3"
            Path = "runners"
            Shared = true
            [runners.cache.s3]
              ServerAddress = "minio.local.choral.io"
              AccessKey = "TL6JVVW85A9L4MFI4985"
              SecretKey = "ILnRAe8cuEJUmbCxTAPOodM3Rhu5gvD4ulZJskEL"
              BucketName = "gitlab-caches"
              BucketLocation = "cn-north-1"
              Insecure = false
    gitlabUrl: http://gitlab-webservice-default:8181

• 安装GitLab

  使用上一步中创建的配置文件安装GitLab。

  helm upgrade --install gitlab --namespace apps-gitlab \
    --values helm-gitlab.yaml \
    gitlab/gitlab

• 配置Ingress入口规则

  创建用于提供http和https协议访问Ingress Route对象。

  cat <<EOF | kubectl apply -f - > /dev/null
  apiVersion: traefik.io/v1alpha1
  kind: IngressRoute
  metadata:
    name: gitlab-http-web
    namespace: apps-gitlab
  spec:
    entryPoints:
      - websecure
    routes:
      - match: Host(\`code.local.choral.io\`)
        kind: Rule
        services:
          - name: gitlab-webservice-default
            kind: Service
            port: 8181
    tls:
      secretName: local-choral-io-tls
  EOF

  创建用于提供SSH协议访问的Ingress Route TCP对象。

  cat <<EOF | kubectl apply -f - > /dev/null
  apiVersion: traefik.io/v1alpha1
  kind: IngressRouteTCP
  metadata:
    name: gitlab-ssh-shell
    namespace: apps-gitlab
  spec:
    entryPoints:
      - git-ssh
    routes:
      - match: HostSNI(\`*\`)
        services:
          - name: gitlab-gitlab-shell
            port: 22
  EOF

• 获取root用户初始密码

  kubectl get secret --namespace apps-gitlab gitlab-gitlab-initial-root-password \
    -o jsonpath="{.data.password}" | base64 -d

  获取密码之后可以使用root用户名和密码登录GitLab（本次演练中访问地址为https://code.local.choral.io/）。

参考资料

• https://docs.gitlab.com/charts/