[Gitlab] ce-14.10.5 Omnibus 버전 OCP에 설치

이번에 프로젝트에서 gitlab 업그레이드 부분이 생겼는데,

gitlab을 새로 올린 뒤 project 를 export->import로 진행하기로 했다. 

 

먼저 AS-IS 환경을 보자.

As-Is 환경

PaaS	Azure Redhat Openshift 4.2
Gitlab-ce	12.3.5
PostgreSQL	10
Redis	3.2.3-alpine

현재 gitlab은 15버전까지 출시되었으며,

stable 버전은

- 13.12

- 14.10

- 15.2

위 3개 정도로 gitlab site에서 확인해보았다. 

 

그래서 최신버전 바로 아래단계인 14버전대로 설치하기로 했다.

To-Be 환경

PaaS	Azure Redhat Openshift 4.10
Gitlab-ce	14.10.5
PostgreSQL	13.7
Redis	6.2.6-alpine

변경사항

gitlab 버전이 상승되면서 생긴 변경사항은

 

GitLab 14 specific changes | GitLab

 

GitLab installation minimum requirements | GitLab

위 페이지를 확인하며 찾아보았는데,

Unicorn이 puma로 변경되었고,

권장되는 Database version과 Redis version이 업그레이드 됬다.

 

Redis Docks Image 체크

redis Tags | Docker Hub

 

Gitlab-ce Docker Image 체크

Docker Hub

 

PostgreSQL OCP Registry Image 체크

redhat에서 postgresql 관련 이미지가 많아 체크해보았다.

 

$ oc describe is postgresql -n openshift
Name:                   postgresql
Namespace:              openshift
Created:                8 hours ago
Labels:                 samples.operator.openshift.io/managed=true
Annotations:            openshift.io/display-name=PostgreSQL
                        openshift.io/image.dockerRepositoryCheck=2022-08-17T00:35:46Z
                        samples.operator.openshift.io/version=4.10.20
Image Repository:       image-registry.openshift-image-registry.svc:5000/openshift/postgresql
Image Lookup:           local=false
Unique Images:          6
Tags:                   9

…
13-el8 (latest)
  tagged from registry.redhat.io/rhel8/postgresql-13:latest
    prefer registry pullthrough when referencing this tag

  Provides a PostgreSQL 13 database on RHEL 8. For more information about using this database image, including OpenShift considerations, see https://github.com/sclorg/postgresql-container/blob/master/README.md.
  Tags: database, postgresql

  * registry.redhat.io/rhel8/postgresql-13@sha256:32f1d6b012ed6c5163a390ba7ff66e01894004674b727744760ffb68bb044d74

template.yaml 작성

apiVersion: v1
items:
  - apiVersion: template.openshift.io/v1
    kind: Template
    labels:
      createdBy: gitlab-ce-template
    metadata:
      annotations:
        description:
          "GitLab. Collaboration and source control management: code, test,
          and deploy together! \n\n GitLab requries that the serviceaccount for the
          main GitLab app be added to the anyuid security context. The service account
          name is: <application name>-user"
        iconClass: icon-gitlab
        tags: instant-app,gitlab,VCS,ci-cd
      name: gitlab-ce
    objects:
      - apiVersion: v1
        kind: ImageStream
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}
        spec:
          tags:
            - from:
                kind: DockerImage
                name: gitlab/gitlab-ce:14.10.5-ce.0
              name: gitlab-14.10.5
      - apiVersion: v1
        kind: ImageStream
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}-redis
        spec:
          tags:
            - from:
                kind: DockerImage
                name: redis:6.2.6-alpine
              name: 6.2.6
      - apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: ${APPLICATION_NAME}-user
      - apiVersion: v1
        kind: DeploymentConfig
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}
        spec:
          replicas: 0
          selector:
            app: ${APPLICATION_NAME}
            deploymentconfig: ${APPLICATION_NAME}
          strategy:
            recreateParams: {}
            resources: {}
            type: Recreate
          template:
            metadata:
              labels:
                app: ${APPLICATION_NAME}
                deploymentconfig: ${APPLICATION_NAME}
            spec:
              containers:
                - env:
                    - name: GITLAB_OMNIBUS_CONFIG
                      value:
                        hostname='${APPLICATION_HOSTNAME}'; external_url "http://#{hostname}/"
                        unless hostname.to_s == ''; root_pass='${GITLAB_ROOT_PASSWORD}'; gitlab_rails['initial_root_password']=root_pass
                        unless root_pass.to_s == ''; postgresql['enable']=false; gitlab_rails['db_host']
                        = '${APPLICATION_NAME}-postgresql'; gitlab_rails['db_password']='${POSTGRESQL_PASSWORD}';
                        gitlab_rails['db_username']='${POSTGRESQL_USER}'; gitlab_rails['db_database']='${POSTGRESQL_DATABASE}';
                        redis['enable'] = false; gitlab_rails['redis_host']='${APPLICATION_NAME}-redis';
                        puma['worker_processes'] = ${PUMA_WORKERS}; manage_accounts['enable']
                        = true; manage_storage_directories['manage_etc'] = false; gitlab_shell['auth_file']
                        = '/gitlab-data/ssh/authorized_keys'; git_data_dirs({ 'default' =>
                        { 'path' => '/gitlab-data/git-data' } }); gitlab_rails['shared_path']
                        = '/gitlab-data/shared'; gitlab_rails['uploads_directory'] = '/gitlab-data/uploads';
                        gitlab_ci['builds_directory'] = '/gitlab-data/builds'; prometheus_monitoring['enable']
                        = false;
                  image: gitlab-ce
                  imagePullPolicy: IfNotPresent
                  livenessProbe:
                    failureThreshold: 3
                    httpGet:
                      path: /help
                      port: 80
                      scheme: HTTP
                    initialDelaySeconds: 900
                    periodSeconds: 10
                    successThreshold: 1
                    timeoutSeconds: 1
                  name: gitlab-ce
                  ports:
                    - containerPort: 22
                      protocol: TCP
                    - containerPort: 80
                      protocol: TCP
                  readinessProbe:
                    failureThreshold: 3
                    httpGet:
                      path: /help
                      port: 80
                      scheme: HTTP
                    initialDelaySeconds: 900
                    periodSeconds: 10
                    successThreshold: 1
                    timeoutSeconds: 1
                  resources:
                    limits:
                      cpu: 4
                      memory: ${MEMORY_LIMIT}
                    requests:
                      cpu: ${CPU_REQUEST}
                      memory: 1Gi
                  terminationMessagePath: /dev/termination-log
                  volumeMounts:
                    - mountPath: /etc/gitlab
                      name: gitlab-ce-volume-1
                    - mountPath: /gitlab-data
                      name: gitlab-ce-volume-2
              dnsPolicy: ClusterFirst
              restartPolicy: Always
              serviceAccount: ${APPLICATION_NAME}-user
              terminationGracePeriodSeconds: 30
              volumes:
                - name: gitlab-ce-volume-1
                  persistentVolumeClaim:
                    claimName: ${APPLICATION_NAME}-etc
                - name: gitlab-ce-volume-2
                  persistentVolumeClaim:
                    claimName: ${APPLICATION_NAME}-data
          test: false
          triggers:
            - type: ConfigChange
            - imageChangeParams:
                automatic: true
                containerNames:
                  - gitlab-ce
                from:
                  kind: ImageStreamTag
                  name: ${APPLICATION_NAME}:gitlab-14.10.5
              type: ImageChange
      - apiVersion: v1
        kind: DeploymentConfig
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}-redis
        spec:
          replicas: 1
          selector:
            app: ${APPLICATION_NAME}
            deploymentconfig: ${APPLICATION_NAME}-redis
          strategy:
            recreateParams: {}
            resources: {}
            type: Recreate
          template:
            metadata:
              labels:
                app: ${APPLICATION_NAME}
                deploymentconfig: ${APPLICATION_NAME}-redis
            spec:
              containers:
                - args:
                    - exec redis-server
                  command:
                    - /bin/sh
                    - -ec
                  image: gitlab-ce-redis
                  imagePullPolicy: IfNotPresent
                  name: gitlab-ce-redis
                  ports:
                    - containerPort: 6379
                      protocol: TCP
                  resources:
                    limits:
                      cpu: "1"
                      memory: 512Mi
                    requests:
                      cpu: 100m
                      memory: 300Mi
                  terminationMessagePath: /dev/termination-log
                  volumeMounts:
                    - mountPath: /data
                      name: gitlab-ce-volume-4
              dnsPolicy: ClusterFirst
              restartPolicy: Always
              terminationGracePeriodSeconds: 30
              volumes:
                - name: gitlab-ce-volume-4
                  persistentVolumeClaim:
                    claimName: ${APPLICATION_NAME}-redis-data
          test: false
          triggers:
            - type: ConfigChange
            - imageChangeParams:
                automatic: true
                containerNames:
                  - gitlab-ce-redis
                from:
                  kind: ImageStreamTag
                  name: ${APPLICATION_NAME}-redis:6.2.6
              type: ImageChange
      - apiVersion: v1
        kind: DeploymentConfig
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}-postgresql
        spec:
          replicas: 1
          selector:
            app: ${APPLICATION_NAME}
            deploymentconfig: ${APPLICATION_NAME}-postgresql
          strategy:
            type: Recreate
          template:
            metadata:
              labels:
                app: ${APPLICATION_NAME}
                deploymentconfig: ${APPLICATION_NAME}-postgresql
            spec:
              containers:
                - env:
                    - name: POSTGRESQL_USER
                      value: ${POSTGRESQL_USER}
                    - name: POSTGRESQL_PASSWORD
                      value: ${POSTGRESQL_PASSWORD}
                    - name: POSTGRESQL_DATABASE
                      value: ${POSTGRESQL_DATABASE}
                    - name: POSTGRESQL_ADMIN_PASSWORD
                      value: ${POSTGRESQL_ADMIN_PASSWORD}
                  image: gitlab-ce-postgresql
                  imagePullPolicy: IfNotPresent
                  livenessProbe:
                    initialDelaySeconds: 30
                    tcpSocket:
                      port: 5432
                    timeoutSeconds: 1
                  name: gitlab-ce-postgresql
                  ports:
                    - containerPort: 5432
                      protocol: TCP
                  readinessProbe:
                    exec:
                      command:
                        - /bin/sh
                        - -i
                        - -c
                        - psql -h 127.0.0.1 -U $POSTGRESQL_USER -q -d $POSTGRESQL_DATABASE
                          -c 'SELECT 1'
                    initialDelaySeconds: 5
                    timeoutSeconds: 1
                  resources:
                    limits:
                      cpu: "1"
                      memory: 512Mi
                    requests:
                      cpu: "1"
                      memory: 512Mi
                  terminationMessagePath: /dev/termination-log
                  volumeMounts:
                    - mountPath: /var/lib/pgsql/data
                      name: gitlab-ce-volume-3
              dnsPolicy: ClusterFirst
              restartPolicy: Always
              serviceAccount: default
              terminationGracePeriodSeconds: 30
              volumes:
                - name: gitlab-ce-volume-3
                  persistentVolumeClaim:
                    claimName: ${APPLICATION_NAME}-postgresql
          test: false
          triggers:
            - type: ConfigChange
            - imageChangeParams:
                automatic: true
                containerNames:
                  - gitlab-ce-postgresql
                from:
                  kind: ImageStreamTag
                  name: postgresql:13-el8
                  namespace: openshift
              type: ImageChange
      - apiVersion: v1
        kind: Service
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}
        spec:
          ports:
            - name: 22-ssh
              port: 22
              protocol: TCP
              targetPort: 22
            - name: 80-http
              port: 80
              protocol: TCP
              targetPort: 80
          selector:
            app: ${APPLICATION_NAME}
            deploymentconfig: ${APPLICATION_NAME}
          sessionAffinity: None
          type: ClusterIP
      - apiVersion: v1
        kind: Service
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}-redis
        spec:
          ports:
            - name: 6379-redis
              port: 6379
              protocol: TCP
              targetPort: 6379
          selector:
            app: ${APPLICATION_NAME}
            deploymentconfig: ${APPLICATION_NAME}-redis
          sessionAffinity: None
          type: ClusterIP
      - apiVersion: v1
        kind: Service
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}-postgresql
        spec:
          ports:
            - name: 5432-postgresql
              port: 5432
              protocol: TCP
              targetPort: 5432
          selector:
            app: ${APPLICATION_NAME}
            deploymentconfig: ${APPLICATION_NAME}-postgresql
          sessionAffinity: None
          type: ClusterIP
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: ${APPLICATION_NAME}-redis-data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: ${REDIS_VOL_SIZE}
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: ${APPLICATION_NAME}-etc
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: ${ETC_VOL_SIZE}
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: ${APPLICATION_NAME}-data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: ${GITLAB_DATA_VOL_SIZE}
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: ${APPLICATION_NAME}-postgresql
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: ${POSTGRESQL_VOL_SIZE}
      - apiVersion: v1
        kind: Route
        metadata:
          labels:
            app: ${APPLICATION_NAME}
          name: ${APPLICATION_NAME}
        spec:
          host: ${APPLICATION_HOSTNAME}
          port:
            targetPort: 80-http
          to:
            kind: Service
            name: ${APPLICATION_NAME}
    parameters:
      - description:
          The name for the application. The service will be named like the
          application.
        displayName: Application name.
        name: APPLICATION_NAME
        value: gitlab-ce
      - description:
          Hostname for service routes. Set this in order to have the GitLab
          display the correct clone urls.
        displayName: Gitlab instance hostname
        name: APPLICATION_HOSTNAME
        required: true
      - description: Maximum amount of memory the gitlab-ce container can use.
        displayName: Memory Limit
        name: MEMORY_LIMIT
        required: true
        value: 8G
      - description: Request amount of cpu the gitlab-ce container can use.
        displayName: Cpu Request
        name: CPU_REQUEST
        required: true
        value: "500m"
      - description:
          Password for the GitLab 'root' user. Must be at least 8 characters
          long. Leave blank if you would rather configure the password using the website
          during first use.
        displayName: GitLab Root User Password
        name: GITLAB_ROOT_PASSWORD
      - description:
          Username for PostgreSQL user that will be used for accessing the
          database.
        displayName: PostgreSQL User
        from: user[A-Z0-9]{3}
        generate: expression
        name: POSTGRESQL_USER
        required: true
      - description: Password for the PostgreSQL user.
        displayName: PostgreSQL Password
        from: "[a-zA-Z0-9]{16}"
        generate: expression
        name: POSTGRESQL_PASSWORD
        required: true
      - description: Password for the PostgreSQL Admin user.
        displayName: PostgreSQL Admin User Password
        from: "[a-zA-Z0-9]{16}"
        generate: expression
        name: POSTGRESQL_ADMIN_PASSWORD
        required: true
      - description: Name of the PostgreSQL database accessed.
        displayName: PostgreSQL Database Name
        name: POSTGRESQL_DATABASE
        required: true
        value: gitlabhq_production
      - description: Number of puma Workers to use per instance. Must be at least 2.
        displayName: Number of puma Workers
        name: PUMA_WORKERS
        required: true
        value: "2"
      - description: Volume size for /etc
        displayName: /etc/gitlab volume size
        name: ETC_VOL_SIZE
        value: 100Mi
      - description: Volume size for GitLab data
        displayName: GitLab data volume size
        name: GITLAB_DATA_VOL_SIZE
        value: 5Gi
      - description: Volume size for postgresql data
        displayName: postgresql volume size
        name: POSTGRESQL_VOL_SIZE
        value: 2Gi
      - description: Volume size for redis data
        displayName: redis volume size
        name: REDIS_VOL_SIZE
        value: 512Mi
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

템플릿을 작성하고 oc 명령어로 등록한다.

$ oc create -f gitlab-ce-tamlplate.yaml -n gitlab

설치

템플릿 설치

ocp콘솔에서 개발자모드로 진입 후 +Add

Developer Catalog > All Services > gitlab-ce 검색

설치대상 namespace, application-name, domain 등 을 입력한다.

application-name에 입력한 값은 <입력값>-user 라는 서비스어카운트를 생성하게 된다. 

Create 버튼을 누르면 redis와 postgresql pod가 기동된다.

 

ServiceAccount 권한부여

gitlab-ce pod는 root로 기동되기 때문에 ocp에서 anyuid 권한을 부여해야 컨테이너를 돌릴 수 있다.

$ oc adm policy add-scc-to-user anyuid -z gitlab-ce-user -n <네임스페이스>
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "gitlab-ce-user"

 

postgresql 패키지 설치

gitlab 가이드 중 성능을 위한 패키지 설치를 권고하고 있다.

postgresql 컨테이너에 접속한 뒤 명령어를 실행시켜주자.

sh-4.4$ psql
psql (13.7)
Type "help" for help.

postgres=# create extension pg_trgm;
CREATE EXTENSION
postgres=# create extension btree_gist;
CREATE EXTENSION
postgres=# \dx
                                     List of installed extensions
    Name    | Version |   Schema   |                            Description                            
------------+---------+------------+-------------------------------------------------------------------
 btree_gist | 1.5     | public     | support for indexing common datatypes in GiST
 pg_trgm    | 1.5     | public     | text similarity measurement and index searching based on trigrams
 plpgsql    | 1.0     | pg_catalog | PL/pgSQL procedural language
(3 rows)

 

gitlab-ce pod 기동

postgresql, redis pod가 정상적으로 기동되고 있다면 gitlab-ce pod를 기동하자.

$ oc scale --replicas=1 dc/gitlab-ce

 

완료

gitlab-ce 14버전이 정상적으로 기동되는데까지 약 10분 정도 걸렸다.

메모리 사용률은 가만히 있어도 약 2.7기가..

(추가적으로 puma process 2개를 사용하려면 권장 cpu:4core 이다.)

 

다음엔 백업&복원 가이드를 올리겠다.