Skip to content

Architecture

CentOS Stream pipeline

In this section we describe what steps are needed for an update to be shipped in CentOS Stream.

Let's imagine we would like to update a package python3.9 to a new version 3.9.17 in CentOS Stream 9.

  1. We create a merge request to the dist-git of the package:

    https://gitlab.com/redhat/centos-stream/rpms/python3.9/-/merge_requests/33

  2. Every merge request triggers a check pipeline on CentOS Stream Zuul CI:

    https://centos.softwarefactory-project.io/zuul/t/centos/buildset/a2e433d73301431690dbe4a87e85ea63

    The pipeline runs set of jobs, like mock-build and rpm-sti-test, and adds a comment to the merge request with the outcome of the test run.

  3. RHEL Engineer1 reviews the Merge Request and its test results and merges the Merge Request.

  4. RHEL Engineer builds a package in CentOS Stream Koji:

    https://kojihub.stream.centos.org/koji/buildinfo?buildID=34118

    After the build is completed it lands in the c9s-gate tag in that Koji instance.

  5. When CentOS package lands in c<N>s-gate tag, internally on Red Hat infrastructure the RHEL build from the same git commit is triggered. When that RHEL build passes automated testing and QE verification, the CentOS build is tagged into c<N>s-pending tag and appears in the CentOS Stream buildroot.

  6. Periodically a Jenkins job picks up the content of the c<N>s-pending tag from Koji and builds a so-called compose (repositories, iso-images, container-images) from it.

  7. Same Jenkins job runs a test of the compose using the t_functional test suite.

  8. If the test pass, the compose can then be published to mirrors.

Three gates

In the process above there are three decision making points which define whether the update will reach the end user. We will call those points "gates".

Three gates in the CentOS Stream pipeline

The same test scenario can be executed in any of the gates depending on the goals, resources and other considerations.

It is important to note that while update is going through the pipeline, it takes different forms: from a patch to sources in the Gitlab Merge request gate to a binary RPM in the Koji gate and then entire repository and images in the compose gate.

Therefore to test a change we need to run the test pipeline so that it can consume the correct type of artifact2. Additionally each gate has its own triggering event (gate entry) and an event, which happens when the change is approved (gate exit).

Merge Request gate

Artifact: patch to dist-git

Gate entry: merge request is created.

Gate exit: merge request is merged.

Koji gate

Artifact: koji build.

Gate entry: package is tagged to c<N>s-gate tag in CentOS Stream Koji

Gate exit: package is tagged to c<N>s-pending tag in CentOS Stream Koji

Compose gate

Artifact: compose.

Gate entry: new compose is built by the compose Jenkins job

Gate exit: compose is pushed to mirrors via Jenkins job - TBA

  1. Any RHEL Engineer has the ability to review and merge Gitlab merge requests, but usually each component(package) has a dedicated person or a team responsible for it. You can find who is responsible for accepting the change checking the Assignee field of the corresponding Jira or Bugzilla issue. ↩

  2. There is a work being done on unifying the artifacts between gates. The idea is to produce compose type of artifact in every gate, including MR and Koji gates, so that test pipelines can be reused between gates easily. ↩