Skip to main content

Build and push an artifact

Add a Build and Push step to your CI pipeline to build your codebase and then push the artifact, such as a Docker image, to a repo. The following repos are supported:

Harness CI also supports uploading any build artifacts to cloud storage:

Video summary

The following video provides a visual summary of the process described in this topic.

Add the Build and Push step​

These steps describe how to create a simple one-step Build stage that does not include testing. It builds the code in a build farm, and then pushes it to a Docker repo.

Kubernetes cluster build infrastructures

With Kubernetes cluster build infrastructures, Build and Push steps use kaniko. Other build infrastructures use drone-docker. Kaniko requires root access to build the Docker image. It doesn't support non-root users.

If your build runs as non-root (runAsNonRoot: true), and you want to run the Build and Push step as root, you can set Run as User to 0 on the Build and Push step to use the root user for that individual step only.

If your security policy doesn't allow running as root, go to Build and push with non-root users.

  1. Go to Pipelines and create a pipeline or edit an existing pipeline. If you're not familiar with creating pipelines, go to the CI pipeline creation overview.

  2. Configure the pipeline's codebase, if you have not already done so. For details, go to Create and configure a codebase.

    tip

    The codebase configuration specifies the repo to use for this pipeline. When you run the pipeline, you specify the specific branch or commit to use for that build.

  3. If your pipeline doesn't already have a Build stage, select Add Stage, and then select Build.

  4. On the Build stage's Infrastructure tab, configure the build infrastructure. For example, you can set up a Kubernetes cluster build infrastructure. For details about all infrastructure settings, go to CI Build stage settings.

  5. In the Build stage's Execution tab, select Add Step, and then select a Build and Push step from the Step Library.

    For all Build and Push steps, you select or create a connector for the target repo, add repo-specific information, and specify Dockerfile information. For information about each Build and Push step's settings, go to the topic that corresponds with your registry provider:

  6. Select Apply Changes to save the step, and then select Save to save the pipeline.

YAML example: CI pipeline with a Build and Push step

Here's a YAML example of a CI pipeline that has a Build stage with a Build and Push step:

pipeline:
  name: CI Quickstart
  identifier: CI_Quickstart
  properties:
    ci:
      codebase:
        connectorRef: account.CI_Quickstart
        repoName: goHelloWorldServer
        build: <+input>
  stages:
    - stage:
        name: Build Test and Push
        identifier: Build_Test_and_Push
        type: CI
        spec:
          cloneCodebase: true
          execution:
            steps:
              - step:
                  type: BuildAndPushDockerRegistry
                  name: Build and push image to Docker Hub
                  identifier: Build_and_push_image_to_Docker_Hub
                  spec:
                    connectorRef: account.Docker_Quickstart
                    repo: cretzman/ciquickstart
                    tags:
                      - <+pipeline.sequenceId>
          infrastructure:
            type: KubernetesDirect
            spec:
              connectorRef: account.cidelegate
              namespace: harness-delegate-uat
  projectIdentifier: CI_Quickstart
  orgIdentifier: default

Run the pipeline​

Select Run Pipeline to run your pipeline. Depending on your pipeline's codebase configuration, you may need to select a Git branch or tag to use for the build.

While the build runs, you can monitor the Build and Push step logs. For example, these are the logs for a step that pushed to a Docker repo in a pipeline using a Kubernetes cluster build infrastructure:

/kaniko/executor --dockerfile=Dockerfile --context=dir://. --destination=cretzman/ciquickstart:13
Retrieving image manifest alpine:3.12
Retrieving image alpine:3.12
Retrieving image manifest alpine:3.12
Retrieving image alpine:3.12
Built cross stage deps: map[]
Retrieving image manifest alpine:3.12
Retrieving image alpine:3.12
Retrieving image manifest alpine:3.12
Retrieving image alpine:3.12
Executing 0 build triggers
Unpacking rootfs as cmd ADD go-sample-app /bin/ requires it.
LABEL maintainer="John Doe <john.doe@example.com>"
Applying label maintainer=John Doe <john.doe@example.com>
Using files from context: [/step-exec/workspace/go-sample-app]
ADD go-sample-app /bin/
Taking snapshot of files...
ENTRYPOINT ["/bin/go-sample-app"]

If the build succeeds, you can find the pushed image in your image repo:

The build is also listed in your Harness project's Builds.

Useful techniques​

Here are some interesting ways you can use or enhance Build and Push steps.

Use Harness expressions for tags​

When you push an image to a repo, you tag the image so you can identify it later. For example, in one pipeline stage, you push the image, and, in a later stage, you use the image name and tag to pull it and run integration tests on it.

There are several ways to tag images, but Harness expressions can be useful.

For example, <+pipeline.sequenceId> is a built-in Harness expression that represents the Build Id number, for example 9.

After the pipeline runs, you can see the Build Id in the output.

The ID also appears as an image tag in your target image repo:

The Build Id tags an image that you pushed in an earlier stage of your pipeline. You can use the Build Id to pull the same image in later stages of the same pipeline. By using a variable expression, rather than a fixed value, you don't have to use the same image name every time.

For example, you can use the <+pipeline.sequenceId> expression as a variable tag to reference images in future pipeline stages by using syntax such as: harnessdev/ciquickstart:<+pipeline.sequenceId>.

As a more specific example, if you have a Background step in a later stage in your pipeline, you can use the <+pipeline.sequenceId> variable to identify the image without needing to call on a fixed value.

Build a Docker image without pushing​

You can use your CI pipeline to test a Dockerfile used in your codebase and verify that the resulting image is correct before you push it to your Docker repository.

  1. In your CI pipeline, go to the Build stage that includes the Build and Push an image to Docker Registry step.
  2. In the Build stage's Overview tab, expand the Advanced section.
  3. Select Add Variable and enter the following:
    • Name: PLUGIN_DRY_RUN
    • Type: String
    • Value: true
  4. Save and run the pipeline.

Build multi-architecture images​

To use a CI pipeline to build multi-architecture images, create a separate stage for building and pushing each architecture.

The following YAML example describes a multi-architecture pipeline with two stages. Both stages have similar components but they are slightly different according to the architecture of the image that the stage builds.

Each stage:

  • Uses a variation of a Kubernetes cluster build infrastructure.
  • Has a Run step that prepares the DockerFile.
  • Has a Build and Push step that builds and uploads the image.
YAML example: Multi-arch image pipeline
pipeline:
  allowStageExecutions: true
  projectIdentifier: my-project
  orgIdentifier: default
  tags:
    CI: ""
  properties:
    ci:
      codebase:
        connectorRef: CI_GitHub
        repoName: Automation.git
        build: <+input>
  stages:
    - stage:
        name: K8
        identifier: upload
        type: CI
        spec:
          cloneCodebase: true
          infrastructure:
            type: KubernetesDirect
            spec:
              connectorRef: K8Linux
              namespace: <+input>
              runAsUser: ""
              automountServiceAccountToken: true
              nodeSelector: {}
              containerSecurityContext:
                runAsUser: ""
              os: Linux
          execution:
            steps:
              - step:
                  type: Run
                  name: CreateDockerFile
                  identifier: CreateDockerFile
                  spec:
                    connectorRef: CI_Docker_Hub
                    image: alpine:latest
                    command: |-
                      touch harnessDockerfileui
                      cat > harnessDockerfileui <<- EOM
                      FROM alpine:latest AS dev-env
                      ARG foo
                      RUN echo "$foo bar"
                      ENTRYPOINT ["pwd"]

                      FROM alpine:latest AS release-env
                      ARG hello
                      RUN echo "$hello world"
                      ENTRYPOINT ["ls"]
                      EOM
                      cat harnessDockerfileui
                    resources:
                      limits:
                        memory: 100M
              - step:
                  type: BuildAndPushDockerRegistry
                  name: DockerPushStep
                  identifier: DockerPushStep
                  spec:
                    connectorRef: my-docker-hub
                    repo: my-repo/ciquickstart
                    tags:
                      - "1.0"
                    dockerfile: harnessDockerfileui
                    target: dev-env
                    resources:
                      limits:
                        memory: 100M
        variables: []
    - stage:
        name: K8s Linux arm
        identifier: CI_Golden_ARM
        type: CI
        spec:
          cloneCodebase: true
          infrastructure:
            type: KubernetesDirect
            spec:
              connectorRef: k8sarm
              namespace: ci-gold-arm-delegate
              automountServiceAccountToken: true
              tolerations:
                - effect: NoSchedule
                  key: kubernetes.io/arch
                  operator: Equal
                  value: arm64
              nodeSelector:
                kubernetes.io/arch: arm64
              os: Linux
          execution:
            steps:
              - step:
                  type: Run
                  name: CreateDockerFile
                  identifier: CreateDockerFile
                  spec:
                    connectorRef: CI_Docker_Hub
                    image: alpine:latest
                    command: |-
                      touch harnessDockerfileui
                      cat > harnessDockerfileui <<- EOM
                      FROM alpine:latest AS dev-env
                      ARG foo
                      RUN echo "$foo bar"
                      ENTRYPOINT ["pwd"]

                      FROM alpine:latest AS release-env
                      ARG hello
                      RUN echo "$hello world"
                      ENTRYPOINT ["ls"]
                      EOM
                      cat harnessDockerfileui
                    resources:
                      limits:
                        memory: 100M
              - step:
                  type: BuildAndPushDockerRegistry
                  name: DockerPushStep
                  identifier: DockerPushStep
                  spec:
                    connectorRef: my-docker-hub
                    repo: my-repo/ciquickstart
                    tags:
                      - "1.0"
                    dockerfile: harnessDockerfileui
                    target: dev-env
                    resources:
                      limits:
                        memory: 100M
        variables: []
  variables: []
  identifier: CI_MultiArch
  name: CI_MultiArch

Set kaniko runtime flags​

With Kubernetes cluster build infrastructures, Build and Push steps use kaniko. Other build infrastructures use drone-docker.

You can set kaniko runtime flags by adding stage variables formatted as PLUGIN_FLAG_NAME. For example, to set --skip-tls-verify, you would add a stage variable named PLUGIN_SKIP_TLS_VERIFY and set the variable value to true.

        variables:
          - name: PLUGIN_SKIP_TLS_VERIFY
            type: String
            description: ""
            required: false
            value: "true"