Skip to main content

CI pipeline creation overview

This topic provides an overview of CI pipeline creation and configuration, including common components, such as Build stages, steps, and codebases, as well as advanced settings.

For definitions of terms like stage, step, build infrastructure, delegate, connector, and so on, go to CI pipeline components.

Pipelines​

A CI pipeline is an end-to-end integration workflow that, in its simplest form, pulls a codebase, builds an artifact, and then uploads the artifact to storage or a registry such as Docker Hub, Google Cloud Registry, JFrog Artifactory, and many others. Pipelines are comprised of stages and steps.

You can run a pipeline manually or set up triggers to automatically run it on a schedule or when an event occurs, such as a Git merge in your codebase.

Create a pipeline in Harness CI
  1. In the Builds (Continuous Integration) modules, select Pipelines, and then select Create a Pipeline.
  2. Enter a Name for the pipeline. Description and Tags are optional.
  3. Select Inline to store your pipeline configuration in Harness, or select Remote to store your pipeline as code in a Git repository.
  4. If you want to use a pipeline template, select Start with Template.
  5. Select Start.

You can now add stages and steps to the pipeline, as well as configure pipeline settings. For a guided experience, try one of the CI tutorials.

tip

You can also import pipelines from Git.

Pipeline settings

In addition to a default codebase, the following settings are configurable at the pipeline level:

tip

Harness Input Sets are collections of runtime inputs for a pipeline run. Overlays are groups of Input Sets. Use Overlays to provide multiple Input Sets when you run a pipeline.

With Input Sets and Overlays, you can use the same pipeline for multiple scenarios. You can define each scenario in an Input Set or Overlay, and then select the appropriate scenario at runtime.

Stages​

A CI stage is a subset of a pipeline that contains one major segment of the CI workflow. All stages have stage settings and one or more steps.

To add a stage to a pipeline, select Add Stage in the Pipeline Studio. The most essential stage for CI pipelines is the Build stage, which includes steps that test code, build and push images, and upload artifacts, among other steps.

CI Build stage settings include codebase configuration, build infrastructure, shared paths, and other advanced settings.

tip

To make pipelines more versatile, you can create templates, use stage variables, and create custom stages, among other optimization techniques.

Codebases​

CI pipelines build and test code that is pulled from a Git code repository. When you add a Build stage to a CI pipeline, you specify the Git account and repository where your code is stored. The codebase declared in the first stage in a pipeline becomes the pipeline's default codebase. You can use this codebase for later stages in the pipeline, or you can use multiple codebases in the same pipeline. For more information, go to Configure codebases.

Harness uses code repo connectors to connect to Git repositories.

Build infrastructure​

All stages have an infrastructure definition, which represents the build infrastructure used by a CI pipeline: the target clusters, hosts, and so on. Build infrastructure components and specifications depend on the build infrastructure you choose. For more information, go to Which build infrastructure is right for me.

Tutorials

Shared Paths​

You can use Shared Paths in a stage to share data across steps or customize cache paths for Cache Intelligence.

When a pipeline runs, it creates a temporary volume called a workspace. During initialization, the stage clones your codebase to the root of the workspace. Then, the steps in the stage run inside the root. The workspace is the current working directory for each step in the stage. The workspace persists for the lifetime of the stage and enables steps in that stage to communicate and share state information. The default shared working directory for a stage is /harness. The workspace is destroyed when the stage ends.

Individual steps can communicate and share state using the workspace filesystem. The workspace is a volume, so filesystem changes persist throughout the stage lifetime. If you need to share additional volumes, you can add Shared Paths. Paths must begin with a forward slash, such as /vol.

For example, the maven m2 repo is stored in /root/.m2 by default. If your Build stage uses Maven, you can specify /root/.m2 as a Shared Path so that all steps in that stage can access that directory.

Steps​

A stage contains one or more steps. Each step is a series of commands that perform a task. For example, A Build and Push step builds an image and pushes it to a cloud repo, a Run step runs a series of shell commands, and so on. Go to the following documentation for details about how you can use the various CI steps in your pipelines:

Advanced stage and step settings​

Stages and steps have advanced settings to control the flow of operations.

Conditional Executions​

Use conditional execution settings to specify when a stage or step should run. For example, you can specify that a particular stage should run only if the prior pipeline or stage failed.

You can specify conditional execution settings for an entire stage and for individual steps. A stage's conditional execution settings apply to all steps in that stage that don't have their own step-level conditional execution settings. A step's conditional execution settings overrides the stage's conditional execution settings.

Looping Strategies​

For information about looping strategies to go:

Failure Strategies​

Failure strategies define how your stages and steps handle different failure conditions.

Each failure strategy is comprised of the following:

  • Error conditions that trigger the failure strategy.
  • Actions to take when the specified error conditions occur.

Failure strategies are a critical pipeline design component that determine what causes a stage or step to fail and what to do when a failure occurs.

See also:

Environment variables​

For information about environment variables, go to the CI environment variables reference.