Skip to main content

Run multiple PostgreSQL instances in Background steps

This topic explains how to use Background steps to run multiple PostgreSQL instances in a Harness CI pipeline.

Requirements

Add Background steps

Add one Background step for each PostgreSQL service you need to run.

For the Background steps to run the PostgreSQL services, the build environment must have the necessary binaries. Depending on the stage's build infrastructure, Background steps can use binaries that exist in the build environment or pull an image, such as a public or private Docker image, that contains the required binaries. For more information about when and how to specify images, go to the Background step Container Registry and Image settings.

In Harness, go to the pipeline where you want to run the PostgreSQL services. In the CI stage, add one Background step for each PostgreSQL database, and group the steps to run in parallel. Required Background step settings depend on your build infrastructure.

            steps:
- parallel: ## Group the Background steps to run in parallel so all PostgreSQL services run simultaneously.
- step:
identifier: Background_1 ## You use the identifier to call background services in later steps in this stage.
type: Background
name: Background_1
spec:
shell: Sh
envVariables: ## Add environment variables for the PostgreSQL user name, database name, and password.
POSTGRES_USER: postgres
POSTGRES_DB: test
POSTGRES_PASSWORD: password ## For security, use an expression to reference a Harness text secret.
portBindings:
"5432": "5433" ## Specify the host port (5432) and the desired container port (such as 5433). Each Background step must have a different container port.
- step:
identifier: Background_2
type: Background
name: Background_2
spec:
shell: Sh
envVariables:
POSTGRES_USER: postgres
POSTGRES_DB: test1
POSTGRES_PASSWORD: password
portBindings: ## Each Background step must have a different container port.
"5432": "5434"
tip

For the POSTGRES_PASSWORD, you can use an expression to reference a Harness text secret.

Test the services

You can add a Run step to confirm your PostgreSQL services are running before running other steps that need to interact with those services.

For the Run step to run psql commands, the build environment must have the necessary binaries. Depending on the stage's build infrastructure, Run steps can use binaries that exist in the build environment or pull an image, such as a public or private Docker image, that contains the required binaries. For more information about when and how to specify images, go to the Run step Container Registry and Image settings.

In the same CI stage where you added the Background steps, add a Run step after the Background steps. Make sure the Run step isn't in the -parallel group. Required Run step settings depend on your build infrastructure.

              - step:
identifier: Run_1
type: Run
name: Run_1
spec:
shell: Sh
command: |- ## Provide the user name, database name, Background step identifier, and container port from each of your Background steps.
sleep 15
psql -U postgres -d test -h Background_1 -p 5433
psql -U postgres -d test1 -h Background_2 -p 5434

In command, you must provide values from your Background steps:

sleep 15
psql -U POSTGRES_USER -d FIRST_DATABASE_NAME -h FIRST_BACKGROUND_STEP_ID -p FIRST_BACKGROUND_STEP_CONTAINER_PORT
psql -U POSTGRES_USER -d SECOND_DATABASE_NAME -h SECOND_BACKGROUND_STEP_ID -p SECOND_BACKGROUND_STEP_CONTAINER_PORT
tip

For the POSTGRES_PASSWORD, you can use an expression to reference a Harness text secret.

After adding the Runstep, run your pipeline to test your PostgreSQL services. You can monitor and review build logs on the Build details page. Once you've confirmed that the services run as expected, you can configure the rest of your pipeline. Note that Background steps don't persist across stages.

Pipeline YAML examples

pipeline:
name: default
identifier: default
projectIdentifier: default
orgIdentifier: default
tags: {}
stages:
- stage:
identifier: stage
type: CI
name: stage
description: ""
spec:
cloneCodebase: false
execution:
steps:
- parallel: ## Group the Background steps to run in parallel so all PostgreSQL services run simultaneously.
- step:
identifier: Background_1 ## You use the identifier to call background services in later steps in this stage.
type: Background
name: Background_1
spec:
shell: Sh
envVariables: ## Add environment variables for the PostgreSQL user name, database name, and password.
POSTGRES_USER: postgres
POSTGRES_DB: test
POSTGRES_PASSWORD: password ## For security, use an expression to reference a Harness text secret.
portBindings:
"5432": "5433" ## Specify the host port (5432) and the desired container port (such as 5433). Each Background step must have a different container port.
- step:
identifier: Background_2
type: Background
name: Background_2
spec:
shell: Sh
envVariables:
POSTGRES_USER: postgres
POSTGRES_DB: test1
POSTGRES_PASSWORD: password
portBindings: ## Each Background step must have a different container port.
"5432": "5434"
- step:
identifier: Run_1
type: Run
name: Run_1
spec:
shell: Sh
command: |- ## Provide the user name, database name, Background step identifier, and container port from each of your Background steps.
sleep 15
psql -U postgres -d test -h Background_1 -p 5433
psql -U postgres -d test1 -h Background_2 -p 5434
platform:
os: Linux
arch: Amd64
runtime:
type: Cloud
spec: {}