Skip to main content

Shell Script Provisioning

Harness has first-class support for Terraform, Terragrunt, AWS CloudFormation, Azure ARM and Blueprint provisioners, but to support different provisioners, or your existing shell script implementations, Harness includes Shell Script provisioning.

This topic provides steps on using Harness to provision a target environment or resources using shell scripts.

Important notes​

  • Harness Shell Script provisioning supports the following deployment types: Physical Data Centers, AWS ECS, AWS Lambda, and Kubernetes on Google Cloud Platform (GKE).

Permissions in your scripts​

You need to give Harness permissions in your target environment so Harness can provision using you provisioner script.

You can add secret text and files to your script using Harness built-in secret manager or your own.

For details on using secret managers and secrets, go to:

Harness role permissions required​

  • Environments: View/Create, Edit, Access, Delete.

Shell Script provisioning summary​

Harness provisioning is categorized into the following use cases:

  • Ad hoc provisioning: temporary and on-demand provisioning of resources for specific tasks or purposes.
  • Dynamic infrastructure provisioning: provision the target deployment environment as part of the same deployment process. Typically, dynamic infrastructure provisioning is for temporary pre-production environments, such as dev, test, and qa. Production environments are usually pre-existing.

For details on Harness provisioning, go to Provisioning overview.

note

Currently, the dynamic provisioning documented in this topic is behind the feature flag CD_NG_DYNAMIC_PROVISIONING_ENV_V2. Contact Harness Support to enable the feature.

Dynamic provisioning steps for different deployment types​

Each of the deployment types Harness supports (Kubernetes, AWS ECS, etc.) require that you map different script outputs to the Harness infrastructure settings in the pipeline stage.

To see how to set up dynamic provisioning for each deployment type, go to the following topics:

$PROVISIONER_OUTPUT_PATH​

To understand Harness Shell Script provisioning, it is important to understand the Harness $PROVISIONER_OUTPUT_PATH environment variable.

The Harness $PROVISIONER_OUTPUT_PATH variable can be used in the Shell Script provisioning script whenever you need to use the JSON output from your script.

For example, here is a shell script that pulls EC2 instance information from AWS:


apt-get -y install awscli  
aws configure set aws_access_key_id $access_key  
aws configure set aws_secret_access_key $secret_key  
aws configure set region us-east-1  
aws ec2 describe-instances --filters Name=tag:Name,Values=harness-provisioner > "$PROVISIONER_OUTPUT_PATH"

This script returns a JSON array describing the instances.

The Harness environment variable $PROVISIONER_OUTPUT_PATH is initialized by Harness and stores the JSON collection returned by your script.

Now, in subsequent steps in your stage, you can use Harness expressions to reference labels in the JSON collection.

tip

Put quotes around $PROVISIONER_OUTPUT_PATH as a best practice. The quotes are only required if the value of the variable will have spaces in it, but they cause no problem in any case.

Here is the YAML for a pipeline that simulates Shell Script provisioning so you can experiment with using the $PROVISIONER_OUTPUT_PATH output variable.

Shell Script provisioning simulation

In the stage's Shell Script Provision step, you will see a JSON array echoed and stored in $PROVISIONER_OUTPUT_PATH.

In subsequent Shell Script steps, you will see the JSON output referenced using the output expressions for the Shell Script Provision step.

Copy and paste the following YAML into a Harness pipeline, replacing the projectIdentifier and orgIdentifier with your own settings and replacing the secret in the secretvar stage variable with your own text secret.


pipeline:
  name: Shell Script ad hoc
  identifier: Shell_Script_ad_hoc
  projectIdentifier: CD_Docs
  orgIdentifier: default
  tags: {}
  stages:
    - stage:
        name: shellscriptprovision
        identifier: shellscriptprovision
        description: ""
        type: Custom
        spec:
          execution:
            steps:
              - step:
                  type: ShellScriptProvision
                  name: shell1
                  identifier: shell1
                  spec:
                    source:
                      type: Inline
                      spec:
                        script: |-
                          echo start
                          echo "{\"Instances\":[{\"Hostname\": \"host1\",\"value\": \"<+stage.variables.testinstance>\"},{\"Hostname\": \"<+stage.variables.secretvar>\",\"value\": \"$test\"}]}" >> "$PROVISIONER_OUTPUT_PATH"
                          echo finish
                    environmentVariables:
                      - name: test
                        type: String
                        value: testvar
                  timeout: 10m
              - step:
                  type: ShellScript
                  name: test output
                  identifier: test_output
                  spec:
                    shell: Bash
                    onDelegate: true
                    source:
                      type: Inline
                      spec:
                        script: |-
                          echo <+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances>
                          if [ "<+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances[0].Hostname>" == "host1" ]; then
                              echo "first host match"
                              exit 0
                          else
                              echo "first host doesnt match"
                              exit 1
                          fi;
                    environmentVariables: []
                    outputVariables: []
                  timeout: 10m
                isNestedGroup: false
              - step:
                  type: ShellScript
                  name: test expression
                  identifier: test_expression
                  spec:
                    shell: Bash
                    onDelegate: true
                    source:
                      type: Inline
                      spec:
                        script: |-
                          echo <+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances>
                          if [ "<+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances[0].value>" == "testhostnamevalue" ]; then
                              echo "value match"
                              exit 0
                          else
                              echo "value doesnt match"
                              exit 1
                          fi;
                    environmentVariables: []
                    outputVariables: []
                  timeout: 10m
              - step:
                  type: ShellScript
                  name: test secret expression
                  identifier: test_secret_expression
                  spec:
                    shell: Bash
                    onDelegate: true
                    source:
                      type: Inline
                      spec:
                        script: |-
                          echo <+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances>
                          if [ "<+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances[1].Hostname>" == "secretHost" ]; then
                              echo "second host match"
                              exit 0
                          else
                              echo "second host doesnt match"
                              exit 1
                          fi;
                    environmentVariables: []
                    outputVariables: []
                  timeout: 10m
              - step:
                  type: ShellScript
                  name: test env var
                  identifier: test_env_var
                  spec:
                    shell: Bash
                    onDelegate: true
                    source:
                      type: Inline
                      spec:
                        script: |-
                          echo <+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances>
                          if [ "<+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances[1].value>" == "testvar" ]; then
                              echo "env var value match"
                              exit 0
                          else
                              echo "env var value doesnt match"
                              exit 1
                          fi;
                    environmentVariables: []
                    outputVariables: []
                  timeout: 10m
        tags: {}
        variables:
          - name: testinstance
            type: String
            description: ""
            value: testhostnamevalue
          - name: secretvar
            type: Secret
            description: ""
            value: shellScriptProvisionSecretHost


When you run this pipeline, the Shell Script Provision step show that the $PROVISIONER_OUTPUT_PATH environment variable has been initialized on the Harness delegate.

"PROVISIONER_OUTPUT_PATH" has been initialized to "/opt/harness-delegate/shellScriptProvisioner/F__QtYt1QdOyAc3QKUFIIA-zTte04H7SsG6iU9VIRxOAA-shell1/output.json"

When you select the Output tab of the executed Shell Script Provision step, you can see all of the outputs from the JSON. To reference an output, you simply copy its Output Name:

For example:

<+pipeline.stages.shellscriptprovision.spec.execution.steps.shell1.output.Instances[0].Hostname>

Using the expression in ad hoc and dynamic provisioning​

The Harness $PROVISIONER_OUTPUT_PATH variable can be used in the Shell Script Provision step when it is used for ad hoc or dynamic provisioning, but it is mandatory for dynamic provisioning and anytime you want to produce some output from the Shell Script Provision step script.

Create Shell Script Provision step​

The Shell Script Provision step runs the an inline or remote Bash script to provision your resources.

Script location​

You can add your provisioning script inline in the Shell Script Provision step or store it in the Harness File Store.

Script​

Your provisioning script can be used for ad hoc or dynamic provisioning.

For details on Harness provisioning, go to Provisioning overview.

If you want to use the outputs from your script, you need to use the Harness $PROVISIONER_OUTPUT_PATH environment variable.

For details on $PROVISIONER_OUTPUT_PATH, go to $PROVISIONER_OUTPUT_PATH in the current topic.

Script Input Variables​

While you can simply declare a variable in your script using a Harness expression or string for its value, using Input Variables provides some additional benefits:

You can more easily identify and manage the Harness expressions used in your script. You can template your script.

You can declare the variable using Name and Value in Script Input Variables and then reference the variable in the script just as you would any other variable: $var_name.

You can also use expressions in Value. For example, if you have an output variable from a previous Shell Script step, you can copy it from the executed step Outputs tab and use it in Value.

To use an expression, in Script Input Variables, select Expression and paste the expression in Value:

In the Script, you declare the variable using the Name from Script Input Variables (in this example, foo).

Rollback​

Shell Script provisioning in Harness does not include rollback steps like the Harness provisioning support using Terraform, etc.

For rollback, you must add your own scripts to the Rollback section of the stage Environment (dynamic provisioning) or Execution (ad hoc provisioning).

Ad hoc provisioning​

You can add the Shell Script Provision step to the Execution section of a CD Deploy stage. In the step, you add a script that provisions resources in your environment.

If you want to use the outputs from your script, you need to use the Harness $PROVISIONER_OUTPUT_PATH environment variable.

For details on $PROVISIONER_OUTPUT_PATH, go to $PROVISIONER_OUTPUT_PATH in the current topic.

Dynamic infrastructure provisioning​

note

Currently, the dynamic provisioning documented in this topic is behind the feature flag CD_NG_DYNAMIC_PROVISIONING_ENV_V2. Contact Harness Support to enable the feature.

For dynamically provisioning the target deployment environment, you add the Shell Script Provision step to the Environment section of the CD Deploy stage.

Next, you map specific outputs from your script to the required settings in the Infrastructure Definition for the stage.

Adding dynamic provisioning to the stage​

  1. In the CD Deploy stage, enable the Provision your target infrastructure dynamically during the execution of your Pipeline option.

  2. In What type of provisioner do you want to use?, select Script.

    The Shell Script Provision step is added.

  3. Configure the Shell Script Provision step with your provisioning script.

  4. For dynamic provisioning, you must map the outputs from your script to the Harness Infrastructure Definition for the stage. You must use the Harness $PROVISIONER_OUTPUT_PATH environment variable to capture the JSON output of your script.

    For details on $PROVISIONER_OUTPUT_PATH, go to $PROVISIONER_OUTPUT_PATH in the current topic.

Mapping script outputs to the Infrastructure Definition​

Once you have added dynamic provisioning to the Environment section of the stage, you must map specific script outputs to the required Harness Infrastructure Definition used by the stage.

The following table shows the Infrastructure Definition settings that are mapped to Shell Script provisioner outputs.

Infra typeInfra settings that require mapping
Kubernetes GCPNamespace, Cluster, Release Name (optional)
Physical Data CenterHost Array Path, Host Attributes
SSH and WinRM on AWSRegion, Tags
SSH and WinRM on AzureSubscription Id, Resource Group, Tags
AWS LambdaRegion
AWS ECSRegion, Cluster

Steps on mapping outputs for each deployment type are covered in their topics: