Skip to main content

Google Cloud Functions deployments

This topic explains how to deploy new Cloud Functions to Google Cloud using Harness.

Supported versions

Harness integration with Google's serverless offering Google Functions.

Harness supports the following:

Harness Cloud Functions 1st gen support

Harness supports the following:

  • Basic deployments.
    • Harness deploys the new function and terminates the old one by sending 100% of traffic to the new function.
  • For rollback, Harness does not perform revision-based rollback. Instead, in case of deployment failure, Harness will take a snapshot of the last known good state of the function and reapply.

Harness Cloud Functions 2nd gen support

Harness supports the following:

  • Basic, blue green, and canary deployments.
  • Harness leverages the Cloud Run revisions capability that Google Cloud offers to configure rollback.

Cloud Functions limitations

  • For Google Cloud Functions 2nd gen, Harness does not support Google Cloud Source Repository at this time. Only Google Cloud Storage is supported.
  • For Google Cloud Functions 1st gen, Harness supports both Google Cloud Storage and Google Cloud Source.

Deployment summary

Here's a high-level summary of the setup steps.

Harness setup summary
  1. Create a Harness CD pipeline.
  2. Add a Deploy stage.
  3. Select the deployment type Google Cloud Functions, and then select Set Up Stage.
  4. Select Add Service.
    1. Add the function definition to the new Cloud Function service. You can paste in the YAML or link to a Git repo hosting the YAML.
    2. Save the new service.
  5. Select New Environment, name the new environment and select Save.
  6. In Infrastructure Definition, select New Infrastructure.
    1. In Google Cloud Provider Details, create or select the Harness GCP connector, GCP project, and GCP region, and select Save.
  7. Select Configure and select the deployment strategy: basic, canary, or blue green.
  8. Harness will automatically add the Deploy Cloud Function step. No further configuration is needed.
    1. For canary and blue green strategies, the Cloud Function Traffic Shift step is also added. In this step's Traffic Percent setting, enter the percentage of traffic to switch to the new revision of the function.
    2. For canary deployments, you can add multiple Cloud Function Traffic Shift steps to rollout the shift.
    3. For blue green deployments, you can simply use 100 in the Traffic Percent setting.
  9. Select Save, and then run the pipeline.

Cloud Functions permission requirements

note

Harness supports Google Cloud Functions 1st and 2nd gen. There are minor differences in the permissions required by each generation. For a detailed breakdown, go to Access control with IAM from Google.

The permissions listed below work for both 1st and 2nd gen.

When you set up a Harness GCP connector to connect Harness with your GCP account, the GCP IAM user or service account must have the appropriate permissions assigned to their account.

Cloud Functions minimum permissions

Cloud Functions supports the basic roles of Editor, Admin, Developer, and Viewer. For details, go to Cloud Functions IAM Roles.

You can use the Admin role or the permissions below.

Here are the minimum permissions required for deploying Cloud Functions.

  • Cloud Functions API:
    • cloudfunctions.functions.create: Allows the user to create new functions.
    • cloudfunctions.functions.update: Allows the user to update existing functions.
    • cloudfunctions.functions.list: Allows the user to list existing functions.
    • cloudfunctions.operations.get: Allows the user to get the status of a function deployment.
  • Cloud Run API:
    • run.services.create: Allows the user to create new Cloud Run services.
    • run.services.update: Allows the user to update existing Cloud Run services.
    • run.services.get: Allows the user to get information about a Cloud Run service.
    • run.revisions.create: Allows the user to create new revisions for a Cloud Run service.
    • run.revisions.get: Allows the user to get information about a revision of a Cloud Run service.
    • run.routes.create: Allows the user to create new routes for a Cloud Run service.
    • run.routes.get: Allows the user to get information about a route of a Cloud Run service.

Note that these are the minimum set of permissions required for deploying Cloud Functions and running them on Cloud Run via API. Depending on your use case, you may need additional permissions for other GCP services such as Cloud Storage, Pub/Sub, or Cloud Logging.

Also, note that in order to call the Cloud Functions and Cloud Run APIs, the user or service account must have appropriate permissions assigned for calling the APIs themselves. This may include permissions like cloudfunctions.functions.getIamPolicy, run.services.getIamPolicy, cloudfunctions.functions.testIamPermissions, run.services.testIamPermissions, and iam.serviceAccounts.actAs.

For details, go to Cloud Functions API IAM permissions and Cloud Functions Access control with IAM.

Harness will also pull the function ZIP file in your Google Cloud Storage.

For Google Cloud Storage (GCS), the following roles are required:

  • Storage Object Viewer (roles/storage.objectViewer)
  • Storage Object Admin (roles/storage.objectAdmin)

For more information, go to the GCP documentation about Cloud IAM roles for Cloud Storage.

Ensure the Harness delegate you have installed can reach storage.cloud.google.com and your GCR registry host name, for example gcr.io.

For Google Cloud Source, the following roles are required:

  • Source Repository Reader (roles/source.reader)
  • Source Repository Reader (roles/cloudsource.writer)

For more information, go to Cloud Source Repositories Access control with IAM.

Cloud Functions services

The Harness Cloud Functions service the following:

  • Function Definition:
    • You can enter the function manifest YAML in Harness for use a remote function manifest YAML in a Git repo.
  • Artifacts:
    • You add a connection to the function ZIP file in Google Cloud Storage (1st and 2nd gen) or Google Cloud Source (1st gen).
How the Function Definition and Artifact work together

The function manifest YAML in Function Definition and the function zip file in Artifacts work together to define and deploy a Google Cloud Function.

The function manifest YAML file is a configuration file that defines the function's name, runtime environment, entry point, and other configuration options. The YAML file is used to specify the function's metadata, but it does not include the function's code.

The function code is packaged in a zip file that contains the function's source code, dependencies, and any other necessary files. The zip file must be created based on the manifest file, which specifies the entry point and runtime environment for the function.

When you deploy a function using the manifest file, the Cloud Functions service reads the configuration options from the YAML file and uses them to create the function's deployment package. The deployment package includes the function code (from the zip file), the runtime environment, and any dependencies or configuration files specified in the manifest file.

Function Definition

The Function Definition is the parameter file you use to define your Google Function. The parameter file you add here maps to one Google Function.

To use Google Cloud Functions, in the Harness service, in Deployment Type, you select Google Cloud Functions.

Next, in Google Cloud Function Environment Version, you select 1st gen or 2nd gen.

1st gen function definition example

Here is a Function Definition example.

function:
name: my-function
runtime: python37
entryPoint: my_function
httpsTrigger:
securityLevel: SECURE_OPTIONAL

In httpsTrigger, you do not need to specify url.

2nd gen function definition examples

Here are some Function Definition examples.

Sample Google Function definition included in Harness
# The following are the minimum set of parameters required to create a Google Cloud Function.
# If you use a remove manifest, please ensure it includes all of these parameters.

function:
name: <functionName>
buildConfig:
runtime: nodejs18
entryPoint: helloGET
environment: GEN_2
function_id: <functionName>
Example 1: A basic Cloud Function that triggers on an HTTP request
function:
name: my-http-function
buildConfig:
runtime: nodejs14
entryPoint: myFunction
environment: GEN_2
function_id: my-http-function

Example 2: A Cloud Function that uses environment variables
function:
name: my-env-function
buildConfig:
runtime: python38
entryPoint: my_function
environment: GEN_2
function_id: my-env-function
environmentVariables:
MY_VAR: my-value

Example 3: A Cloud Function that uses Cloud Storage as a trigger
function:
name: my-storage-function
buildConfig:
runtime: go111
entryPoint: MyFunction
environment: GEN_2
function_id: my-storage-function
trigger:
eventType: google.storage.object.finalize
resource: projects/_/buckets/my-bucket
availableMemoryMb: 512
timeout: 180s

Example 4: A Cloud Function that uses service config and environment variables
function:
name: canaryDemo-<+env.name>
serviceConfig:
environment_variables:
MY_ENV_VAR: 'True'
GCF_FF_KEY: '<+env.variables.GCF_FF_KEY>'
buildConfig:
runtime: python39
entryPoint: hello_world
environment: GEN_2
function_id: canaryDemo-<+env.name>
Example 5: A Cloud Function that uses secret environment variables

This example uses the secret_environment_variables parameter. It has the information necessary to fetch the secret value from secret manager and expose it as an environment variable. For more information, go to SecretEnvVar in Google docs.

# The following are the minimum set of parameters required to create a Google Cloud Function.
# If you use a remove manifest, please ensure it includes all of these parameters.

function:
name: "my-secret-env-func"
description: "Using Secret Environment Variables"
region: "us-east1"
runtime: "nodejs16"
entryPoint: myFunction
max_instances: 1
eventTrigger:
event_type: "providers/cloud.pubsub/eventTypes/topic.publish"
resource: "projects/<project>/topics/<topic>"
environment_variables:
MY_ENV_VAR1: value1
MY_ENV_VAR2: value2
secret_environment_variables: [{ key: "MY_SECERT_ENV_VAR1", project_id: <project_id>, secret: "<secretName1>", version: "<secretVersion1>" }, { key: "MY_SECERT_ENV_VAR2", project_id: <project_id>, secret: "<secretName2>", version: "<secretVersion2>" }]

Artifacts

In Artifacts, you add the location of the function ZIP file in Google Cloud Storage that corresponds to the YAML manifest file you added in Function Definition.

You use a Harness GCP connector to connect to your Cloud Storage bucket. The GCP connector credentials should meet the requirements in Cloud Functions permission requirements.

Adding a Cloud Function service

Here's how you add a Harness Cloud Function service.

Here's a Cloud Functions service YAML example.

service:
name: helloworld
identifier: Google_Function
serviceDefinition:
type: GoogleCloudFunctions
spec:
manifests:
- manifest:
identifier: GoogleFunction
type: GoogleCloudFunctionDefinition
spec:
store:
type: Harness
spec:
files:
- /GoogleFunctionDefinition.yaml
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- spec:
connectorRef: gcp_connector
project: cd-play
bucket: cloud-functions-automation-bucket
artifactPath: helloworld
identifier: helloworld
type: GoogleCloudStorage

Using service variables in manifest YAML

Service variables are a powerful way to template your services or make them more dynamic.

In the Variables section of the service, you can add service variables and then reference them in any of the manifest YAML file you added to the service.

For example, you could create a variable named entryPoint for the manifest entryPoint setting and set its value as a fixed value, runtime input, or expression.

Next, in your manifest YAML file, you could reference the variable like this (see <+serviceVariables.entryPoint>):

Now, when you add that service to a pipeline, you will be prompted to enter a value for this variable in the pipeline Services tab. The value you provide is then used as the entryPoint in your manifest YAML.

Cloud Functions environments

The Cloud Function environment contains an Infrastructure Definition that identifies the GCP account, project, and region to use.

Here's an example of a Cloud Functions environment.

environment:
name: GCF
identifier: GCF
description: "GCF environment"
tags: {}
type: PreProduction
orgIdentifier: default
projectIdentifier: serverlesstest
variables: []

Define the infrastructure

You define the target infrastructure for your deployment in the Environment settings of the pipeline stage. You can define an environment separately and select it in the stage, or create the environment within the stage Environment tab.

There are two methods of specifying the deployment target infrastructure:

  • Pre-existing: the target infrastructure already exists and you simply need to provide the required settings.
  • Dynamically provisioned: the target infrastructure will be dynamically provisioned on-the-fly as part of the deployment process.

For details on Harness provisioning, go to Provisioning overview.

GCP connector

You will need a Harness GCP Connector with correct permissions to deploy Cloud Functions in GCP.

You can pick the same GCP connector you used in the Harness service to connect to Google Cloud Storage for the artifact, or create a new connector.

Pre-existing Functions infrastructure

Here's a YAML example of a Cloud Function infrastructure definition.

infrastructureDefinition:
name: dev
identifier: dev
description: "dev google cloud infrastructure"
tags: {}
orgIdentifier: default
projectIdentifier: serverlesstest
environmentRef: dev
deploymentType: GoogleCloudFunctions
type: GoogleCloudFunctions
spec:
connectorRef: gcp_connector
project: cd-play
region: us-central1
allowSimultaneousDeployments: false

Dynamically provisioned Functions infrastructure

note

Currently, the dynamic provisioning documented in this topic is behind the feature flag CD_NG_DYNAMIC_PROVISIONING_ENV_V2. Contact Harness Support to enable the feature.

Here is a summary of the steps to dynamically provision the target infrastructure for a deployment:

  1. Add dynamic provisioning to the CD stage:

    1. In a Harness Deploy stage, in Environment, enable the option Provision your target infrastructure dynamically during the execution of your Pipeline.

    2. Select the type of provisioner that you want to use.

      Harness automatically adds the provisioner steps for the provisioner type you selected.

    3. Configure the provisioner steps to run your provisioning scripts.

    4. Select or create a Harness infrastructure in Environment.

  2. Map the provisioner outputs to the Infrastructure Definition:

    1. In the Harness infrastructure, enable the option Map Dynamically Provisioned Infrastructure.
    2. Map the provisioning script/template outputs to the required infrastructure settings.

Supported provisioners

The following provisioners are supported for Google Functions deployments:

  • Terraform
  • Terragrunt
  • Terraform Cloud
  • Shell Script

Adding dynamic provisioning to the stage

To add dynamic provisioning to a Harness pipeline Deploy stage, do the following:

  1. In a Harness Deploy stage, in Environment, enable the option Provision your target infrastructure dynamically during the execution of your Pipeline.

  2. Select the type of provisioner that you want to use.

    Harness automatically adds the necessary provisioner steps.

  3. Set up the provisioner steps to run your provisioning scripts.

For documentation on each of the required steps for the provisioner you selected, go to the following topics:

Mapping provisioner output

Once you set up dynamic provisioning in the stage, you must map outputs from your provisioning script/template to specific settings in the Harness Infrastructure Definition used in the stage.

  1. In the same CD Deploy stage where you enabled dynamic provisioning, select or create (New Infrastructure) a Harness infrastructure.

  2. In the Harness infrastructure, in Select Infrastructure Type, select Google Cloud Platform if it is not already selected.

  3. In Google Cloud Provider Details, enable the option Map Dynamically Provisioned Infrastructure.

    A Provisioner setting is added and configured as a runtime input.

  4. Map the provisioning script/template outputs to the required infrastructure settings.

To provision the target deployment infrastructure, Harness needs specific infrastructure information from your provisioning script. You provide this information by mapping specific Infrastructure Definition settings in Harness to outputs from your template/script.

For Google Cloud Functions, Harness needs the following settings mapped to outputs:

  • Project
  • Region
note

Ensure the Project and Region settings are set to the Expression option.

For example, here's a snippet of a Terraform script that provisions the infrastructure for a Cloud Function deployment and includes the required outputs:


provider "google" {
credentials = file("<PATH_TO_YOUR_GCP_SERVICE_ACCOUNT_KEY_JSON>")
project = "<YOUR_GCP_PROJECT_ID>"
region = "us-central1" # Replace with your desired region
}

resource "google_project_service" "cloudfunctions" {
project = "<YOUR_GCP_PROJECT_ID>"
service = "cloudfunctions.googleapis.com"
}

resource "google_cloudfunctions_function" "my_function" {
name = "my-cloud-function"
description = "My Cloud Function"
runtime = "nodejs14"
trigger_http = true

available_memory_mb = 256
timeout = "60s"

# Optionally, you can define environment variables for your function
environment_variables = {
VAR1 = "value1"
VAR2 = "value2"
}
}

output "project_name" {
value = google_cloudfunctions_function.my_function.project
}

output "region_name" {
value = google_cloudfunctions_function.my_function.region
}

In the Harness Infrastructure Definition, you map outputs to their corresponding settings using expressions in the format <+provisioner.OUTPUT_NAME>, such as <+provisioner.project_name>.

Figure: Mapped outputs

Cloud Functions execution strategies

Harness ASG deployments support the following deployment strategies:

  • Basic
  • Canary
  • Blue Green

Basic

The basic deployment execution strategy uses the Deploy Cloud Function step. The Deploy Cloud Function step deploys the new function version and routes 100% of traffic over to the new function version.

YAML example of Deploy Cloud Function step
          execution:
steps:
- step:
name: Deploy Cloud Function
identifier: deployCloudFunction
type: DeployCloudFunction
timeout: 10m
spec: {}

Canary

Harness provides a step group to perform the canary deployment.

The step group consists of:

  • Deploy Cloud Function With No Traffic step: deploys the new function version but does not route any traffic over to the new function version.
  • First Cloud Function Traffic Shift step: routes 10% of traffic over to the new function version.
  • Second Cloud Function Traffic Shift step: routes 100% of traffic over to the new function version.
YAML example of canary deployment step group
          execution:
steps:
- stepGroup:
name: Canary Deployment
identifier: canaryDepoyment
steps:
- step:
name: Deploy Cloud Function With No Traffic
identifier: deployCloudFunctionWithNoTraffic
type: DeployCloudFunctionWithNoTraffic
timeout: 10m
spec: {}
- step:
name: Cloud Function Traffic Shift
identifier: cloudFunctionTrafficShiftFirst
type: CloudFunctionTrafficShift
timeout: 10m
spec:
trafficPercent: 10
- step:
name: Cloud Function Traffic Shift
identifier: cloudFunctionTrafficShiftSecond
type: CloudFunctionTrafficShift
timeout: 10m
spec:
trafficPercent: 100

Blue Green

Harness provides a step group to perform the blue green deployment.

The step group consists of:

  • Deploy Cloud Function With No Traffic step: deploys the new function version but does not route any traffic over to the new function version.
  • Cloud Function Traffic Shift step: routes 100% of traffic over to the new function version.

You can also route traffic incrementally using multiple Cloud Function Traffic Shift steps with gradually increasing routing percentages.

YAML example of blue green deployment step group
          execution:
steps:
- stepGroup:
name: Blue Green Deployment
identifier: blueGreenDepoyment
steps:
- step:
name: Deploy Cloud Function With No Traffic
identifier: deployCloudFunctionWithNoTraffic
type: DeployCloudFunctionWithNoTraffic
timeout: 10m
spec: {}
- step:
name: Cloud Function Traffic Shift
identifier: cloudFunctionTrafficShift
type: CloudFunctionTrafficShift
timeout: 10m
spec:
trafficPercent: 100

Rollbacks

If deployment failure occurs, the stage or step failure strategy is initiated. Typically, this runs the Rollback Cloud Function step in the Rollback section of Execution. Harness adds the Rollback Cloud Function step automatically.

The Harness rollback capabilities are based on the Google Cloud Function revisions available in Google Cloud.

If the function already exists (for example, revision 10) and a new revision is deployed (for example, revision 11) but a step after the deployment there is a failure and rollback is triggered, then Harness will deploy a new function revision (revision 12) but it will contain the artifact and metadata for revision 10.