Skip to main content

Optimize Cloud Costs for Kubernetes

Background on Cloud Costs​

As engineers, we are natural optimizers. Responding to users in more expeditious and efficient ways are core to optimizations. There are always tradeoffs between architecture and costs. With the rise of public cloud consumption with on the surface “nearly infinite” resources, this unlocks the next generation of distributed architecture. Though this is not without cost. From an engineering perspective, cost/billing data from the public cloud vendors depending on your organization might not be available to you or can take multiple billing cycles to be disseminated.

Resources on the public cloud are certainly not free. Public cloud vendors not only bill for the core infrastructure but also depending on the services leveraged could be billing dimensions for the control/management planes and non-obvious dimensions such as network/data transmissions. When capacity planning for infrastructure, a natural inclination is to err on the side of caution and provision extra capacity in case of a spike.

Optimizing costs focuses on right sizing resource usage to match workload actuals. Coupled with finding more emphermial infrastructure e.g spot instances can reduce costs. Monitoring solutions typically monitor for when usage exceeds a certain threshold, but the inverse if usage is under utilized, traditional monitoring solutions might not alert on that. Harness Cloud Cost Management, or CCM, can help unlock insights based on usage in how to optimize costs. This example will connect Kubernetes workload(s) on a public cloud vendor to Harness CCM to start getting cost insights and recommendations.

Get Started with Optimizing Your Kubernetes Cloud Costs​

By connecting your public cloud Kubernetes cluster to Harness, you can start to get recommendations on right-sizing the workloads that are running. This example is assuming there are some workloads running in the Kubernetes cluster.

Overview

To get actual costing data from your public cloud vendor, it is recommended to connect your public cloud billing API [usage report, billing export, etc] to Harness CCM. This is not a requirement. Harness CCM will poll usage data from the Kubernetes Metric Server which comes installed by default with GKE and AKS. If using EKS, installing the Kubernetes Metric Server is needed.

Prepare Your Kubernetes Cluster for Optimization Recommendations​

If you have not already, make sure to sign up for a Harness CCM Account. Once signed up, to start receiving optimizations for your cluster, you will need to install a Harness Delegate, e.g the Harness worker node, into your cluster.

Install Delegate​

You will also need to wire in a Kubernetes Delegate if you have not done so already.

Install Delegate

What is Harness Delegate?​

Harness Delegate is a lightweight worker process that is installed on your infrastructure and communicates only via outbound HTTP/HTTPS to the Harness Platform. This enables the Harness Platform to leverage the delegate to execute the CI/CD and other tasks on your behalf, without any of your secrets leaving your network.

You can install the Harness Delegate on either Docker or Kubernetes.

Install Harness Delegate​

Create a new delegate token​

Log in to the Harness Platform and go to Account Settings -> Account Resources -> Delegates. Select the Tokens tab. Select +New Token, and enter a token name, for example firstdeltoken. Select Apply. Harness Platform generates a new token for you. Select Copy to copy and store the token in a temporary file. You will provide this token as an input parameter in the next installation step. The delegate will use this token to authenticate with the Harness Platform.

Get your Harness account ID​

Along with the delegate token, you will also need to provide your Harness accountId as an input parameter during delegate installation. This accountId is present in every Harness URL. For example, in the following URL:

https://app.harness.io/ng/#/account/6_vVHzo9Qeu9fXvj-AcQCb/settings/overview

6_vVHzo9Qeu9fXvj-AcQCb is the accountId.

Now you are ready to install the delegate on either Docker or Kubernetes.

Prerequisite​

Ensure that you have access to a Kubernetes cluster. For the purposes of this tutorial, we will use minikube.

Install minikube

  • On Windows: 
choco install minikube
  • On macOS: 
brew install minikube

Now start minikube with the following config.

minikube start --memory 4g --cpus 4

Validate that you have kubectl access to your cluster.

kubectl get pods -A

Now that you have access to a Kubernetes cluster, you can install the delegate using any of the options below.

Install the Helm chart​

As a prerequisite, you must have Helm v3 installed on the machine from which you connect to your Kubernetes cluster.

You can now install the delegate using the delegate Helm chart. First, add the harness-delegate Helm chart repo to your local Helm registry.

helm repo add harness-delegate https://app.harness.io/storage/harness-download/delegate-helm-chart/
helm repo update
helm search repo harness-delegate

We will use the harness-delegate/harness-delegate-ng chart in this tutorial.

NAME                                    CHART VERSION   APP VERSION DESCRIPTION                                
harness-delegate/harness-delegate-ng    1.0.8           1.16.0      A Helm chart for deploying harness-delegate

Now we are ready to install the delegate. The following example installs/upgrades firstk8sdel delegate (which is a Kubernetes workload) in the harness-delegate-ng namespace using the harness-delegate/harness-delegate-ng Helm chart.

To install the delegate, do the following:

  1. In Harness, select Deployments, then select your project.

  2. Select Delegates under Project Setup.

  3. Select Install a Delegate to open the New Delegate dialog.

  4. Select Helm Chart under Install your Delegate.

  5. Copy the helm upgrade command.

  6. Run the command.

The command uses the default values.yaml located in the delegate-helm-chart GitHub repo. If you want change one or more values in a persistent manner instead of the command line, you can download and update the values.yaml file as per your need. You can use the updated values.yaml file as shown below.

helm upgrade -i firstk8sdel --namespace harness-delegate-ng --create-namespace \
  harness-delegate/harness-delegate-ng \
  -f values.yaml \
  --set delegateName=firstk8sdel \
  --set accountId=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
  --set delegateToken=PUT_YOUR_DELEGATE_TOKEN_HERE \
  --set managerEndpoint=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE \
  --set delegateDockerImage=harness/delegate:23.02.78306 \
  --set replicas=1 --set upgrader.enabled=false

Deploy using a custom role​

During delegate installation, you have the option to deploy using a custom role. To use a custom role, you must edit the delegate YAML file.

Harness supports the following custom roles:

  • cluster-admin
  • cluster-viewer
  • namespace-admin
  • custom cluster roles

To deploy using a custom cluster role, do the following:

  1. Open the delegate YAML file in your text editor.

  2. Add the custom cluster role to the roleRef field in the delegate YAML.

    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: harness-delegate-cluster-admin
    subjects:
      - kind: ServiceAccount
        name: default
        namespace: harness-delegate-ng
    roleRef:
      kind: ClusterRole
      name: cluster-admin
      apiGroup: rbac.authorization.k8s.io
    ---

    In this example, the cluster-admin role is defined.

  3. Save the delegate YAML file.

Verify delegate connectivity​

Select Continue. After the health checks pass, your delegate is available for you to use. Select Done and verify your new delegate is listed.

Helm chart & Terraform Helm provider​

Delegate Available

Kubernetes manifest​

Delegate Available

Docker​

Delegate Available

You can now route communication to external systems in Harness connectors and pipelines by selecting this delegate via a delegate selector.

Delegate selectors do not override service infrastructure connectors. Delegate selectors only determine the delegate that executes the operations of your pipeline.

Troubleshooting​

The delegate installer provides troubleshooting information for each installation process. If the delegate cannot be verified, select Troubleshoot for steps you can use to resolve the problem. This section includes the same information.

Harness asks for feedback after the troubleshooting steps. You are asked, Did the delegate come up?

If the steps did not resolve the problem, select No, and use the form to describe the issue. You'll also find links to Harness Support and to Delegate docs.

Use the following steps to troubleshoot your installation of the delegate using Helm.

  1. Verify that Helm is correctly installed:

    Check for Helm:

    helm

    And then check for the installed version of Helm:

    helm version

    If you receive the message Error: rendered manifests contain a resource that already exists..., delete the existing namespace, and retry the Helm upgrade command to deploy the delegate.

    For further instructions on troubleshooting your Helm installation, go to Helm troubleshooting guide.

  2. Check the status of the delegate on your cluster:

    kubectl describe pods -n <namespace>

  3. If the pod did not start, check the delegate logs:

    kubectl logs -f <harnessDelegateName> -n <namespace>

    If the state of the delegate pod is CrashLoopBackOff, check your allocation of compute resources (CPU and memory) to the cluster. A state of CrashLoopBackOff indicates insufficent Kubernetes cluster resources.

  4. If the delegate pod is not healthy, use the kubectl describe command to get more information:

    kubectl describe <pod_name> -n <namespace>

Pointing Harness CCM to Your Cluster​

With the Harness Delegate running, the next step is to create a Cloud Integration which will represent your Kubernetes cluster.

Harness Platform -> Cloud Costs -> Setup -> Cloud Integration + New Cluster/Cloud Account

Select Kubernetes.

New Cloud Integration

Once Kubernetes is selected, now we can configure the Kubernetes Cluster connector.

Name: my_kubernetes_cluster

Click Continue then in the Details section, leverage the Harness Delegate to connect to the Kubernetes cluster.

Use Harness Delegate

Click Continue, then select the Harness Delegate that is running inside your cluster.

Use Specific Harness Delegate

Enabling Cloud Costs​

Click Save and Continue and a connection test will occur and validate connectivity. With the Kubernetes cluster wired in, you can enable the wirings for the reporting.

Under the “Features Enabled” column, click Enable Cloud Costs.

Enable CCM

In the click-through, validation will occur and then click Finish. If additional permissions are needed, the wizard will guide you through enabling them.

Enabling CCM

Once you click Finish, Harness will start to analyze data around workload usage.

View Costs

Note, that initial recommendation and costing data can take up to 24 hours to start being reported/calculated.

After a few hours, cost information will start to be imported. Since in this example we did not connect a public cloud billing bucket, list pricing will be used in cost calculation.

Harness -> Cloud Costs -> Setup -> Cloud Integration -> my_kubernetes_cluster -> View Costs

Initial Costs

After about a day, more usage data will be captured and recommendations will start to appear.

First Costs

Digging into the savings recommendation, Harness CCM will provide recommendations off of the usage data captured.

Looking at Your First Cost Optimizations​

Digging into the recommendations list, looking at a recommendation that corresponds to the Kubernetes cluster, can see that the cluster is sized much larger than the resources that are being consumed.

Harness -> Cloud Costs -> Recommendations -> Your Resource

First Rec

Per this recommendation, resizing the worker node machine size is a prudent move. Taking a look at the back work for the recommendation, can take a look at the actual usage back in the previous perspective. Actions to be taken to reduce costs would be to right size the nodes and if not already, can place resource requests and limits onto the Kubernetes workloads based on what Harness CCM is reporting. This is just the start of the capabilities with Harness CCM, which can also provide additional rules and capabilities around auto-stopping of workloads and give additional infrastructure and workload recommendations. If additional changes are needed, can leverage the Harness Platform to experiment and make those changes.