Testkube - Guide to Kubernetes-Native Test Automation

Tuyen Nguyen
Nov 5, 2024
9 min read

#testkube #kubernetes #docker #cloudtesting #k8native #cicd #k8s

Testkube is an open-source, Kubernetes-native testing framework designed to help teams run, orchestrate, and manage tests within Kubernetes environments. It acts as a testing layer on top of Kubernetes, allowing developers and DevOps teams to integrate, automate, and manage tests as part of their CI/CD workflows in cloud-native environments.

Key features of Testkube

Kubernetes-Native Testing

Testkube is designed specifically for Kubernetes, enabling tests to run as Kubernetes jobs. This approach simplifies running tests in any Kubernetes cluster and utilizes Kubernetes’ native capabilities for scaling, isolation, and resource management.

Test Orchestration

With Testkube, you can orchestrate a wide range of test types and frameworks, from unit and integration tests to end-to-end (E2E) tests. It supports common testing frameworks like Postman, Cypress, JMeter, and more.

CI/CD Integration

Testkube integrates seamlessly with CI/CD tools (e.g., Jenkins, GitHub Actions, GitLab CI) and can be triggered as part of the CI/CD pipeline, automating testing as new changes are pushed to a cluster.

Multiple Test Types and Frameworks

Testkube can handle various testing types, including API tests, performance tests, functional tests, and load tests. It is compatible with many popular testing tools, making it versatile for multiple use cases.

Centralized Test Management

Testkube provides a centralized platform to manage all tests, schedules, configurations, and results, enabling easy access to logs and reports across all test executions.

Distributed Test Execution

By leveraging Kubernetes’ architecture, testkube enables distributed test execution, allowing for multiple tests to run simultaneously, speeding up the testing process.

Built-in Reporting and Observability

Testkube captures logs, metrics, and detailed execution results, allowing teams to analyze test performance, monitor errors, and gather insights to improve their test suites over time.

How Testkube works

Test Execution

Tests are configured and stored in Testkube, and when executed, Testkube runs them as Kubernetes jobs within the cluster. This allows it to use Kubernetes’ containerized environment for test isolation and consistency.

Components

Testkube consists of components:

API Server: Acts as the control plane, receiving requests and commands.
Testkube Agent: The agent responsible for orchestrating and managing test jobs in the Kubernetes cluster.
Dashboard and CLI: Provides a user-friendly interface to manage and monitor tests.

Event-driven and Scalable

Testkube can be configured to run tests based on events, making it ideal for use in microservices and event-driven architectures. Additionally, the framework is scalable, allowing teams to manage thousands of test cases across clusters.

Enough of what it is and its advantages, let's start our journey with how to set up the Testkube.

Testkube Cloud Architecture

Testkube contains two sub-systems:

A Control Plane which includes the Dashboard, Storage for Results/Artifacts, Cluster Federation, etc
An Agent running in your cluster that manages Testkube resources, runs tests, gathers results, etc.

The two sub-systems can be deployed in variety of ways to give flexibility into where your tests are executed and where test logs and artifacts are stored.

To our best simplicity but still remains our control in Testkube, it is recommended to try with Testkube Cloud. Testkube Cloud has a control plane that is managed by the Testkube team. You only have to run a stateless agent which makes operations a breeze.

ree — Testkube Cloud Architecture - testkube.io

For more details, Deployment Architecture.

Install Docker

As a prerequisite, we should have Docker is installed in our system. An easy way to install Docker is using the Docker Desktop appropriate for our operating system and processor.

We need Docker Desktop to run local Kubernetes clusters using Docker containers as nodes.

Install kubectl

kubectl allows you to run commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. For more information including a complete list of kubectl operations, see the kubectl reference documentation.

Installing on Windows

If you have Chocolatey Package Manager already, then use following command:

$ choco install kubernetes-cli

Check the version you installed is up-to-date:

$ kubectl version --client

Navigate to home directory:

# If you are using cmd, run cd %USERPROFILE#
$ cd ~

Create .kube directory:

$ mkdir .kube

Change to the .kube directory you just created:

$ cd .kube

Configure kubectl to use a remote Kubernetes cluster, below for Powershell:

$ New-Item config -type file

For other way of installing in Windows, please head to install kubectl for Windows.

Installing on MacOS

If Homebrew Package Manager is installed, do the following command:

$ brew install kubectl
# or
$ brew install kubernetes-cli

Check the version you installed is up-to-date:

$ kubectl version --client

For other way of installing in MacOS, please head to install kubectl for MacOS.

Installing on Linux

For Linux, please head to install kubectl for Linux.

Install kind

kind is a tool for running local Kubernetes clusters using Docker container "nodes".

kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.

Installing on Windows

With Chocolatey:

$ choco install kind

With Winget:

$ winget install Kubernetes.kind

Installing on MacOS

With Homebrew:

$ brew install kind

With MacPorts:

$ sudo port selfupdate && sudo port install kind

Verify the successful installation:

$ kind version

Create cluster

Create a local Kubernetes cluster with default configuration:

$ kind create cluster

There are some options we can try while creating a cluster:

Create cluster with configuration file

For example, let's write a simplistic configuration in tuyenConfig.yaml file:

kind: Cluster
apiVersion: kind.x-k8s.io/1
name: tuyen-kind

Specify that config file in create command:

$ kind create cluster --config tuyenConfig.yaml

Create cluster with specific version of Kubernetes image

$ kind create cluster --image kindnest/mode:v1.30.0

Check the created cluster:

$ kind get cluster
kind-testkube

Check if the corresponding container is created:

$ docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED
1ae8811317d6   kindest/node:v1.30.0   "/usr/local/bin/entr…"   3 hours ago

Or via Docker Desktop application:

Or get confirmation of the nodes via kubectl:

$ kubectl get nodes
NAME                          STATUS   ROLES           AGE   VERSION
kind-testkube-control-plane   Ready    control-plane   24h   v1.30.0

Install testkube CLI

The Testkube CLI enables you to manage your Testkube installation and interact with resources like creating Test Workflows, starting test executions, managing triggers, and more. For a complete list of commands, refer to the CLI Commands Reference.

Installing on Windows

Using Chocolatey:

$ choco source add --name=kubeshop_repo --source=https://chocolatey.kubeshop.io/chocolatey
choco install testkube -y

Installing on MacOS

Using brew:

$ brew install testkube

Validate the successful installation:

$ testkube version

For other systems/methods, Testkube CLI.

Install testkube

Since we decided to follow Testkube Cloud Architecture, we need to sign in to Testkube Cloud first.

Then it will ask us to create new environment, let's select option with having a K8s cluster or Docker available:

After providing your own preferred name for environment, we will be provided with cli command to set up Testkube Agent:

Upon successful set up:

⚠️ Caution: Cannot upgrade Testkube

If you meet the issue of testkube is not updated to new version with testkube upgrade, we may need to uninstall and reinstall the testkube CLI.

For Windows:

choco uninstall testkube

For MacOS:

brew uninstall testkube

And then reinstall using provided methods in above section Install testkube CLI.

⚠️ Caution: Issue with setting pro environment context

If you experience the issue such as Unauthorized, error 401 in Setting Pro Environment Context, it is likely your configs between your Agent and Testkube Cloud environment has been broken.

First, we may need to upgrade testkube:

testkube upgrade

Then update config that specifically for current context of environment. About <MY_API_KEY>, you can click on Manage your API Tokens here to generate one:

Lastly, you can rerun below command to check everything is working:

$ testkube version

Define the test workflow

Test Workflow is a cutting-edge and highly effective method for defining and executing tests within Kubernetes clusters. With its versatility, Test Workflow offers the following advantages:

Running tests with different versions of testing tools and dependencies.
Defining multiple steps within test execution workflows.
Gaining greater control over test execution, including resource allocation and setup/tearDown processes.
Customizing tool-specific commands and arguments to suit your needs.

It is using specific workflow language wrapped in a CRD. Below is high-level structure:

Example - Test Workflow for Postman

The Postman collection for running below TestWorkflow is from Testkube repository:

postman-executor-smoke-without-envs.postman_collection.json

Example - Test Workflow for k6

Below is an example for k6 load test, which we just add inline test for simplicity.

Run the Test Workflow - using Dashboard

Since we are utilizing Test Cloud Architecture with Test Agent hosted in our own cluster, we fortunately have 2 options for running our Test Workflows. The first option is using the Dashboard in Testkube Cloud.

We need to log into your Testkube Cloud and navigate to Test Workflow to create a new one:

And there are 4 selections:

Create from Wizard

This will walk you through a series of steps to create your Test Workflow:

The name and the testing tool/template you want to write with corresponding settings.
The content of the test.
The command to run and where Testkube should scrape the generated artifacts.
The summary showing the generated workflow YAML from your inputs above.

When you are happy with the workflow, then we can click Create & Run to create and run it right away to check our generated workflow.

Start from an example

You will be fulfilled with varied Testkube Workflows examples which you can then further modify to your specific needs.

There will be a selector that you can filter out the example from desired test frameworks/tools.
At the bottom, it is the YAML example, you can modify freely before using Create & Run button to create and test your workflow.

Combine existing Workflows

This allows you to create a Workflow that orchestrates multiple workflows to run either in sequence or parallel. But you need to have some available Workflows in order to chain/organize them in any desired order/combination.

Once done, click Next to see corresponding YAML and then select Create & Run to create and run it accordingly.

Import from YAML

The last option allows us to past/edit a Workflow YAML from scratch. So we can copy and paste above two examples to try.

Once done, select Create & Run to create and run the workflow accordingly.

Run the Test Workflow - using Testkube CLI

Since we have Test Agent in our created cluster, we are allowed to create new workflow yaml as CRD and use command to add and run it.

Create a yaml file, such as my-workflow.yaml.
Copy and paste the content of workflow (we can use the two provided examples above).
Execute the below command:

$ kubectl apply -f my-workflow.yaml && kubectl testkube run tw my-test --watch

$ kubectl apply -f <yaml file of workflow> && kubectl testkube run tw <name of test worfklow> --watch

View the status and logs from the Workflow as it is executing.

Check the past executions

We have both Dashboard in Testkube Cloud and Testkube CLI so we have 2 options to check the past executions.

Using Dashboard

Navigate to Workflows, then Execution in your Testkube Cloud.
All recent executions of specific workflow are listed at the bottom.

By clicking on each execution, you can view the execution's logs.
We can also download the test artifacts in Artifacts tab.

Using CLI

List recent executions:

$ kubectl testkube get twe

Context: cloud (2.1.48)   Namespace: testkube   Org: Tuyen Nguyen Organization   Env: TN-personal-env
-----------------------------------------------------------------------------------------------------
  ID                       | NAME        | TEST WORKFLOW NAME | STATUS | LABELS       | TAGS
---------------------------+-------------+--------------------+--------+--------------+-------
  6727aa96b0e702a749432363 | k6-sample-3 | k6-sample          | passed | docs=example |
  6727aa78b0e702a749432362 | k6-sample-2 | k6-sample          | passed | docs=example |
  6727aa25b0e702a749432361 | k6-sample-1 | k6-sample          | passed | docs=example |

List artifacts of specific execution:

$ kubectl testkube get artifacts 6727aa96b0e702a749432363

Context: cloud (2.1.48)   Namespace: testkube   Org: Tuyen Nguyen Organization   Env: TN-personal-env
-----------------------------------------------------------------------------------------------------
  EXECUTION                | NAME                | SIZE (KB)
---------------------------+---------------------+------------
  6727aa96b0e702a749432363 | k6-test-report.html |    161241

Download the specific artifact:

$ kubectl testkube download artifact 6727aa96b0e702a749432363 k6-test-report.html .

Context: cloud (2.1.48)   Namespace: testkube   Org: Tuyen Nguyen Organization   Env: TN-personal-env
-----------------------------------------------------------------------------------------------------
File k6-test-report.html downloaded.

Fetch the logs of the execution:

$ kubectl testkube get twe 6727aa96b0e702a749432363 --logs-only

Integrate with CI/CD Pipelines

Testkube Workflow execution can easily be integrated in your CI pipelines, run on a cron schedule or trigger on a Kubernetes event.

Trigger from CI/CD

Navigate to your Test Workflow within the Dashboard.
Click on CI/CD Integrations tab.
There will be list of supported CI/CD providers that Testkube can fully integrate with.
Follow the instructions that Testkube provided for further detailed guide on your desired provider.

Testkube shows instructions for GitHub Actions, GitLab CI, CircleCI, Jenkins, Azure DevOps, Tekton and ArgoCD, but can be integrated into any CI/CD pipeline by automating the Testkube CLI and API by our own will.

For more details, Testkube Integrations.

Trigger on a schedule

Navigate to your Test Workflow within the Dashboard.
Click on Settings tab, and then Definition of current workflow.
Or we can click Edit in the dotted menu top right.
Then make the following changes and save the workflow:

apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata: …
spec:
+  events:
+  - cronjob:
+      crob: '*/5 * * * *'
  content: …
  steps: …

Above added code lines will trigger the run every 5 minutes.

For more details, Scheduling Tests.

Trigger on a Kubernetes event

Navigate to Integrations, then Kubernetes Triggers tab in your Testkube Cloud.
Input the trigger name, type of K8s resource, resource identifier and type of event. In below example, the trigger will be executed when there is deployment image update on resource kind-testkube:test-app.

Input Testkube action and Testkube resource. In below example, the workflow postman-test will run upon above trigger event started.

More Triggers

Other ways to trigger a Test Workflow in Testkube include:

By creating an Execution CRD.
Via the Testkube CLI.
Via the REST API.
From another Workflow - as described in Test Workflows - Test Suites.

Conclusions

In summary, Testkube brings powerful, Kubernetes-native automation to testing workflows, allowing teams to manage, scale, and streamline testing directly within the cluster. With support for various frameworks and easy CI/CD integration, Testkube turns testing into an efficient part of continuous delivery, enhancing collaboration and quality across applications.

As Kubernetes leads modern cloud infrastructure, Testkube is an essential tool for scalable, automated testing. Embrace Testkube, and simplify your path to robust, integrated Kubernetes-native testing.

What are we expecting in the next blog post?

It will be an extremely interesting topic - Approach to GitOps Cloud Native Testing with Testkube. Please stay tuned for it!!!

AUTOMATED
TESTING

With TUYEN

Testkube - Guide to Kubernetes-Native Test Automation

Key features of Testkube

Kubernetes-Native Testing

Test Orchestration

CI/CD Integration

Multiple Test Types and Frameworks

Centralized Test Management

Distributed Test Execution

Built-in Reporting and Observability

How Testkube works

Test Execution

Components

Event-driven and Scalable

Testkube Cloud Architecture

Install Docker

Install kubectl

Install kind

Create cluster

Install testkube CLI

Install testkube

Define the test workflow

Example - Test Workflow for Postman

Example - Test Workflow for k6

Run the Test Workflow - using Dashboard

Run the Test Workflow - using Testkube CLI

Check the past executions

Integrate with CI/CD Pipelines

Conclusions

Recent Posts

Join my mailing list

AUTOMATED TESTING

With TUYEN

Key features of Testkube

Kubernetes-Native Testing

Test Orchestration

CI/CD Integration

Multiple Test Types and Frameworks

Centralized Test Management

Distributed Test Execution

Built-in Reporting and Observability

How Testkube works

Test Execution

Components

Event-driven and Scalable

Testkube Cloud Architecture

Install Docker

Install kubectl

Install kind

Create cluster

Install testkube CLI

Install testkube

Define the test workflow

Example - Test Workflow for Postman

Example - Test Workflow for k6

Run the Test Workflow - using Dashboard

Run the Test Workflow - using Testkube CLI

Check the past executions

Integrate with CI/CD Pipelines

Conclusions

Join my mailing list

AUTOMATED
TESTING