Compass

Overview

NOTE: Compass is a new, experimental component in Kyma. To learn how to enable it, read the installation document.

Compass is a central, multi-tenant system that allows you to connect Applications and manage them across multiple Kyma Runtimes. Using Compass, you can control and monitor your Application landscape in one central place. As an integral part of Kyma, Compass uses a set of features that Kyma provides, such as Istio, Prometheus, Monitoring, and Tracing. It also includes Compass UI Cockpit that exposes Compass APIs to users. Compass allows you to:

  • Connect and manage Applications and Kyma Runtimes in one central place
  • Store Applications and Runtimes configurations
  • Group Applications and Runtimes to enable integration
  • Communicate the configuration changes to Applications and Runtimes
  • Establish a trusted connection between Applications and Runtimes using various authentication methods

Compass by design does not participate in direct communication between Applications and Runtimes. It only sets up the connection. In case the cluster with Compass is down, the Applications and Runtimes cooperation still works.

Architecture

Components

Components

Application

Application represents any external system that you want to register to Compass with its API and Event definitions. These are the types of possible integration levels between an Application and Compass:

  • Manual integration - the Administrator manually provides API or Events metadata to Compass. Use this type of integration for simple use-case scenarios as it does not support all features.
  • Built-in integration - integration with Compass is built in the Application.
  • Proxy - a highly Application-specific proxy component provides the integration.
  • Integration System - a central service provides integration for the dedicated group of Applications across multiple tenants. It manages multiple instances of the given Application type. You can integrate multiple Integration Systems to support different types of Applications.

Kyma Runtime

Runtime is a system to which you can apply configuration provided by Compass. Your Runtime must get a trusted connection to Compass. It must also allow for fetching Application definitions and using these Applications in one-tenant scope. By default, Compass is integrated with Kyma (Kubernetes), but its usage can also be extended to other platforms, such as CloudFoundry or Serverless.

Agent

Agent is an integral part of every Kyma Runtime and it fetches the latest configuration from Compass. In the future releases, Agent will:

  • Provide Runtime specific information that will be displayed in the Compass UI, such as Runtime UI URL
  • Provide Compass with Runtime configuration, such as Event Gateway URL, that should be passed to an Application
  • Send Runtime health checks to Compass

Cockpit

Cockpit is a UI that calls Compass APIs. This component is interchangeable.

Gateway

Gateway proxies the tenant's incoming requests to the Director component. All communication, whether it comes from an Application or other external components, flows through Gateway.

Connector

Connector establishes trust between Applications and Runtimes. Currently, only client certificates are supported.

Director

Director handles the process of managing Applications and Runtimes. It also requests Application webhook APIs for credentials and exposes health information about Runtimes. This component has access to the storage.

Runtime Provisioner

Runtime Provisioner handles the creation, modification, and deletion of Runtimes. This component is interchangeable.

Basic architecture

The diagram presents the basic workflow between Applications, Runtimes, and Compass:

Basic architecture

  1. Administrator adds Runtimes and Applications, and configures them using Compass.
  2. Agent continuously fetches the latest configuration from Compass.
  3. If an Application has optional webhooks configured, Compass notifies an Application about any Events that concern the given Application.

Compass by design does not participate in direct communication between Applications and Runtimes. It only sets up the connection. After establishing a trusted connection between an Application and a Runtime, they communicate directly with each other.

Scenarios

In order to connect and group your Applications and Runtimes, assign them to the same scenario. A scenario is a simple label with the scenarios key. If an Application is not explicitly assigned to any scenario, it belongs to the default one. The Application is automatically removed from the default scenario after you assign it to any other scenario. By default, Runtimes are not assigned to any scenario. You can assign Applications and Runtimes to multiple scenarios. See the example:

Scenarios

Application 2 belongs to the campaign and marketing scenarios. Assuming that Application 2 only exposes APIs, you can assign both Runtime 1 and Runtime 2 to it. However, if Application 2 also exposes Events, you can assign it only to one Runtime. If you try to connect more Runtimes, Compass will not allow you to do so. Communication between components that do not belong to the same scenario, such as Application 3 and Runtime 1, is not possible.

Details

Runtime Provisioner

The Runtime Provisioner is a Compass component responsible for provisioning, installing, and deprovisioning clusters with Kyma (Kyma Runtimes). The relationship between clusters and Runtimes is 1:1.

NOTE: Kyma installation is not implemented yet.

It is powered by Hydroform and it allows you to provision the clusters in two ways:

Note that the operations of provisioning and deprovisioning are asynchronous. They return the operation ID, which you can use to check the Runtime Operation Status.

The Runtime Provisioner also allows you to clean up Runtime data. This operation removes a given Runtime and all its data from the database and frees up the Runtime ID for reuse. It is useful when your cluster has died or when the operation of deprovisioning has failed.

The Runtime Provisioner exposes an API to manage cluster provisioning, installation, and deprovisioning.

Find the specification of the API here.

To access the Runtime Provisioner, forward the port that the GraphQL Server is listening on:

Click to copy
kubectl -n compass-system port-forward svc/compass-provisioner 3000:3000

Installation

Enable Compass in Kyma

To enable Compass in Kyma, follow the instructions for the custom component installation and enable the compass and compass-runtime-agent modules. You can also install Kyma on a cluster with the ready-to-use configurations for different modes. There are two modes in which you can enable Compass in Kyma: default Kyma installation and Compass as a Central Management Plane.

Default Kyma installation

This is a single-tenant mode, which provides the complete cluster Kyma installation with all components, including Compass and Agent. In this mode, Agent is already connected to Compass and they both work in a single-tenant mode as well. Using this mode, you can register external Applications in Kyma. To enable it, follow the cluster Kyma installation and use the installer-cr-cluster-with-compass.yaml.tpl configuration file.

Kyma mode1

Compass as a Central Management Plane

This is a multi-cluster mode in which you need one cluster with Compass and at least one cluster with Kyma Runtime, which you can connect and manage using Compass. This mode allows you to integrate your Runtimes with Applications and manage them in one central place.

Kyma mode2

Kyma Compass

This is a multi-tenant and multi-Runtime mode that provides a cluster with Compass and only the selected Kyma components that Compass uses. To enable this mode, create this ConfigMap and then perform the cluster Kyma installation using the installer-cr-cluster-compass.yaml.tpl configuration file:

Click to copy
apiVersion: v1
kind: ConfigMap
metadata:
name: compass-overrides
namespace: kyma-installer
labels:
installer: overrides
component: compass
kyma-project.io/installation: ""
data:
# The parameter that enables the Compass gateway, as the default Kyma gateway is disabled in this installation mode
gateway.gateway.enabled: "true"
# The name of the currently used gateway
global.istio.gateway.name: compass-istio-gateway
# The Namespace of the currently used gateway
global.istio.gateway.namespace: compass-system
# The parameter that disables preconfiguration for the Compass Agent
global.agentPreconfiguration: "false"
# The Namespace with a Secret that contains a certificate for the Connector Service
global.connector.secrets.ca.namespace: compass-system

Kyma Runtime

This is a single-tenant mode that provides complete cluster Kyma installation with Agent. To enable this mode, follow the cluster Kyma installation and use the installer-cr-cluster-runtime.yaml.tpl configuration file.

Configuration

Runtime Provisioner chart

To configure the Runtime Provisioner chart, override the default values of its values.yaml file. This document describes the parameters that you can configure.

TIP: To learn more about how to use overrides in Kyma, see Helm overrides for Kyma installation.

Configurable parameters

This table lists the configurable parameters, their descriptions, and default values:

ParameterDescriptionDefault value
SchemaFilePathFilepath for the database schemaassets/database/provisioner.sql

Tutorials

Provision clusters on Google Cloud Platform

This tutorial shows how to provision clusters with Kyma Runtimes on Google Cloud Platform (GCP).

Prerequisites

  • Existing project on GCP
  • Service account on GCP with the following roles:
    • Compute Admin
    • Kubernetes Engine Admin
    • Kubernetes Engine Cluster Admin
    • Service Account User
  • Key generated for your service account, downloaded in the JSON format
  • Secret from the service account key created in the compass-system Namespace, with the key credentials and the value encoded with base64

NOTE: To access the Runtime Provisioner, forward the port on which the GraphQL Server is listening.

Steps

To provision Kyma Runtime, make a call to the Runtime Provisioner with this example mutation:

NOTE: The cluster name must start with a lowercase letter followed by up to 39 lowercase letters, numbers, or hyphens, and cannot end with a hyphen.

Click to copy
mutation {
provisionRuntime(
id:"309051b6-0bac-44c8-8bae-3fc59c12bb5c"
config: {
clusterConfig: {
gcpConfig: {
name: "{CLUSTER_NAME}"
projectName: "{GCP_PROJECT_NAME}"
kubernetesVersion: "1.13"
bootDiskSizeGB: 30
numberOfNodes: 1
machineType: "n1-standard-4"
region: "europe-west3-a"
}
}
kymaConfig: {
version: "1.5"
modules: Backup
}
credentials: {
secretName: "{SECRET_NAME}"
}
}
)
}

A successful call returns the ID of the provisioning operation:

Click to copy
{
"data": {
"provisionRuntime": "e9c9ed2d-2a3c-4802-a9b9-16d599dafd25"
}
}

The operation of provisioning is asynchronous. Use the provisioning operation ID (provisionRuntime) to check the Runtime Operation Status and verify that the provisioning was successful. Use the Runtime ID (id) to check the Runtime Status.

Provision clusters through Gardener

This tutorial shows how to provision clusters with Kyma Runtimes on Google Cloud Platform (GCP), Microsoft Azure, and Amazon Web Services (AWS) using Gardener.

Prerequisites

  • GCP
  • Azure
  • AWS

NOTE: To access the Runtime Provisioner, forward the port on which the GraphQL Server is listening.

Steps

  • GCP
  • Azure
  • AWS

The operation of provisioning is asynchronous. Use the provisioning operation ID (provisionRuntime) to check the Runtime Operation Status and verify that the provisioning was successful. Use the Runtime ID (id) to check the Runtime Status.

Check Runtime Operation Status

This tutorial shows how to check the Runtime operation status for the operations of Runtime provisioning and deprovisioning.

Steps

NOTE: To access the Runtime Provisioner, forward the port on which the GraphQL Server is listening.

Make a call to the Runtime Provisioner to verify that (de)provisioning succeeded. Pass the ID of the operation as id.

Click to copy
query {
runtimeOperationStatus(id: "e9c9ed2d-2a3c-4802-a9b9-16d599dafd25") {
operation
state
message
runtimeID
}
}

A successful call returns a response which includes the status of the (de)provisioning operation (state) and the id of the (de)provisioned Runtime (runtimeID):

Click to copy
{
"data": {
"runtimeOperationStatus": {
"operation": "{"Provision"|"Deprovision"}",
"state": "Succeeded",
"message": "Operation succeeded.",
"runtimeID": "309051b6-0bac-44c8-8bae-3fc59c12bb5c"
}
}
}

The Succeeded status means that the operation of provisioning or deprovisioning was successful and the cluster was created or deleted respectively.

If you get the InProgress status, it means that the (de)provisioning has not yet finished. In that case, wait a few moments and check the status again.

Check Runtime Status

This tutorial shows how to check the Runtime status.

Steps

Make a call to the Runtime Provisioner to check the Runtime status. Pass the Runtime ID as id.

Click to copy
query { runtimeStatus(id: "309051b6-0bac-44c8-8bae-3fc59c12bb5c") {
runtimeConnectionStatus {
status errors {
message
}
}
lastOperationStatus {
message operation state runtimeID id
}
runtimeConfiguration {
kubeconfig kymaConfig {
version modules
} clusterConfig {
__typename ... on GCPConfig { bootDiskSizeGB name numberOfNodes kubernetesVersion projectName machineType zone region }
... on GardenerConfig { name workerCidr region diskType maxSurge nodeCount volumeSizeGB projectName machineType targetSecret autoScalerMin autoScalerMax provider maxUnavailable kubernetesVersion providerSpecificConfig
{ __typename
... on GCPProviderConfig { zone }
... on AzureProviderConfig {vnetCidr}
... on AWSProviderConfig {zone internalCidr vpcCidr publicCidr}
}
} } } } }

An example response for a successful request looks like this:

Click to copy
{
"data": {
"runtimeStatus": {
"runtimeConnectionStatus": {
"status": "Pending",
"errors": null
},
"lastOperationStatus": {
"message": "Operation succeeded.",
"operation": "Provision",
"state": "Succeeded",
"runtimeID": "309051b6-0bac-44c8-8bae-3fc59c12bb5c",
"id": "e9c9ed2d-2a3c-4802-a9b9-16d599dafd25"
},
"runtimeConfiguration": {
"kubeconfig": "{KUBECONFIG}",
"kymaConfig": {
"version": "1.5",
"modules": [
"Backup"
]
},
"clusterConfig": {CLUSTER_CONFIG}
}
}
}
}

Deprovision clusters

This tutorial shows how to deprovision clusters with Kyma Runtimes.

Steps

NOTE: To access the Runtime Provisioner, forward the port on which the GraphQL Server is listening.

To deprovision a Runtime, make a call to the Runtime Provisioner with a mutation like this:

Click to copy
mutation { deprovisionRuntime(id: "61d1841b-ccb5-44ed-a9ec-45f70cd1b0d3") }

A successful call returns the ID of the deprovisioning operation:

Click to copy
{
"data": {
"deprovisionRuntime": "c7e6727f-16b5-4748-ac95-197d8f79d094"
}
}

The operation of deprovisioning is asynchronous. Use the deprovisioning operation ID (deprovisionRuntime) to check the Runtime Operation Status and verify that the deprovisioning was successful. Use the Runtime ID (id) to check the Runtime Status.

Clean up Runtime data

This tutorial shows how to clean up Runtime data. This operation removes a given Runtime and all its data from the database and frees up the Runtime ID for reuse.

Steps

NOTE: To access the Runtime Provisioner, forward the port on which the GraphQL Server is listening.

To clean up Runtime data for a given Runtime, make a call to the Runtime Provisioner with a mutation like this:

Click to copy
mutation { cleanupRuntimeData(id: "61d1841b-ccb5-44ed-a9ec-45f70cd1b0d3")}

A successful call returns the ID of the wiped Runtime:

Click to copy
{
"data": {
"cleanupRuntimeData": "61d1841b-ccb5-44ed-a9ec-45f70cd1b0d3"
}
}

Use the Runtime ID to check the Runtime Status and make sure it has been wiped out.