Service Brokers

Overview

A Service Broker is a server compatible with the Open Service Broker API specification. A Service Broker manages the lifecycle of one or more services. It advertises a catalog of service offerings and service plans to a platform.

The Service Catalog lists all services that the Service Brokers offer. Use the Service Brokers to:

  • Provision and de-provision an instance of a service
  • Create and delete a service binding

Create a service binding to link a service instance to an application. During this process, credentials are delivered in Secrets to provide you with the information necessary to connect to the service instance. The process of deleting a service binding is known as unbinding.

Each of the Service Brokers available in Kyma performs these operations in a different way. See the documentation for a given Service Broker to learn how it operates.

Kyma provides these Service Brokers to use with the Service Catalog:

  • Azure Broker
  • Remote Environment Broker
  • Helm Broker

Follow the Open Service Broker API specification to build your own Service Broker. Register every new Service Broker in the Service Catalog to make the services and plans available to the users. For more information on registering Service Brokers in the Service Catalog, see the Service Catalog Demonstration Walkthrough.

Azure Broker

The Microsoft Azure Broker is an open source, Open Service Broker-compatible API server that provisions managed services in the Microsoft Azure public cloud. The Azure Broker provides these Service Classes to use with the Service Catalog:

  • Azure SQL Database
  • Azure Database for MySQL
  • Azure Redis Cache

See the details of each Service Class and its specification in the Service Catalog UI. For more information about the Service Brokers, see the Service Brokers Overview document.

NOTE: Kyma uses the Microsoft Azure Broker open source project. To ensure the best performance and stability of the product, Kyma uses a version of the Azure Broker that precedes the newest version released by Microsoft.

Remote Environment Broker

The Remote Environment Broker (REB) provides remote environments in the Service Catalog. A remote environment represents the environment connected to the Kyma instance. The Remote Environment Broker enables the integration of independent remote environments within Kyma. It also allows you to extend the functionality of existing systems.

The REB observes all the remote environment custom resources and exposes their APIs and/or Events as ServiceClasses to the Service Catalog. When the list of remote environments' ServiceClasses is available in the Service Catalog, you can create an EnvironmentMapping, provision those ServiceClasses, and enable them for Kyma services.

The REB implements the Service Broker API. For more details about the Service Brokers, see the Service Brokers Overview documentation.

Helm Broker

The Helm Broker is an implementation of a service broker which runs in the Kyma cluster and deploys Kubernetes native resources using Helm and Kyma bundles. A bundle is an abstraction layer over a Helm chart which allows you to represent it as a ClusterServiceClass in the Service Catalog. For example, a bundle can provide plan definitions or binding details. The Helm Broker fetches bundle definitions from an HTTP server. By default, the Helm Broker fetches bundles from the stable repository.

The Helm Broker implements the Service Broker API. For more information about the Service Brokers, see the Service Brokers overview document.

Configure Helm Broker

The Helm Broker fetches bundle definitions from an HTTP server defined in the values.yaml file. The config.repository.URL attribute defines the HTTP server URL.

Configuring the Helm Broker externally

Follow these steps to change the configuration and make the Helm Broker fetch bundles from a custom HTTP server:

  1. Create a remote bundles repository. Your remote bundle repository must include the following resources:

    • A yaml file which defines available bundles, for example bundles.yaml. This file must have the following structure:

      apiVersion: v1
      entries:
        {bundle_name}:
          - name: {bundle_name}
            description: {bundle_description}
            version: {bundle_version}
      

      This is an example of a yaml file for the Redis bundle:

      apiVersion: v1
      entries:
        redis:
          - name: redis
            description: Redis service
            version: 0.0.1
      
    • A {bundle_name}-{bundle_version}.tgz file for each bundle version defined in the yaml file. The .tgz file is an archive of your bundle's directory.

  2. In the values.yaml provide your server's URLs in the repository.URLs attribute as a list of URLs separated by a semicolon:

     repository:
       URLs: "http://custom.bundles-repository/bundles.yaml;http://another.bundles-repository/bundles.yaml"
    

    NOTE: You can skip the yaml filename in the URL if the name of the file is index.yaml. In that case, your URL should be equal to http://custom.bundles-repository/.

  3. Install Kyma on Minikube. See the Install Kyma locally from the release document to learn how.

Configure repository URLs in the runtime

Follow these steps to add the repository URL:

  1. Add a new bundle repository URL:

    URLS=$(kubectl get -n kyma-system deployment/core-helm-broker --output=jsonpath='{.spec.template.spec.containers[0].env[?(@.name=="APP_REPOSITORY_URLS")].value}')
    kubectl set env -n kyma-system deployment/core-helm-broker -e APP_REPOSITORY_URLS="$URLS;http://custom.bundles-repository/bundles.yaml"
    
  2. Wait for the Helm Broker to run using the following command:

    kubectl get pod -n kyma-system -l app=core-helm-broker
    

Running the Helm Broker triggers the Service Catalog synchronization automatically. New ClusterServiceClasses appear after a half-minute.

How to create a bundle

To create your own bundle, you must create a directory with the following structure:

sample-bundle/
  ├── meta.yaml                             # A file which contains the metadata information about this bundle
  ├── chart/                                # A directory which contains a Helm chart that installs your Kubernetes resources
  │    └── <chart-name>/                    # A Helm chart directory
  │         └── ....                        # Helm chart files
  └── plans/                                # A directory which contains the possible plans for an installed chart
       ├── example-enterprise               # A directory of files for a specific plan
       │   ├── meta.yaml                    # A file which contains the metadata information about this plan
       │   ├── bind.yaml                    # A file which contains information about the values that the Helm Broker returns when it receives the bind request
       │   ├── create-instance-schema.json  # The JSON Schema definitions for creating a ServiceInstance
       │   └── values.yaml                  # The default configuration values in this plan for a chart defined in the `chart` directory
       └── ....

NOTE: All the file names in the bundle directory are case-sensitive.

The bundle meta.yaml file

The meta.yaml file is mandatory as it contains information about the bundle. Set the following fields to create service objects which comply with the Open Service Broker API.

Field Name Required Description
name true The bundle name. It has the same restrictions as defined in the Open Service Broker API.
version true The bundle version. It is a broker service identifier. It has the same restrictions as defined in the Open Service Broker API.
id true A broker service identifier. It has the same restrictions as defined in the Open Service Broker API.
description true A short description of the service. It has the same restrictions as defined in the Open Service Broker API.
tags false The keywords describing the provided service, separated by commas.
bindable false The bindable field described in the Open Service Broker API.
displayName true The displayName field described in the Open Service Broker API.
providerDisplayName false The providerDisplayName field described in the Open Service Broker API.
longDescription false The longDescription field described in the Open Service Broker API.
documentationURL false The documentationURL field described in the Open Service Broker API.
supportURL false The supportURL field described in the Open Service Broker API.
imageURL false The imageURL field described in the Open Service Broker API. You must provide the image as an SVG.

The chart directory

In the mandatory chart directory, create a folder with the same name as your chart. Put all the files related to your chart in this folder. The system supports chart version 2.6.

If you are not familiar with the chart definitions, see the Charts specification.

NOTE: Helm Broker uses the helm wait option to ensure that all the resources that a chart creates are available. If you set your Deployment replicas to 1, you must set maxUnavailable to 0 as a part of the rolling update strategy.

The plans directory

The mandatory plans directory must contain at least one plan. A directory for a specific plan must contain the meta.yaml file. Other files, such as create-instance-schema.json, bind.yaml and values.yaml are not mandatory.

The meta.yaml file

The meta.yaml file contains information about a bundle plan. Set the following fields to create the plan objects, which comply with the Open Service Broker API.

Field Name Required Description
name true The plan name. It has the same restrictions as defined in the Open Service Broker API.
id true The plan ID. It has the same restrictions as defined in the Open Service Broker API.
description true The plan description. It has the same restrictions as defined in the Open Service Broker API.
displayName true The plan display name. It has the same restrictions as defined in the Open Service Broker API.
bindable false The plan bindable attribute. It has the same restrictions as defined in the Open Service Broker API.

The bind.yaml file

The bind.yaml file contains the information required for the binding action in a specific plan. If you defined in the meta.yaml file that your plan is bindable, you must also create a bind.yaml file. For more information about the content of the bind.yaml file, see the Binding bundles document.

The values.yaml file

The values.yaml file provides the default configuration values in a concrete plan for the chart definition located in the chart directory. This file is not required. For more information about the content of the values.yaml file, see the Values Files specification.

The create-instance-schema.json file

The create-instance-schema.json file contains a schema used to define the parameters. Each input parameter is expressed as a property within a JSON object. This file is not required. For more information about the content of the create-instance-schema.json file, see the Input parameters specification.

Troubleshooting

Use the dry-run mode to check the generated manifests of the chart without installing it. The --debug option prints the generated manifests. As a prerequisite, you must install Helm on your machine to run this command:

 helm install --dry-run {path-to-chart} --debug

For more details, read the Helm official documentation.

Binding bundles

If you defined in the meta.yaml file that your plan is bindable, you must also create a bind.yaml file. The bind.yaml file supports the Service Catalog binding concept. The bind.yaml file contains information the system uses in the binding process. The bind.yaml file is mandatory for all bindable plans. Currently, Kyma supports only the credentials-type binding.

NOTE: Resolving the values from the bind.yaml file is a post-provision action. If this operation ends with an error, the provisioning also fails.

Details

This section provides an example of the bind.yaml file. It further describes the templating, the policy concerning credential name conflicts, and the detailed bind.yaml file specification.

Example usage

# bind.yaml
credential:
  - name: HOST
    value: redis.svc.cluster.local
  - name: PORT
    valueFrom:
      serviceRef:
        name: redis-svc
        jsonpath: '{ .spec.ports[?(@.name=="redis")].port }'
  - name: REDIS_PASSWORD
    valueFrom:
      secretKeyRef:
        name: redis-secrets
        key: redis-password

credentialFrom:
  - configMapRef:
    name: redis-config
  - secretRef:
    name: redis-v2-secrets

In this example of the binding action, the Helm Broker returns the following values:

  • A HOST key with the defined inlined value.
  • A PORT key with the value from the field specified by the JSONPath expressions. The redis-svc Service runs this expression.
  • A REDIS_PASSWORD key with a value selected by the redis-password key from the redis-secrets Secret.
  • All the key-value pairs fetched from the redis-config ConfigMap.
  • All the key-value pairs fetched from the redis-v2-secrets Secrets.

Templating

In the bind.yaml file, you can use the Helm Chart templates directives.

# bind.yaml
credential:
  - name: HOST
    value: {{ template "redis.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local
{{- if .Values.usePassword }}
  - name: REDIS_PASSWORD
    valueFrom:
      secretKeyRef:
        name: {{ template "redis.fullname" . }}
        key: redis-password
{{- end }}

In this example, the system renders the bind.yaml file. The system resolves all the directives enclosed in the double curly braces in the same way as in the files located in the templates directory in your Helm chart.

Credential name conflicts policy

The following rules apply in cases of credential name conflicts:

  • When the credential and the credentialFrom sections have duplicate values, the system uses the values from the credential section.
  • When you duplicate a key in the credential section, an error appears and informs you about the name of the key that the conflict refers to.
  • When a key exists in the multiple sources defined by the credentialFrom section, the value associated with the last source takes precedence.

File specification

Field Name Description
credential The list of the credential variables to return during the binding action.
credentialFrom The list of the sources to populate the credential variables on the binding action. When the key exists in multiple sources, the value associated with the last source takes precedence. The variables from the credential section override the values if duplicate keys exist.

Credential

Field Name Description
name The name of the credential variable.
value A variable value. You can also use the Helm Chart templating directives.
valueFrom The source of the credential variable's value. You cannot use it if the value is not empty.
ValueFrom
Field Name Description
configMapKeyRef Selects a ConfigMap key in the Helm chart release Namespace.
secretKeyRef Selects a Secret key in the Helm Chart release Namespace.
serviceRef Selects a Service resource in the Helm Chart release Namespace.
ConfigMapKeyRef
Field Name Description
name The name of the ConfigMap.
key The name of the key from which the value is retrieved.
SecretKeyRef
Field Name Description
name The name of the Secret.
key The name of the key from which the value is retrieved.
ServiceRef
Field Name Description
name The name of the Service.
jsonpath The JSONPath expression used to select the specified field value. For more information, see the User Guide.

CredentialFrom

Field Name Description
configMapRef The ConfigMap to retrieve the values from. It must be available in the Helm Chart release Namespace.
secretRef The Secret to retrieve the values from. It must be available in the Helm Chart release Namespace.
ConfigMapRef
Field Name Description
name The name of the ConfigMap.
SecretRef
Field Name Description
name The name of the Secret.

Enable the Azure Broker for local deployment

NOTE: To enable the Azure Broker, you need a Microsoft Azure subscription.

By default, the Azure Broker is disabled for local installation and does not install along with other Kyma core components. To enable the installation of the Azure Broker, export these Azure Broker-specific environment variables before you install Kyma:

  • AZURE_BROKER_TENANT_ID
  • AZURE_BROKER_SUBSCRIPTION_ID
  • AZURE_BROKER_CLIENT_ID
  • AZURE_BROKER_CLIENT_SECRET

Export these variables using the details of your Microsoft Azure subscription, for example:

export AZURE_BROKER_TENANT_ID='{YOUR_TENANT_ID}'

The Remote Environment Broker architecture

The Remote Environment Broker (REB) workflow consists of the following steps:

  1. The Remote Environment Broker watches for RemoteEnvironments (REs) in the cluster and EnvironmentMappings (EMs) in all Namespaces.
  2. The user creates an EnvironmentMapping custom resource in a given Environment. The EnvironmentMapping activates services offered by a RE. The EnvironmentMapping must have the same name as the RE.
  3. The Remote Environment Broker creates a remote-env-broker Service Broker (SB) inside the Environment in which the EnvironmentMapping is created. This Service Broker contains data of all services provided by the activated RemoteEnvironments. There is always only one remote-env-broker Service Broker per Environment, even if there are more EnvironmentMappings.
  4. The Service Catalog fetches services that the remote-env-broker Service Broker exposes.
  5. The Service Catalog creates a ServiceClass for each service received from the Service Broker.

REB architecture

When this process is complete, you can provision and bind your services.

Provisioning and binding for an API ServiceClass

This ServiceClass has a bindable flag set to true which means that you have to provision a ServiceInstance and bind it to the service or lambda to connect to the given API. The provisioning and binding workflow for an API ServiceClass consists of the following steps:

  1. Select a given API ServiceClass from the Service Catalog.
  2. Provision this ServiceClass by creating its ServiceInstance in the given Environment.
  3. Bind your ServiceInstance to the service or lambda. During the binding process, ServiceBinding and ServiceBindingUsage resources are created.
    • ServiceBinding contains a Secret with a GatewayURL needed to connect to the given API.
    • ServiceBindingUsage injects the Secret, together with the label given during the registration process, to the lambda or service.
  4. The service or lambda calls the API through the Application Connector. The Application Connector verifies the label to check if you have the authorization to access this API.
  5. After verifying the label, the Application Connector allows you to access the remote environment API.

API Service Class

Provisioning and binding for an Event ServiceClass

This ServiceClass has a bindable flag set to false which means that after provisioning a ServiceClass in the Environment, given Events are ready to use for all services. The provisioning workflow for an Event ServiceClass consists of the following steps:

  1. Select a given Event ServiceClass from the Service Catalog.
  2. Provision this ServiceClass by creating a ServiceInstance in the given Environment.
  3. During the provisioning process, the EventActivation resource is created together with the ServiceInstance. EventActivation allows you to create an Event Bus Subscription.
  4. A Subscription is a custom resource by which an Event Bus triggers the lambda for a particular type of Event in this step.
  5. The RemoteEnvironment sends an Event to the Application Connector.
  6. The Application Connector sends an Event to the lambda through the Event Bus.

Event Service Class

Provisioning and binding for both the API and Event ServiceClass

This ServiceClass has a bindable flag set to true. The provisioning and binding workflow for both the API and Event ServiceClass is a combination of steps described for an API ServiceClass and an Event ServiceClass.

Helm Broker architecture

The Helm Broker workflow starts with the registration process, during which the Helm Broker fetches bundles from the Kyma bundles directory, or from a remote HTTP server.

Registration process

The registration process in case of the Helm Broker consists of the following steps:

  1. The Helm Broker fetches bundles from the remote HTTP server.
  2. The Helm Broker registers bundles as ServiceClasses in the Service Catalog.

Helm Broker registration

Bundles provisioning and binding

After the registration, the process of bundle provisioning and binding takes place. It consists of the following steps:

  1. Select a given bundle ServiceClass from the Service Catalog.
  2. Provision this ServiceClass by creating its ServiceInstance in the given Environment.
  3. Bind your ServiceInstance to the service or lambda. During the binding process, ServiceBinding and ServiceBindingUsage resources are created.
  4. The service or lambda calls a bundle.

Helm Broker architecture

EventActivation

The eventactivations.applicationconnector.kyma-project.io Custom Resource Definition (CRD) is a detailed description of the kind of data and the format used to create an Event Bus Subscription and to get an Event schema. To get the up-to-date CRD and show the output in the yaml format, run this command:

kubectl get crd eventactivations.applicationconnector.kyma-project.io -o yaml

Sample custom resource

This is a sample resource that allows you to consume Events sent from the service with the ac031e8c-9aa4-4cb7-8999-0d358726ffaa ID in a production Namespace.

apiVersion: applicationconnector.kyma-project.io/v1alpha1
kind: EventActivation
metadata:
  name: "ac031e8c-9aa4-4cb7-8999-0d358726ffaa"
  namespace: production
spec:
  displayName: "Orders"
  sourceId: "prod"

Custom resource parameters

This table lists all the possible parameters of a given resource together with their descriptions:

Parameter Mandatory Description
metadata.name YES Specifies the name of the CR and the ID of the Remote Environment service. This field is also used to fetch Event schemas from the Minio storage.
metadata.namespace YES Specifies the Namespace in which the CR is created.
spec.displayName YES Specifies a human-readable name of the Remote Environment service.
spec.sourceId YES Used to construct a Publish-Subscribe (Pub/Sub) topic name where the Events are send and from where the Events are consumed.

These are the resources related to this CR:

Custom resource Description
RemoteEnvironment Describes a service from which the user receives Events.
Subscription Contains information on how to create an infrastructure for consuming Events. Works only if the EventActivation is enabled.

These components use this CR:

Component Description
Remote Environment Broker Uses this CR to enable the user to receive Events from a given service.
Event Bus Uses this CR to control the consumption of an Event.
Serverless Lambda UI sends a GraphQL query to UI API Layer to list EventActivations.
UI API Layer Exposes the given CR to the Console UI.