Service Brokers

Service Brokers

A Service Broker is a server compatible with the Open Service Broker API specification. A Service Broker manages the lifecycle of one or more services. It advertises a catalog of service offerings and service plans to a platform.

The Service Catalog lists all services that the Service Brokers offer. Use the Service Brokers to:

  • Provision and de-provision an instance of a service
  • Create and delete a service binding

Create a service binding to link a service instance to an application. During this process, credentials are delivered in Secrets to provide you with the information necessary to connect to the service instance. The process of deleting a service binding is known as unbinding.

Each of the Service Brokers available in Kyma performs these operations in a different way. See the documentation for a given Service Broker to learn how it operates.

Kyma provides these Service Brokers to use with the Service Catalog:

  • Azure Broker
  • Google Cloud Platform Service Broker (Experimental)
  • Application Broker
  • Helm Broker

Follow the Open Service Broker API specification to build your own Service Broker. Register every new Service Broker in the Service Catalog to make the services and plans available to the users. For more information on registering Service Brokers in the Service Catalog, see the Service Catalog Demonstration Walkthrough.

Azure Broker

The Microsoft Azure Broker is an open source, Open Service Broker-compatible API server that provisions managed services in the Microsoft Azure public cloud. The Azure Broker provides these Service Classes to use with the Service Catalog:

  • Azure SQL Database
  • Azure Database for MySQL
  • Azure Redis Cache

See the details of each Service Class and its specification in the Service Catalog UI. For more information about the Service Brokers, see the Service Brokers Overview document.

NOTE: Kyma uses the Microsoft Azure Broker open source project. To ensure the best performance and stability of the product, Kyma uses a version of the Azure Broker that precedes the newest version released by Microsoft.

Google Cloud Platform Broker

NOTE: The Google Cloud Platform (GCP) Service Broker is in the experimental phase.

Google Cloud Platform (GCP) Service Broker is an implementation of the Open Service Broker (OSB) API hosted on GCP. It simplifies the delivery of GCP services to applications that run on Kyma. By creating GCP resources and managing their corresponding permissions, Service Broker makes it easy to consume GCP services from within a Kubernetes cluster.

Kyma provides Namespace-scoped GCP Service Broker. In each Namespace, you can configure the GCP Broker against different Google Cloud Platforms. Install the GCP Service Broker by provisioning the Google Cloud Platform Service Broker Provider class provided by the Helm Broker.

Service Catalog view without GCP Classes

Once you provision the Google Cloud Platform Service Broker Provider class, the GCP Service Broker classes are available in the Service Catalog view in a given Namespace.

Service Catalog view without GCP Classes

For more information about provisiong the Google Cloud Platform Service Broker Provider class, go to the service class overview in the Service Catalog UI.

NOTE: Google does not provide the ability to specify the broker's version so the most up-to-date version of the GCP Broker is installed.

Application Broker

The Application Broker (AB) provides Applications in the Service Catalog. An Application represents an external solution connected to the Kyma instance. The AB enables the integration of independent Applications within Kyma. It also allows you to extend the functionality of existing systems.

The AB observes all the Application custom resources and exposes their APIs and Events as ServiceClasses to the Service Catalog. When the list of remote the ServiceClasses of an Application is available in the Service Catalog, you can create an ApplicationMapping, provision those ServiceClasses, and enable them for Kyma services.

The AB implements the Open Service Broker API. For more details about the Service Brokers, see the Service Brokers Overview documentation.

Helm Broker

The Helm Broker is an implementation of a service broker which runs in the Kyma cluster and deploys Kubernetes native resources using Helm and Kyma bundles. A bundle is an abstraction layer over a Helm chart which allows you to represent it as a ClusterServiceClass in the Service Catalog. For example, a bundle can provide plan definitions or binding details. The Helm Broker fetches bundle definitions from an HTTP server. By default, the Helm Broker fetches bundles from the newest release with the Semantic Versioning pattern available at the bundles repository.

Using bundles, you can also install other brokers, such as the Google Cloud Platform Service Broker.

For more information about the Service Brokers, see the Service Brokers overview document.

The Helm Broker implements the Open Service Broker API (OSB API). To be compliant with the Service Catalog version used in Kyma, the Helm Broker supports only the following versions of the OSB API:

  • v2.13
  • v2.12
  • v2.11

NOTE: The Helm Broker does not implement the OSB API update operation.

Configuration

Configure Helm Broker

The Helm Broker fetches bundle definitions from an HTTP server defined in the values.yaml file. The config.repository.URL attribute defines the HTTP server URL.

Configuring the Helm Broker externally

Follow these steps to change the configuration and make the Helm Broker fetch bundles from a custom HTTP server:

  1. Create a remote bundles repository. Your remote bundle repository must include the following resources:

    • A yaml file which defines available bundles, for example bundles.yaml. This file must have the following structure:

      Click to copy
      apiVersion: v1
      entries:
      {bundle_name}:
      - name: {bundle_name}
      description: {bundle_description}
      version: {bundle_version}

      This is an example of a yaml file for the Redis bundle:

      Click to copy
      apiVersion: v1
      entries:
      redis:
      - name: redis
      description: Redis service
      version: 0.0.1
    • A {bundle_name}-{bundle_version}.tgz file for each bundle version defined in the yaml file. The .tgz file is an archive of your bundle's directory.

  2. In the values.yaml provide your server's URLs in the repository.URLs attribute as a list of URLs separated by a semicolon:

    Click to copy
    repository:
    URLs: "http://custom.bundles-repository/bundles.yaml;http://another.bundles-repository/bundles.yaml"

    NOTE: You can skip the yaml filename in the URL if the name of the file is index.yaml. In that case, your URL should be equal to http://custom.bundles-repository/.

  3. Install Kyma on Minikube. See the Install Kyma locally from the release document to learn how.

Configure repository URLs in the runtime

Follow these steps to add the repository URL:

  1. Add a new bundle repository URL:

    Click to copy
    URLS=$(kubectl get -n kyma-system deployment/core-helm-broker --output=jsonpath='{.spec.template.spec.containers[0].env[?(@.name=="APP_REPOSITORY_URLS")].value}')
    kubectl set env -n kyma-system deployment/core-helm-broker -e APP_REPOSITORY_URLS="$URLS;http://custom.bundles-repository/bundles.yaml"
  2. Wait for the Helm Broker to run using the following command:

    Click to copy
    kubectl get pod -n kyma-system -l app=core-helm-broker

Running the Helm Broker triggers the Service Catalog synchronization automatically. New ClusterServiceClasses appear after a half-minute.

How to create a bundle

To create your own bundle, you must create a directory with the following structure:

Click to copy
sample-bundle/
├── meta.yaml # A file which contains the metadata information about this bundle
├── chart/ # A directory which contains a Helm chart that installs your Kubernetes resources
│ └── <chart-name>/ # A Helm chart directory
│ └── .... # Helm chart files
└── plans/ # A directory which contains the possible plans for an installed chart
├── example-enterprise # A directory of files for a specific plan
│ ├── meta.yaml # A file which contains the metadata information about this plan
│ ├── bind.yaml # A file which contains information about the values that the Helm Broker returns when it receives the bind request
│ ├── create-instance-schema.json # The JSON Schema definitions for creating a ServiceInstance
│ ├── bind-instance-schema.json # The JSON Schema definitions for binding a ServiceInstance
│ ├── update-instance-schema.json # The JSON Schema definitions for updating a ServiceInstance
│ └── values.yaml # The default configuration values in this plan for a chart defined in the `chart` directory
└── ....

NOTE: All the file names in the bundle directory are case-sensitive.

The bundle meta.yaml file

The meta.yaml file is mandatory as it contains information about the bundle. Set the following fields to create service objects which comply with the Open Service Broker API.

Field NameRequiredDescription
nametrueThe bundle name. It has the same restrictions as defined in the Open Service Broker API.
versiontrueThe bundle version. It is a broker service identifier. It has the same restrictions as defined in the Open Service Broker API.
idtrueA broker service identifier. It has the same restrictions as defined in the Open Service Broker API.
descriptiontrueA short description of the service. It has the same restrictions as defined in the Open Service Broker API.
tagsfalseThe keywords describing the provided service, separated by commas.
bindablefalseThe bindable field described in the Open Service Broker API.
displayNametrueThe displayName field described in the Open Service Broker API.
providerDisplayNamefalseThe providerDisplayName field described in the Open Service Broker API.
longDescriptionfalseThe longDescription field described in the Open Service Broker API.
documentationURLfalseThe documentationURL field described in the Open Service Broker API.
supportURLfalseThe supportURL field described in the Open Service Broker API.
imageURLfalseThe imageURL field described in the Open Service Broker API. You must provide the image as an SVG.
labelsfalseTo organize your project, add arbitrary labels as key/value pairs. Use labels to indicate different elements, such as environments, services, or teams.
bindingsRetrievablefalseThe bindingRetrievable field specifies whether fetching a ServiceBinding using a GET call on the resource's endpoint is supported for all plans. The default value is false.
planUpdatablefalseThe planUpdatable field specifies whether instances of this service can be updated to a different plan. The default value is false
requiresfalseThe requires field defines a list of permissions the user must grant to the instances of this service.

The chart directory

In the mandatory chart directory, create a folder with the same name as your chart. Put all the files related to your chart in this folder. The system supports chart version 2.6.

If you are not familiar with the chart definitions, see the Charts specification.

NOTE: Helm Broker uses the helm wait option to ensure that all the resources that a chart creates are available. If you set your Deployment replicas to 1, you must set maxUnavailable to 0 as a part of the rolling update strategy.

The plans directory

The mandatory plans directory must contain at least one plan. A directory for a specific plan must contain the meta.yaml file. Other files, such as create-instance-schema.json, bind.yaml and values.yaml are not mandatory.

The meta.yaml file

The meta.yaml file contains information about a bundle plan. Set the following fields to create the plan objects, which comply with the Open Service Broker API.

Field NameRequiredDescription
nametrueThe plan name. It has the same restrictions as defined in the Open Service Broker API.
idtrueThe plan ID. It has the same restrictions as defined in the Open Service Broker API.
descriptiontrueThe plan description. It has the same restrictions as defined in the Open Service Broker API.
displayNametrueThe plan display name. It has the same restrictions as defined in the Open Service Broker API.
bindablefalseThe plan bindable attribute. It has the same restrictions as defined in the Open Service Broker API.
freefalseThe attribute which specifies whether the instance is free or not. The default value is false.

The bind.yaml file

The bind.yaml file contains the information required for the binding action in a specific plan. If you defined in the meta.yaml file that your plan is bindable, you must also create a bind.yaml file. For more information about the content of the bind.yaml file, see the Binding bundles document.

The values.yaml file

The values.yaml file provides the default configuration values in a concrete plan for the chart definition located in the chart directory. This file is not required. For more information about the content of the values.yaml file, see the Values Files specification.

The create-instance-schema.json file

The create-instance-schema.json file contains a schema used to define the parameters for a provision operation of the ServiceInstance. Each input parameter is expressed as a property within a JSON object. This file is not required. For more information about the content of the create-instance-schema.json file, see the Schemas specification.

The update-instance-schema.json file

The update-instance-schema.json file contains a schema used to define the parameters for an update operation of the ServiceInstance. Each input parameter is expressed as a property within a JSON object. This file is not required. For more information about the content of the update-instance-schema.json file, see the Schemas specification.

The bind-instance-schema.json file

The bind-instance-schema.json file contains a schema used to define the parameters for a bind operation. Each input parameter is expressed as a property within a JSON object. This file is not required. For more information about the content of the bind-instance-schema.json file, see the Schemas specification.

Troubleshooting

Use the dry-run mode to check the generated manifests of the chart without installing it. The --debug option prints the generated manifests. As a prerequisite, you must install Helm on your machine to run this command:

Click to copy
helm install --dry-run {path-to-chart} --debug

For more details, read the Helm official documentation.

Binding bundles

If you defined in the meta.yaml file that your plan is bindable, you must also create a bind.yaml file. The bind.yaml file supports the Service Catalog binding concept. The bind.yaml file contains information the system uses in the binding process. The bind.yaml file is mandatory for all bindable plans. Currently, Kyma supports only the credentials-type binding.

NOTE: Resolving the values from the bind.yaml file is a post-provision action. If this operation ends with an error, the provisioning also fails.

Details

This section provides an example of the bind.yaml file. It further describes the templating, the policy concerning credential name conflicts, and the detailed bind.yaml file specification.

Example usage

Click to copy
# bind.yaml
credential:
- name: HOST
value: redis.svc.cluster.local
- name: PORT
valueFrom:
serviceRef:
name: redis-svc
jsonpath: '{ .spec.ports[?(@.name=="redis")].port }'
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: redis-secrets
key: redis-password
credentialFrom:
- configMapRef:
name: redis-config
- secretRef:
name: redis-v2-secrets

In this example of the binding action, the Helm Broker returns the following values:

  • A HOST key with the defined inlined value.
  • A PORT key with the value from the field specified by the JSONPath expressions. The redis-svc Service runs this expression.
  • A REDIS_PASSWORD key with a value selected by the redis-password key from the redis-secrets Secret.
  • All the key-value pairs fetched from the redis-config ConfigMap.
  • All the key-value pairs fetched from the redis-v2-secrets Secrets.

Templating

In the bind.yaml file, you can use the Helm Chart templates directives.

Click to copy
# bind.yaml
credential:
- name: HOST
value: {{ template "redis.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local
{{- if .Values.usePassword }}
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: {{ template "redis.fullname" . }}
key: redis-password
{{- end }}

In this example, the system renders the bind.yaml file. The system resolves all the directives enclosed in the double curly braces in the same way as in the files located in the templates directory in your Helm chart.

Credential name conflicts policy

The following rules apply in cases of credential name conflicts:

  • When the credential and the credentialFrom sections have duplicate values, the system uses the values from the credential section.
  • When you duplicate a key in the credential section, an error appears and informs you about the name of the key that the conflict refers to.
  • When a key exists in the multiple sources defined by the credentialFrom section, the value associated with the last source takes precedence.

File specification

Field NameDescription
credentialThe list of the credential variables to return during the binding action.
credentialFromThe list of the sources to populate the credential variables on the binding action. When the key exists in multiple sources, the value associated with the last source takes precedence. The variables from the credential section override the values if duplicate keys exist.

Credential

Field NameDescription
nameThe name of the credential variable.
valueA variable value. You can also use the Helm Chart templating directives.
valueFromThe source of the credential variable's value. You cannot use it if the value is not empty.
ValueFrom
Field NameDescription
configMapKeyRefSelects a ConfigMap key in the Helm chart release Namespace.
secretKeyRefSelects a Secret key in the Helm Chart release Namespace.
serviceRefSelects a Service resource in the Helm Chart release Namespace.
ConfigMapKeyRef
Field NameDescription
nameThe name of the ConfigMap.
keyThe name of the key from which the value is retrieved.
SecretKeyRef
Field NameDescription
nameThe name of the Secret.
keyThe name of the key from which the value is retrieved.
ServiceRef
Field NameDescription
nameThe name of the Service.
jsonpathThe JSONPath expression used to select the specified field value. For more information, see the User Guide.

CredentialFrom

Field NameDescription
configMapRefThe ConfigMap to retrieve the values from. It must be available in the Helm Chart release Namespace.
secretRefThe Secret to retrieve the values from. It must be available in the Helm Chart release Namespace.
ConfigMapRef
Field NameDescription
nameThe name of the ConfigMap.
SecretRef
Field NameDescription
nameThe name of the Secret.

Enable the Azure Broker for local deployment

NOTE: To enable the Azure Broker, you need a Microsoft Azure subscription.

By default, the Azure Broker is disabled for local installation and does not install along with other Kyma core components. To enable the installation of the Azure Broker, export these Azure Broker-specific environment variables before you install Kyma:

  • AZURE_BROKER_TENANT_ID
  • AZURE_BROKER_SUBSCRIPTION_ID
  • AZURE_BROKER_CLIENT_ID
  • AZURE_BROKER_CLIENT_SECRET

Export these variables using the details of your Microsoft Azure subscription, for example:

Click to copy
export AZURE_BROKER_TENANT_ID='{YOUR_TENANT_ID}'

Architecture

The Application Broker architecture

The Application Broker (AB) workflow consists of the following steps:

  1. The Application Broker watches for Applications (Apps) in the cluster and ApplicationMappings (AMs) in all Namespaces.
  2. The user creates an ApplicationMapping custom resource in a given Environment. The AM activates services offered by an App. The AM must have the same name as the App.
  3. The Application Broker creates an application-broker Service Broker (SB) inside the Environment in which the APM is created. This Service Broker contains data of all services provided by the activated Applications. There is always only one application-broker Service Broker per Environment, even if there are more AMs.
  4. The Service Catalog fetches services that the application-broker Service Broker exposes.
  5. The Service Catalog creates a ServiceClass for each service received from the Service Broker.

AB architecture

When this process is complete, you can provision and bind your services.

Provisioning and binding for an API ServiceClass

This ServiceClass has a bindable flag set to true which means that you have to provision a ServiceInstance and bind it to the service or lambda to connect to the given API. The provisioning and binding workflow for an API ServiceClass consists of the following steps: 1. Select a given API ServiceClass from the Service Catalog. 2. Provision this ServiceClass by creating its ServiceInstance in the given Environment. 3. Bind your ServiceInstance to the service or lambda. During the binding process, ServiceBinding and ServiceBindingUsage resources are created. ServiceBinding contains a Secret with a GatewayURL needed to connect to the given API. ServiceBindingUsage injects the Secret, together with the label given during the registration process, to the lambda or service. 4. The service or lambda calls the API through the Application Connector. The Application Connector verifies the label to check if you have the authorization to access this API. 5. After verifying the label, the Application Connector allows you to access the Application API.

API Service Class

Provisioning and binding for an Event ServiceClass

This ServiceClass has a bindable flag set to false which means that after provisioning a ServiceClass in the Environment, given Events are ready to use for all services. The provisioning workflow for an Event ServiceClass consists of the following steps: 1. Select a given Event ServiceClass from the Service Catalog. 2. Provision this ServiceClass by creating a ServiceInstance in the given Environment. 3. During the provisioning process, the EventActivation resource is created together with the ServiceInstance. EventActivation allows you to create an Event Bus Subscription. 4. A Subscription is a custom resource by which an Event Bus triggers the lambda for a particular type of Event in this step. 5. The RemoteEnvironment sends an Event to the Application Connector. 6. The Application Connector sends an Event to the lambda through the Event Bus.

Event Service Class

Provisioning and binding for both the API and Event ServiceClass

This ServiceClass has a bindable flag set to true. The provisioning and binding workflow for both the API and Event ServiceClass is a combination of steps described for an API ServiceClass and an Event ServiceClass.

Helm Broker architecture

The Helm Broker workflow starts with the registration process, during which the Helm Broker fetches bundles from the Kyma bundles directory, or from a remote HTTP server.

Registration process

The registration process in case of the Helm Broker consists of the following steps: 1. The Helm Broker fetches bundles from the remote HTTP server. 2. The Helm Broker registers bundles as ServiceClasses in the Service Catalog.

Helm Broker registration

Bundles provisioning and binding

After the registration, the process of bundle provisioning and binding takes place. It consists of the following steps:

  1. Select a given bundle ServiceClass from the Service Catalog.
  2. Provision this ServiceClass by creating its ServiceInstance in the given Environment.
  3. Bind your ServiceInstance to the service or lambda. During the binding process, ServiceBinding and ServiceBindingUsage resources are created.
  4. The service or lambda calls a bundle.

Helm Broker architecture

Custom Resource

EventActivation

The eventactivations.applicationconnector.kyma-project.io CustomResourceDefinition (CRD) is a detailed description of the kind of data and the format used to create an Event Bus Subscription and to get an Event schema. To get the up-to-date CRD and show the output in the yaml format, run this command:

Click to copy
kubectl get crd eventactivations.applicationconnector.kyma-project.io -o yaml

Sample custom resource

This is a sample resource that allows you to consume Events sent from the service with the ac031e8c-9aa4-4cb7-8999-0d358726ffaa ID in a production Namespace.

Click to copy
apiVersion: applicationconnector.kyma-project.io/v1alpha1
kind: EventActivation
metadata:
name: "ac031e8c-9aa4-4cb7-8999-0d358726ffaa"
namespace: production
spec:
displayName: "Orders"
sourceId: "prod"

Custom resource parameters

This table lists all the possible parameters of a given resource together with their descriptions:

ParameterMandatoryDescription
metadata.nameYESSpecifies the name of the CR and the ID of the Application service. This field is also used to fetch Event schemas from the Minio storage.
metadata.namespaceYESSpecifies the Namespace in which the CR is created.
spec.displayNameYESSpecifies a human-readable name of the Application service.
spec.sourceIdYESUsed to construct a Publish-Subscribe (Pub/Sub) topic name where the Events are send and from where the Events are consumed.

These are the resources related to this CR:

Custom resourceDescription
RemoteEnvironmentDescribes a service from which the user receives Events.
SubscriptionContains information on how to create an infrastructure for consuming Events. Works only if the EventActivation is enabled.

These components use this CR:

ComponentDescription
Application BrokerUses this CR to enable the user to receive Events from a given service.
Event BusUses this CR to control the consumption of an Event.
ServerlessLambda UI sends a GraphQL query to UI API Layer to list EventActivations.
UI API LayerExposes the given CR to the Console UI.