Service Catalog

Overview

The Service Catalog is a grouping of reusable, integrated services from all Service Brokers registered in Kyma. Its purpose is to provide an easy way for Kyma users to access services that the Service Brokers manage and use them in their applications.

Due to the fact that Kyma runs on Kubernetes, you can easily instantiate a service instance that a third party provides and maintains, such as a database. You can consume it from Kyma without extensive knowledge about the clustering of such a datastore service and the responsibility for its upgrades and maintenance. You can also easily provision an instance of the software offering that a Service Broker registered in Kyma exposes, and bind it with an application running in the Kyma cluster.

You can perform the following cluster-wide operations in the Service Catalog:

  • Expose the consumable services by listing them with all the details, including the documentation and the consumption plans.
  • Consume the services by provisioning them in a given Environment, which is Kyma's representation of the Kubernetes Namespace.
  • Bind the services to the applications through Secrets.

Details

Resources

This document includes an overview of resources that the Kyma Service Catalog provides.

NOTE: The "Cluster" prefix in front of resources means they are cluster-wide. The corresponding resources without the prefix refer to the Environment scope.

  • ClusterServiceBroker is an endpoint for a set of managed services that a third party offers and maintains.

  • ClusterServiceClass is a managed service exposed by a given ClusterServiceBroker. When a cluster administrator registers a new Service Broker in the Service Catalog, the Service Catalog controller obtains new services exposed by the Service Broker and renders them in the cluster as ClusterServiceClasses. A ClusterServiceClass is synonymous with a service in the Service Catalog.

  • ClusterServicePlan is a variation of a ClusterServiceClass that offers different levels of quality, configuration options, and the cost of a given service. Contrary to the ClusterServiceClass, which is purely descriptive, the ClusterServicePlan provides technical information to the ClusterServiceBroker on this part of the service that the ClusterServiceBroker can expose.

  • Secret is a basic resource to transfer logins and passwords to the Deployment. The service binding process leads to the creation of a Secret.

  • ServiceBinding is a link between a ServiceInstance and an application that cluster users create to obtain access credentials for their applications.

  • ServiceBindingUsage is a Kyma custom resource that allows the ServiceBindingUsage controller to inject Secrets into a given application.

  • ServiceInstance is a provisioned instance of a ClusterServiceClass to use in one or more cluster applications.

  • UsageKind is a Kyma custom resource that allows you to bind a ServiceInstance to any resource.

Add a service to the Catalog

In general, the Service Catalog can expose a service from any Service Broker that is registered in Kyma in accordance with the Open Service Broker API specification.

The Kyma Service Catalog is currently integrated with the following Service Brokers:

  • Azure Broker
  • Remote Environment Broker
  • Helm Broker (experimental)

For details on how to build and register your own Service Broker to expose more services and plans to the cluster users, see the Service Brokers Overview document.

NOTE: The Service Catalog has the Istio sidecar injected. To enable the communication between the Service Catalog and Service Brokers, either inject Istio sidecar into all brokers or disable mutual TLS authentication.

Provisioning and binding

Provisioning a service means creating an instance of a service. When you consume a specific ClusterServiceClass and the system provisions a ServiceInstance, you need credentials for this service. To obtain credentials, create a ServiceBinding resource using the API of the Service Catalog. One instance can have numerous bindings to use in the Deployment or Function. When you raise a binding request, the system returns the credentials in the form of a Secret. The system creates a Secret in a given Environment.

NOTE: The security in Kyma relies on the Kubernetes concept of a Namespace. Kyma Environment is a security boundary. If the Secret exists in the Environment, the administrator can inject it to any Deployment. The Service Broker cannot prevent other applications from consuming a created Secret. Therefore, to ensure a stronger level of isolation and security, use a dedicated Environment and request separate bindings for each Deployment.

The Secret allows you to run the service successfully. However, a problem appears each time you need to change the definition of the yaml file in the Deployment to specify the Secrets' usage. The manual process of editing the file is tedious and time-consuming. Kyma handles it by offering a custom resource called ServiceBindingUsage. This custom resource applies the Kubernetes PodPreset resource and allows you to enforce an automated flow in which the ServiceBindingUsage controller injects ServiceBindings into a given Application or Function.

Details

This section provides a simplified, graphic representation of the basic operations in the Service Catalog.

Provisioning and binding flow

The diagram shows an overview of interactions between all resources related to Kyma provisioning and binding, and the reverting, deprovisioning, and unbinding operations.

Kyma provisioning and binding

The process of provisioning and binding invokes the creation of three custom resources:

  • ServiceInstance
  • ServiceBinding
  • ServiceBindingUsage

The system allows you to create these custom resources in any order, but within a timeout period.

When you invoke the deprovisioning and unbinding actions, the system deletes all three custom resources. Similar to the creation process dependencies, the system allows you to delete ServiceInstance and ServiceBinding in any order, but within a timeout period. However, before you delete the ServiceBinding, make sure you remove the ServiceBindingUsage first. For more details, see the section on deleting a ServiceBinding.

Provision a service

To provision a service, create a ServiceInstance custom resource. Generally speaking, provisioning is a process in which the Service Broker creates a new instance of a service. The form and scope of this instance depends on the Service Broker.

Kyma provisioning

Deprovision a service

To deprovision a given service, delete the ServiceInstance custom resource. As part of this operation, the Service Broker deletes any resources created during the provisioning. When the process completes, the service becomes unavailable.

Kyma deprovisioning

NOTE: You can deprovision a service only if no corresponding ServiceBinding for a given ServiceInstance exists.

Create a ServiceBinding

Kyma binding operation consists of two phases:

  • The system gathers the information necessary to connect to the ServiceInstance and authenticate it. The Service Catalog handles this phase directly, without the use of any additional Kyma custom resources.
  • The system must make the information it collected available to the application. Since the Service Catalog does not provide this functionality, you must create a ServiceBindingUsage custom resource.

Kyma binding

NOTE: The system allows you to create the ServiceBinding and ServiceBindingUsage resources at the same time.

Bind with other resources

The UsageKind is a cluster-wide custom resource which allows you to bind a ServiceInstance to any resource. By default, Kyma provides two UsageKinds which enable binding either to a Deployment or Function. You can add more UsageKinds if you want to bind your ServiceInstance to other types of resources. The UsageKind contains information on the way in which binding to this custom resource is conducted. The ServiceBindingUsage uses this information to inject Secrets to the Application.

Kyma UsageKind

Delete a ServiceBinding

Kyma unbinding operation consists of two phases: 1. Delete the ServiceBindingUsage. 2. Delete the ServiceBinding.

Kyma unbinding

NOTE: The order in which you delete the two resources is important because the ServiceBindingUsage depends on the ServiceBinding. As long as the System Catalog does not automatically block deletions of the ServiceBinding with the ServiceBindingUsage connected to it, follow the recommended deletion order.

See the Corner Case document that explains the consequences of deleting a ServiceBinding for an existing ServieBindingUsage.

Unbinding corner case

As mentioned in the Provisioning and binding document, it is crucial that you follow the order in which you delete the ServiceBindingUsage and the ServiceBinding resources from the Service Catalog during the unbinding operation. According to the deletion process, you must delete the ServiceBindingUsage first and the ServiceBinding afterward, due to dependencies between the two resources.

This diagram shows the consequences of deleting the ServiceBinding for the existing ServiceBindingUsage:

Unbinding Corner case

When you delete a ServiceBinding, the Service Catalog does not populate this information to the ServiceBindingUsage. It is possible that after you delete the ServiceBinding, the Service Catalog does not clearly show that the ServiceBindingUsage no longer works properly.

After you remove the resources in an incorrect order, the application which consumes the resources that are linked using ServiceBindingUsage can be in one of the following states:

  • If you do not restart the application, it still works correctly if the Service Broker does not discredit the injected information.
  • If you do not restart the application, it can stop working correctly if the Service Broker discredits the injected information.
  • If you restart the application, it does not start again because of the missing Secrets.

Etcd Database

The Service Catalog requires an etcd database cluster for a production use. It has a separate etcd cluster defined in the Service Catalog etcd-stateful sub-chart. The etcd-backup-operator executes the backup procedure.

Details

This section describes the backup and restore processes of the etcd cluster for the Service Catalog.

Backup

To execute the backup process, you must set the following values in the core chart:

Property nameDescription
global.etcdBackup.enabledIf set to true, the etcd-operator chart and the Service Catalog sub-chart installs the CronJob which executes periodically the Etcd Backup application. The etcd-operator also creates the Secret with the storage-account and storage-key keys. For more information on how to configure the backup CronJob, see the Etcd Backup documentation.
global.etcdBackup.containerNameThe ABS container to store the backup.
etcd-operator.backupOperator.abs.storageAccountThe name of the storage account for the Azure Blob Storage (ABS). It stores the value for the storage-account Secret key.
etcd-operator.backupOperator.abs.storageKeyThe key value of the storage account for the ABS. It stores the value for the storage-key Secret key.

NOTE: If you set the storageAccount, storageKey, and containerName properties, the global.etcdBackup.enabled must be set to true.

Restore

Follow this instruction to restore an etcd cluster from the existing backup.

  1. Export the ABS_PATH environment variable with the path to the last successful backup file.
Click to copy
export ABS_PATH=$(kubectl get cm -n kyma-system sc-recorded-etcd-backup-data -o=jsonpath='{.data.abs-backup-file-path-from-last-success}')
export BACKUP_FILE_NAME=etcd.backup
  1. Download the backup to the local workstation. You can do it from the portal or by using azure cli. Set the downloaded file path:
Click to copy
export BACKUP_FILE_NAME=/path/to/downloaded/file
  1. Copy the backup file to every running Pod of the StatefulSet.
Click to copy
for i in {0..2};
do
kubectl cp ./$BACKUP_FILE_NAME kyma-system/core-catalog-etcd-stateful-$i:/$BACKUP_FILE_NAME
done
  1. Restore the backup on every Pod of the StatefulSet.
Click to copy
for i in {0..2};
do
remoteCommand="etcdctl snapshot restore /$BACKUP_FILE_NAME "
remoteCommand+="--name core-catalog-etcd-stateful-$i --initial-cluster "
remoteCommand+="core-catalog-etcd-stateful-0=https://core-catalog-etcd-stateful-0.core-catalog-etcd-stateful.kyma-system.svc.cluster.local:2380,"
remoteCommand+="core-catalog-etcd-stateful-1=https://core-catalog-etcd-stateful-1.core-catalog-etcd-stateful.kyma-system.svc.cluster.local:2380,"
remoteCommand+="core-catalog-etcd-stateful-2=https://core-catalog-etcd-stateful-2.core-catalog-etcd-stateful.kyma-system.svc.cluster.local:2380 "
remoteCommand+="--initial-cluster-token etcd-cluster-1 "
remoteCommand+="--initial-advertise-peer-urls https://core-catalog-etcd-stateful-$i.core-catalog-etcd-stateful.kyma-system.svc.cluster.local:2380"
kubectl exec core-catalog-etcd-stateful-$i -n kyma-system -- sh -c "rm -rf core-catalog-etcd-stateful-$i.etcd"
kubectl exec core-catalog-etcd-stateful-$i -n kyma-system -- sh -c "rm -rf /var/run/etcd/backup.etcd"
kubectl exec core-catalog-etcd-stateful-$i -n kyma-system -- sh -c "$remoteCommand"
kubectl exec core-catalog-etcd-stateful-$i -n kyma-system -- sh -c "mv -f core-catalog-etcd-stateful-$i.etcd /var/run/etcd/backup.etcd"
kubectl exec core-catalog-etcd-stateful-$i -n kyma-system -- sh -c "rm $BACKUP_FILE_NAME"
done
  1. Delete old Pods.
Click to copy
kubectl delete pod core-catalog-etcd-stateful-0 core-catalog-etcd-stateful-1 core-catalog-etcd-stateful-2 -n kyma-system

Architecture

The diagram and steps describe the basic Service Catalog workflow:

Service Catalog flow

  1. During the Kyma installation, the system registers the default Service Brokers in the Kyma cluster. The cluster administrator can manually register other Service Brokers in the Kyma cluster.

  2. Each ClusterServiceBroker specifies ClusterServiceClasses and service variations called ClusterServicePlans that are available to Kyma users.

  3. The user gets a list of the available services in the Kyma web console or CLI.

  4. The user chooses a ClusterServiceClass and requests to create its new instance in a given Environment.

  5. The user creates a binding to the ServiceInstance to allow the user's application to access the provisioned service.

CLI reference

Management of the Service Catalog is based on Kubernetes resources and the custom resources specifically defined for Kyma. Manage all of these resources through kubectl.

Details

This section describes the resource names to use in the kubectl command line, the command syntax, and examples of use.

Resource types

Service Catalog operations use the following resources:

Singular namePlural name
clusterserviceclassclusterserviceclasses
clusterserviceplanclusterserviceplans
clusterservicebrokerclusterservicebrokers
serviceinstanceserviceinstances
servicebindingservicebindings
servicebindingusageservicebindingusages
secretsecrets

Syntax

Follow the kubectl syntax, kubectl {command} {type} {name} {flags}, where:

  • {command} is any command, such as describe.
  • {type} is a resource type, such as clusterserviceclass.
  • {name} is the name of a given resource type. Use {name} to make the command return the details of a given resource.
  • {flags} specifies the scope of the information. For example, use flags to define the Namespace from which to get the information.

Examples

The following examples show how to create a ServiceInstance, how to get a list of ClusterServiceClasses and a list of ClusterServiceClasses with human-readable names, a list of ClusterServicePlans, and a list of all ServiceInstances.

  • Create a ServiceInstance using the example of the Redis ServiceInstance for the 0.1.3 version of the Service Catalog:
Click to copy
cat <<EOF | kubectl create -f -
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceInstance
metadata:
name: my-instance
namespace: stage
spec:
clusterServiceClassExternalName: redis
clusterServicePlanExternalName: micro
parameters:
"imagePullPolicy": "Always"
EOF
  • Get the list of all ClusterServiceClasses:
Click to copy
kubectl get clusterserviceclasses
  • Get the list of all ClusterServiceClasses and their human-readable names:
Click to copy
kubectl get clusterserviceclasses -o=custom-columns=NAME:.metadata.name,EXTERNAL\ NAME:.spec.externalName
  • Get the list of all ClusterServicePlans and associated ClusterServiceClasses:
Click to copy
kubectl get clusterserviceplans -o=custom-columns=NAME:.metadata.name,EXTERNAL\ NAME:.spec.externalName,EXTERNAL\ SERVICE\ CLASS:.spec.clusterServiceClassRef
  • Get the list of all ServiceInstances from all Namespaces:
Click to copy
kubectl get serviceinstances --all-namespaces

Custom Resource

ServiceBindingUsage

The servicebindingusages.servicecatalog.kyma.cx Custom Resource Definition (CRD) is a detailed description of the kind of data and the format used to inject Secrets to the application. To get the up-to-date CRD and show the output in the yaml format, run this command:

Click to copy
kubectl get crd servicebindingusages.servicecatalog.kyma.cx -o yaml

Sample custom resource

This is a sample resource in which the ServiceBindingUsage injects a Secret associated with the redis-instance-binding ServiceBinding to the redis-client Deployment in the production Namespace. This example has the conditions.status field set to true, which means that the ServiceBinding injection is successful. If this field is set to false, the message and reason fields appear.

Click to copy
apiVersion: servicecatalog.kyma.cx/v1alpha1
kind: ServiceBindingUsage
metadata:
name: redis-client-binding-usage
namespace: production
spec:
serviceBindingRef:
name: redis-instance-binding
usedBy:
kind: deployment
name: redis-client
parameters:
envPrefix:
name: "pico-bello"
status:
conditions:
- lastTransitionTime: 2018-06-26T10:52:05Z
lastUpdateTime: 2018-06-26T10:52:05Z
status: "True"
type: Ready

Custom resource parameters

This table lists all the possible parameters of a given resource together with their descriptions:

ParameterMandatory?Description
metadata.nameYESSpecifies the name of the CR.
metadata.namespaceYESSpecifies the Namespace in which the CR is created.
spec.serviceBindingRef.nameYESSpecifies the name of the ServiceBinding.
spec.usedByYESSpecifies the application into which the Secret is injected.
spec.usedBy.kindYESSpecifies the name of the UsageKind.
spec.usedBy.nameYESSpecifies the name of the application.
spec.parameters.envPrefixNODefines the prefix of environment variables environment variables that the ServiceBindingUsage injects. The prefixing is disabled by default.
spec.parameters.envPrefix.nameYESSpecifies the name of the prefix. This field is mandatory if envPrefix is specified.
status.conditionsNOSpecifies the state of the ServiceBindingUsage.
status.conditions.lastTransitionTimeNOSpecifies the time when the Binding Usage Controller processes the ServiceBindingUsage for the first time or when the status.conditions.status field changes.
status.conditions.lastUpdateTimeNOSpecifies the time of the last ServiceBindingUsage condition update.
status.conditions.statusNOSpecifies whether the status of the status.conditions.type field is True or False.
status.conditions.typeNODefines the type of the condition. The value of this field is always Ready.
messageNODescribes in a human-readable way why the ServiceBinding injection has failed.
reasonNOSpecifies a unique, one-word, CamelCase reason for the condition's last transition.

UsageKind

The usagekinds.servicecatalog.kyma.cx Custom Resource Definition (CRD) is a detailed description of the kind of data and the format used to define which resources can be bound with the ServiceBinding and how to bind them. To get the up-to-date CRD and show the output in the yaml format, run this command:

Click to copy
kubectl get crd usagekinds.servicecatalog.kyma.cx -o yaml

Sample custom resource

This is a sample resource that allows you to bind a given resource with the ServiceBinding. This example has a resource section specified as function. You can adjust this section to point to any other kind of resource.

Click to copy
apiVersion: servicecatalog.kyma.cx/v1alpha1
kind: UsageKind
metadata:
name: function
spec:
displayName: Function
resource:
group: kubeless.io
kind: function
version: v1beta1
labelsPath: spec.deployment.spec.template.metadata.labels

Custom resource parameters

This table lists all the possible parameters of a given resource together with their descriptions:

ParameterMandatory?Description
metadata.nameYESSpecifies the name of the CR.
spec.displayNameYESSpecifies a human-readable name of the UsageKind.
spec.resourceYESSpecifies a resource which is bound with the ServiceBinding. The target resource is specified by its resource group, kind, and version.
spec.resource.groupYESSpecifies the group of the resource.
spec.resource.kindYESSpecifies the kind of the resource.
spec.resource.versionYESSpecifies the version of the resource.
spec.labelsPathYESSpecifies a path to the key that contains labels which are later injected into Pods.