Service Mesh

Overview

Kyma Service Mesh is the component responsible for service-to-service communication, proxying, service discovery, traceability, and security. Kyma Service Mesh is based on Istio open platform. The main principle of Kyma Service Mesh operation is the process of injecting Pods of every service with an Envoy - a sidecar proxy which intercepts the communication between the services and regulates it by applying and enforcing the rules you create. Kyma Dex, which is also a part of the Service Mesh, allows you to integrate any OpenID Connect-compliant identity provider or a SAML2-based enterprise authentication server with your solution.

By default, the Kyma implementation of Istio has mutual TLS (mTLS) enabled. The only part of the Service Mesh that doesn't have mTLS enabled is the Mesh Control Plane. Read this document for more information about mTLS in Istio.

See this Istio diagram to understand the relationship between the Istio components and Services.

Details

Sidecar Proxy Injection

By default, the Istio sidecar injector watches all Pod creation operations on all Namespaces and injects the newly created Pods with a sidecar proxy.

You can disable sidecar proxy injection for either an entire Namespace or a single Deployment.

  • To disable sidecar proxy injection for a Namespace, set the istio-injection label value to disabled for the Namespace in which you want to disable the sidecar proxy injection. Use this command: kubectl label namespace {YOUR_NAMESPACE} istio-injection=disabled
  • To disable sidecar proxy injection for a Deployment, add this annotation to the Deployment configuration file: sidecar.istio.io/inject: "false"

Read the Installing the Sidecar document to learn more about sidecar proxy injection.

Istio patch

As a core component, Istio installs with every Kyma deployment by default. The installation consists of two steps:

  1. Istio installs using the official charts from the currently supported release. The charts that are currently used are stored in the resources/istio directory. The installation is customized as described in this document

  2. A custom Istio patch job runs and checks if the detected Istio deployment meets the following criteria:

  • A specific version of Istio is installed. The required version is defined in the values file of the patch.
  • Mutual TLS (mTLS) policy is enabled and set to strict.
  • Istio policy enforcement is enabled.
  • Automatic sidecar injection is enabled.
  • Istio policies.authentication.istio.io CustomResourceDefinition (CRD) is present in the system.

If the Istio deployment fails to meet any of these criteria, the patch fails, which results in a failed installation. If the installation failed, get the Istio patch logs to find out which criteria the Istio deployment didn't' meet. Run:

Click to copy
kubectl logs -n istio-system -l app=istio-kyma-patch

To learn more about the custom Istio patch applied in Kyma, see the components/istio-kyma-patch/ directory.

NOTE: Istio patch is an optional component and can be disabled. However, it is enabled by default. Read this to learn how to disable components.

Use an existing Istio installation with Kyma

You can use an existing installation of Istio running on Kubernetes with Kyma. In such a case is required to enable the patch component to verify if all the required options are set.

To allow such implementation, you must install Kyma without Istio. Read this document for more details.

Istio RBAC configuration

As a core component, Istio is installed with Kyma by default. The ClusterRbacConfig custom resource (CR), which defines the global behavior of Istio, is created as a part of the installation process.

The default Istio RBAC configuration is defined in this file.

Override the default configuration

To override the default configuration of Istio RBAC, edit the ClusterRbacConfig CR on a running cluster. This CR is created in the kyma-system Namespace and therefore requires admin permissions to edit it.

To show the current Istio RBAC configuration in the yaml format, run:

Click to copy
kubectl get -n kyma-system clusterrbacconfig -o yaml

To edit the Istio RBAC configuration, run:

Click to copy
kubectl edit -n kyma-system clusterrbacconfig

NOTE: The ClusterRbacConfig object is a singleton, which means that only a single object of this kind can exist in a cluster. Additionally, the only valid name for the object is default. As such, the best way to customize Istio RBAC is by editing the existing ClusterRbacConfig object.

Istio setup in Kyma

Istio is installed using the official charts from the currently supported release. The charts are customized for Kyma and are stored in the resources/istio directory.

Istio components

This list shows the available Istio components and the component enabled by default:

ComponentEnabled?
Gateways
Sidecar Injector
Galley
Mixer
Pilot
Security
Node agent⛔️
Grafana⛔️
Prometheus⛔️
Servicegraph⛔️
Tracing⛔️
Kiali⛔️

Kyma-specific configuration

These configuration changes are applied to customize Istio for use with Kyma:

  • Only the Ingress gateway is enabled.
  • The Secret Discovery Service is enabled in the Ingress Gateway.
  • Automatic sidecar injection is enabled by default, excluding the istio-system and kube-system Namespaces.
  • New resource limits for Istio sidecars are introduced: CPU: 100m, memory: 128Mi.
  • Mutual TLS (mTLS) is enabled cluster-wide with the exception of the Istio Control Plane.
  • Global tracing is set to use the Zipkin installation provided by Kyma.
  • Ingress Gateway is expanded to handle ports 80 and 443 for local Kyma deployments.
  • DestinationRules are created by default, which disables mTLS for the istio-ingressgateway.istio-system.svc.cluster.local service.

The Customization sub-chart

The Kyma-specific configuration is applied through the proprietary customization sub-chart added to the official Istio charts. The sub-chart is located in the resources/istio/charts/customization directory.

Troubleshooting

Overview

The troubleshooting section aims to identify the most common recurring problems with the Kyma Service Mesh, as well as the most suitable solutions to these problems.

If you can't find a solution to your problem, don't hesitate to create a GitHub issue or reach out to the "service-mesh" Slack channel to get direct support from the community.

Can't access Console UI or other endpoints

The 503 status code received when you try to access the Console UI or any other endpoint in Kyma can be caused by a configuration error in the Istio Ingress gateway. As a result, the endpoint you call is not exposed. To fix this problem, restart the Pods of the gateway.

  1. List all available endpoints:

    Click to copy
    kubectl get virtualservice --all-namespaces
  2. Check all ports used by the gateway:

    Click to copy
    kubectl exec -t -n istio-system $(kubectl get pod -l app=istio-ingressgateway -n istio-system | grep "istio-ingressgateway" | awk '{print $1}') -- netstat -lptnu
  3. If ports 80 and 443 are not used, restart the Pods of the gateway to force them to recreate their configuration:

    Click to copy
    kubectl delete pod -l app=istio-ingressgateway -n istio-system

Connection refused errors after upgrade

Starting with Kyma 1.0, mutual TLS (mTLS) is enabled in the Service Mesh by default. As a result, every element of the Service Mesh must have an Istio sidecar to allow TLS communication. Alternatively, you can whitelist a service and disable TLS traffic for it by creating a DestinationRule.

  • To enable sidecar injection for Pods of existing Services, restart them after upgrading to Kyma 1.0 or higher.
  • To create a DestinationRule for a Service, follow the official Istio documentation.