Cannot connect to a service exposed by an API Rule - 401 Unauthorized or 403 Forbidden


When you try to reach your service, you get 401 Unauthorized or 403 Forbidden in response.


Make sure that the following conditions are met:

  • You are using an access token with proper scopes and it is active:

    1. Export the credentials of your OAuth2Client as environment variables:

      NOTE: Export the CLIENT_NAMESPACE and CLIENT_NAME variables before you proceed with step 1.

      Click to copy
      export CLIENT_ID="$(kubectl get secret -n $CLIENT_NAMESPACE $CLIENT_NAME -o jsonpath='{.data.client_id}' | base64 --decode)"
      export CLIENT_SECRET="$(kubectl get secret -n $CLIENT_NAMESPACE $CLIENT_NAME -o jsonpath='{.data.client_secret}' | base64 --decode)"
    2. Encode your client credentials and export them as an environment variable:

      Click to copy
      export ENCODED_CREDENTIALS=$(echo -n "$CLIENT_ID:$CLIENT_SECRET" | base64)
    3. Check the access token status:

      Click to copy
      curl -X POST "https://oauth2.{CLUSTER_DOMAIN}/oauth2/introspect" -H "Authorization: Basic $ENCODED_CREDENTIALS" -F "token={ACCESS_TOKEN}"
    4. Generate a new access token if needed.

  • Your client from the OAuth2Client resource is registered properly in Hydra OAuth2 and the OpenID Connect server. You need to call the Hydra administrative endpoint /client from inside of the cluster. Follow these steps:

    1. Fetch the Client ID from the Secret specified in the OAuth2Client resource:

      Click to copy
      kubectl get secrets {SECRET_NAME} -n {SECRET_NAMESPACE} -o jsonpath='{ .data.client_id }' | base64 --decode
    2. Create a simple curl Pod:

      Click to copy
      cat <<EOF | kubectl apply -f -
      apiVersion: v1
      kind: Pod
      app: ory-curl
      name: ory-curl
      namespace: {SECRET_NAMESPACE}
      - name: curl
      image: alpine
      terminationMessagePolicy: "FallbackToLogsOnError"
      - /bin/sh
      - -c
      - |
      apk add curl jq
      curl ory-hydra-admin.kyma-system.svc.cluster.local:4445/clients | jq '.'
    3. Check logs from the ory-curl Pod:

      Click to copy
      kubectl logs -n {SECRET_NAMESPACE} ory-curl curl
    4. If the Client ID from step 1 is not available on the client list, make sure Hydra has access to the database and/or restart the Hydra Measter Pod. You can check the logs using the following commands:

      Click to copy
      # Check logs from the Hydra-Maester controller application
      kubectl logs -n kyma-system -l "" -c hydra-maester
      # Example output
      2020-05-04T12:19:04.472Z INFO controller-runtime.controller Starting EventSource {"controller": "oauth2client", "source": "kind source: /, Kind="}2020-05-04T12:19:04.472Z INFO setup starting manager
      2020-05-04T12:19:04.573Z INFO controller-runtime.controller Starting Controller {"controller": "oauth2client"}
      2020-05-04T12:19:04.673Z INFO controller-runtime.controller Starting workers {"controller": "oauth2client", "worker count": 1}
      2020-05-04T12:26:30.819Z INFO controllers.OAuth2Client using default client
      2020-05-04T12:26:30.835Z INFO controllers.OAuth2Client using default client
      # This log entry informs us that a client has been created, and should be visible within Hydra
      2020-05-04T12:26:31.468Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "oauth2client", "request": "test-ns/test-client"}
      Click to copy
      # Check logs from the Hydra application
      kubectl logs -n kyma-system -l "" -c hydra