RBAC Cluster sync (KubeConfig) 🖥️

Managing cluster permissions has never been easier with Komodor.

Introduction

Some users prefer interacting with their clusters using the terminal and CLI tools, but managing Kubernetes RBAC for multiple clusters can be a tedious task for administrators.

The new RBAC Cluster Sync feature by Komodor simplifies this process, allowing cluster administrators to set permissions for multiple resources across multiple clusters from a single, easy-to-use interface, affecting both the Komodor UI and CLI interfaces for their users.

Overview

RBAC Cluster Sync is an opt-in feature, allowing all users in the account to request for a kubeconfig and allow them to interact with the clusters directly using kubectl or other tools
Admins can grant a default-allow-get-kubeconfig policy, enabling users to download a kubeconfig file for the clusters they have access to.
Users can go to their settings page and fetch a kubeconfig containing all clusters they have access to.
Komodor automatically syncs cluster permissions and komodor permissions in near-real-time.

--In order for the feature to work and for users to be able to see the Get KubeConfig button in the UI a pre requisite is to activate the feature via the features tab under Organization Settings--

How does it work?

Komodor translates user roles and permissions set in the platform into a set of Kubernetes resources: ServiceAccounts, ClusterRoles, RoleBindings, and Secrets. These resources are then synchronized to the cluster whenever there is a change in the user’s permissions.

When a user requests a kubeconfig, the Komodor backend retrieves permissions for all clusters the user has access to, generates a token for each, and compiles a kubeconfig file with all clusters included as contexts.

When a user employs a kubeconfig generated by Komodor, they authenticate against the Kubernetes API using their generated service account and the token, thereby gaining all the permissions associated with their service account.

FAQ

Will I need re-download a kubeconfig every time I change the policy?

It depends on the change. As long as the user does not gain access to a new cluster, their token remains valid, and they will gain new permissions seamlessly without needing to re-download the kubeconfig.

What happens if I delete a user?

When a user is deleted, Komodor will remove their service account from the cluster. Any previously downloaded kubeconfig files will no longer allow the user to authenticate against the cluster.

What are those new komodor-xxx resources created in my cluster

When feature is enabled, komodor backend creates service accounts cluster roles and bindings for all users in the account. They can be easily identified using the komodor-prefix

Is there any ingress needed for a Cluster Server URL?

The Komodor backend does not require any access to the API server, the connection is done directly from the clients to the server url. there is no traffic proxying via komodor.

To manage your cluster using the KubeConfig provided by Komodor, you can use one of the following approaches to ensure connectivity:

Expose the Kubernetes Service to Your Client's Network
Configure your Kubernetes service to be accessible from your client’s network, ensuring the required security measures are in place.
Set Up a Jump Host
Deploy a jump host within the cluster’s network. Engineers can connect to this jump host, which acts as a gateway for secure access to the cluster.
Utilize a VPN
Establish a VPN connection to the cluster’s network, allowing secure and seamless access for cluster management.

Each method offers a secure way to manage your cluster, and the choice depends on your network configuration and security policies.

I want to connect to a cluster, but it is set to unavailable, what should i do?

There are multiple reasons why a cluster may be unavailable:

Komodor-agent Chart Version is Below 2.5.4:
- Ensure that the komodor-agent chart version is updated to at least 2.5.4.
Komodor Syncing error with the Cluster:
- Manual changes to the Kubernetes resources created automatically by komodor or network connectivity issues - contact komodor support

My downloaded kubeconfig has an empty server address. What should I do?

Komodor retrieves the external cluster URL automatically for EKS and AKS clusters. For other clusters, for example GKE, it may not find one. The kubeconfig then still includes the cluster, but with server: "" and a komodor/errors extension that lists what went wrong (for example did not find kubeconfig in secret data).

To fix it, set the cluster's API server URL once, either in the Komodor UI or with the komodor-agent Helm chart, and then download the kubeconfig again. See how to configure cluster URLs.

Guides

How to - Enable RBAC cluster sync

As not all account admins want to allow access to kuebctl by default. RBAC Cluster Sync is an opt-in feature, once the feature is enabled. Komodor will automatically start syncing all users and policies onto the cluster.

Steps to Configure Cluster URLs:

Go to the Settings page.
Under Features → RBAC Cluster Sync, click Enable.

Click I understand, enable.

How to - Configure API Server URLs

Komodor automatically detects external URLs for EKS and AKS clusters needed for kubectl access. However, in many cases, access to the cluster URL must be done using a proxy or a VPN.

To address this need and to support additional cloud providers, an administrator can set the cluster API URL, which will then be used in the kubeconfig file. You can set it in the Komodor UI (option 1) or with the komodor-agent Helm chart (option 2).

Option 1 – In the Komodor UI:

Go to the Settings page.
Under Features → RBAC Cluster Sync, click Configure.
Add API Server URL for the cluster required
Click Save.

Note: Until a URL is set, a kubeconfig downloaded for the cluster has an empty server address and can't be used to connect to it.

Option 2 – With the komodor-agent Helm chart (chart version 2.10.0 or later):

Add these values to the cluster's komodor-agent installation:

capabilities:
  rbacClusterSyncParams:
    enabled: true
    clusterURL: https://YOUR-API-SERVER-ADDRESS

Or with helm upgrade:

--set capabilities.rbacClusterSyncParams.enabled=true --set capabilities.rbacClusterSyncParams.clusterURL=https://YOUR-API-SERVER-ADDRESS

clusterURL is used only when enabled is true.
Komodor picks up the URL after the agent restarts with the new values. Then download the kubeconfig again.
If a URL is set both here and in the UI, the Helm value is used.
By default, the kubeconfig's certificate-authority-data is read from the ca.crt key of the kube-system/kube-root-ca.crt ConfigMap. If the API server URL presents a certificate signed by a different CA (for example, behind an ingress or proxy), set capabilities.rbacClusterSyncParams.ingressCertConfiguration to the resource that holds it: namespace, kind (Secret or ConfigMap), name, and dataPath (the key that holds the certificate).

How to - Allow Users to Download a Kubeconfig

Steps to grant access to kubeconfig feature:

Go to the Komodor settings page.

For the user you want to allow downloading kubeconfig for click the Edit icon
Add policy default-allow-get-kubeconfig to the list of policies of the role default-account-admin gets the ability to download by default

click Save

How to - Download and use a Kubeconfig File

Steps to download a kubeconfig file:

Go to the Komodor settings page.

Click Get Kubeconfig

Select the clusters you want included in your kubeconfig file from the list of available clusters and click "Download Kubeconfig"

A kubeconfig file will be generated shortly, allowing you to access the clusters using standard kubectl commands.

# Running a Single Command with the Kubeconfig File
kubectl --kubeconfig <path-to-downloaded-kubeconfig-file> --context <cluster-name-in-komodor>

# Changing context
kubectl --kubeconfig <path-to-downloaded-kubeconfig-file> use-context <context-name>

# Using as a kubeconfig file
export KUBECONFIG=~/.kube/config
kubectl config use-context <context-name>

How to - Create custom permissions

Kubernetes permissions are very extensible, allowing granular permissions to different resources. For cases where a user wants to grant specific permissions to specific resources, Komodor now offers a “create action” wizard. All actions created by Komodor’s users are account-exclusive and only apply to the clusters’ kubectl permissions.

In settings page, click the new Actions tab, then click Add action
Pick an action Name, this name will be used in policies, Description is optional and will help you distinguish between policies