Adopting IAM Role for Service Accounts (IRSA) for credential propagation in Domino
Article topics
Security, Identity and Access Management (IAM), Role Based Access Control (RBAC), OpenID Connect (OIDC), Web Identity and Attribute Based Access Control (ABAC)
Intended audiences
Cloud administrators, IT professionals, Security teams
Overview and goals
Domino’s Enterprise AI Platform is built on top of the Kubernetes framework. Domino Nexus is a comprehensive platform add-on that allows you to execute data science and machine learning workloads across any compute cluster. That cluster can reside in the cloud, a specific cloud region, or on-premises. By unifying data science silos across the enterprise, Domino Nexus provides a centralized hub for building, deploying, and monitoring models.
Such a hybrid architecture enables a wide range of use cases. For example, you could perform data processing in the AWS cloud and store a training dataset in S3. Next, you could trigger an ML training job on a GPU on an on-premises data plane for better cost performance.
The fact that control and execution take place on different platforms, may raise several questions:
- How should the on-premises data plane authenticate to AWS IAM to read data from S3?
- Can the user assume an IAM role transparently based on the policy defined in the agreement between Domino and the AWS Administration teams?
- How can two teams working on separate Domino data planes that are hosted on different cloud providers consume data or resources from each other's host cloud provider? Each data plane will need to assume an identity in the other cloud.
Using Web Identities based on an OpenID Connect Provider supports such use-cases. In this article, we will describe the setup where using IAM Role for Service Accounts (IRSA) we can configure Kubernetes Service Accounts attached to an EKS workload to assume IAM Role(s). This is similar to attaching an IAM Role to an EC2 instance at startup. This mechanism allows EKS workloads to assume IAM roles across AWS Accounts. You are not limited to assuming an IAM role in the AWS Account hosting the EKS cluster.
Note: This document will provide you with detailed guidance. Please speak with your Domino Professional Services contact for additional information, assistance, and implementation support.
When should you consider using IRSA with Domino?
- Does your enterprise policy forbid the use of long-lived credentials?
- Would you like to transparently and securely propagate the IAM Role to all Domino workloads (Apps, Jobs, and Workspaces), including a Domino Scheduled Job?
- Would you like to configure a Domino workload with a set of IAM Roles based on a set of complex custom criteria?
Why does using IRSA with Domino help achieve the above goals?
- When using the IRSA support in Domino, any Domino-hosted workload can assume IAM roles in any AWS account configured to trust the Domino Enterprise AI Platform
- IRSA support in Domino allows the configuration of multiple roles for a Domino workload where the boto3 library can switch across the multiple profiles configured in the ~/.aws/config file.
- The mapping of a workload to a set of roles can be configured based on custom configurable rules. This allows for complex mappings to be supported.
How does IRSA work with Domino?
- For a Domino workload (implemented as a K8s Pod) to assume an IAM Role, it must use the STS:AssumeRoleWithWebIdentity API which returns temporary security credentials. But first the caller (in this case, the pod) needs to be authenticated with a Web Identity Provider that is compatible with the OpenID connect (OIDC) protocol.
- Starting 1.12 Kubernetes allows an OIDC provider to be associated with it. An EKS Cluster provides a unique OIDC provider bound to the KubeAPIServer server. A JSON Web Token (JWT) created by this OIDC provider can be mounted as a projected volume in the Pod. This JWT asserts the identity of the K8s Service Account associated with the workload.
- Next, the OIDC provider attached to the EKS cluster is configured as an Identity Provider in the AWS Identity and Access Management (IAM) service. This configuration is done within the AWS account the Domino workloads want to assume an IAM Role. This allows Domino workloads to assume IAM roles across multiple AWS accounts. Speak to your Domino representative about the options.
- An AWS Admin will now need to configure the IAM Role Trust Policy to enable a specific K8s Service Account (as identified by the JWT above) to assume the Role. AWS Security Token Service (STS) will validate the workload Identity and return temporary IAM Role Credentials to the workload if an IAM Role Trust Policy allows it. This is the simplest approach offering high availability of credential propagation under AWS loads. More complex configurations are also possible. Domino Professional Services can provide guidance based on your unique Infosec requirements.
- Domino’s adaptation of IRSA operates on a shared trust policy. A Domino mutating webhook framework called Domsed configures a Domino workload definition to enable passing an identity issued by the OIDC Provider to IAM. We will refer to this identity as the Workload Identity. An AWS Administrator on the other hand configures a trust relationship between an IAM Role and this Workload Identity. This allows a Domino workload to assume one or more roles if those IAM Roles have their Trust Policy configured to permit the workload identity to assume them.
Additional Domino Blueprints articles will explore how cloud provider neutrality allows workloads in Domino data planes to assume AWS IAM Roles and still run on-premises or in other cloud providers like GCP and Azure.
