Construct a safe knowledge visualization utility utilizing the Amazon Redshift Knowledge API with AWS IAM Id Middle

In at this time’s data-driven world, securely accessing, visualizing, and analyzing knowledge is important for making knowledgeable enterprise choices. Tens of 1000’s of shoppers use Amazon Redshift for contemporary knowledge analytics at scale, delivering as much as thrice higher price-performance and 7 occasions higher throughput than different cloud knowledge warehouses.

The Amazon Redshift Knowledge API simplifies entry to your Amazon Redshift knowledge warehouse by eradicating the necessity to handle database drivers, connections, community configurations, knowledge buffering, and extra.

With the newly launched function of Amazon Redshift Knowledge API assist for single sign-on and trusted identification propagation, you may construct knowledge visualization purposes that combine single sign-on (SSO) and role-based entry management (RBAC), simplifying person administration whereas imposing acceptable entry to delicate info.

As an illustration, a worldwide sports activities gear firm promoting merchandise throughout a number of areas wants to visualise its gross sales knowledge, which incorporates country-level particulars. To keep up the suitable stage of entry, the corporate desires to limit knowledge visibility primarily based on the person’s position and area. Regional gross sales managers ought to solely see gross sales knowledge for his or her particular area, resembling North America or Europe. Conversely, the worldwide gross sales executives require full entry to the complete dataset, overlaying all international locations.

On this put up, we dive into the newly launched function of Amazon Redshift Knowledge API assist for SSO, Amazon Redshift RBAC for row-level safety (RLS) and column-level safety (CLS), and trusted identification propagation with AWS IAM Id Middle to let company identities hook up with AWS providers securely. We display the way to combine these providers to create a knowledge visualization utility utilizing Streamlit, offering safe, role-based entry that simplifies person administration whereas ensuring that your group could make data-driven choices with enhanced safety and ease.

Resolution overview

We use a number of AWS providers and open supply instruments to construct a easy knowledge visualization utility with SSO to entry knowledge in Amazon Redshift with RBAC. The important thing parts that energy the answer are as follows:

• IAM Id Middle and trusted identification propagation – IAM Id Middle can simplify person administration by enabling SSO throughout AWS providers. This permits customers to authenticate with their company credentials managed of their company identification supplier (IdP) like Okta, offering seamless entry to the appliance. We discover how trusted identification propagation allows managing application-level entry management at scale and exercise logging throughout AWS providers, like Amazon Redshift, by propagating and sustaining identification context all through the workflow.
• Exterior IdP – We use Okta as an exterior IdP to handle person authentication. Okta connects to IAM Id Middle, permitting customers to authenticate from exterior programs whereas sustaining centralized identification administration inside AWS. This makes certain that person entry and roles are persistently maintained throughout each AWS providers and exterior instruments.
• Amazon Redshift Serverless workgroup, Amazon Redshift Knowledge API, and Amazon Redshift RBAC – Amazon Redshift is a completely managed knowledge warehouse service that enables for quick querying and evaluation of huge datasets. On this answer, we use the Redshift Knowledge API, which provides a easy and safe HTTP-based connection to Amazon Redshift, eliminating the necessity for JDBC or ODBC driver-based connections. The Redshift Knowledge API is the really useful technique to attach with Amazon Redshift for internet purposes. We additionally use RBAC in Amazon Redshift to display entry restrictions on gross sales knowledge primarily based on the area column, ensuring that regional gross sales managers solely see knowledge for his or her assigned areas, whereas international gross sales managers have full entry.
• Streamlit utility – Streamlit is a broadly used open supply instrument that permits the creation of interactive knowledge purposes with minimal code. On this answer, we use Streamlit to construct a user-friendly interface the place gross sales managers can view and analyze gross sales knowledge in a visible, accessible format. The applying will combine with Amazon Redshift, offering customers with entry to the info primarily based on their roles and permissions.

The next diagram illustrates the answer structure for SSO with the Redshift Knowledge API utilizing IAM Id Middle.

The person workflow for the info visualization utility consists of the next steps:

1. The person (whether or not a regional gross sales supervisor or international gross sales supervisor) accesses the Streamlit utility, which is built-in with SSO to supply a seamless authentication expertise.
2. The applying redirects the person to authenticate by Okta, the exterior IdP. Okta verifies the person’s credentials and returns an ID token to the appliance.
3. The applying makes use of the token issued by Okta to imagine a task and momentary AWS Id and Entry Administration (IAM) session credentials to name the IAM Id Middle AssumeRoleWithWebIdentity API and IAM AssumeRole API in later steps.
4. The applying exchanges the Okta ID token for a token issued by IAM Id Middle by calling the IAM Id Middle CreateTokenWithIAM API utilizing the momentary IAM credentials from the earlier step. This token makes certain that the person is authenticated with AWS providers and is tied to the IAM Id Middle person profile.
5. The applying requests an identity-enhanced IAM position session utilizing the IAM Id Middle token by calling the AssumeRole
6. The applying makes use of the identity-enhanced IAM position session credentials to securely question Amazon Redshift for gross sales knowledge. The credentials make it possible for solely licensed customers can work together with the Redshift knowledge.
7. Because the question is processed, Amazon Redshift checks the identification context offered by IAM Id Middle. It verifies the person’s position and group membership, resembling being part of the North American area or the worldwide gross sales supervisor group.
8. Based mostly on the person’s identification and group membership, and utilizing Amazon Redshift RBAC and row-level safety, Amazon Redshift makes an authorization choice. The teams for the illustration could be broadly labeled into the next classes:
   1. Regional gross sales managers can be granted entry to view gross sales knowledge just for the particular nation or area they handle. As an illustration, the AMER North American Gross sales Supervisor will solely see gross sales knowledge associated to North America. Equally, the entry management primarily based on EMEA and APAC areas will present row-level safety for the respective areas.
   2. The worldwide gross sales managers can be granted full entry to all areas, enabling them to view the complete international dataset.

The setup consists of two important steps:

1. Provision the sources for IAM Id Middle, Amazon Redshift and Okta:
   1. Allow IAM Id Middle and configure Okta because the IdP to handle person authentication and group provisioning.
   2. Create an Okta utility to authenticate customers accessing the Streamlit utility.
   3. Arrange an Amazon Redshift IAM Id Middle connection utility to allow trusted identification propagation for safe authentication.
   4. Provision an Amazon Redshift Serverless
   5. Create the tables and configure RBAC throughout the Redshift workgroup to implement row-level safety for various IAM Id Middle federated roles, mapped to IAM Id Middle teams.
2. Obtain, configure, and run the Streamlit utility:
   1. Create a buyer managed utility in IAM Id Middle for the Redshift Knowledge API shopper (Streamlit utility) to allow safe API-based queries and create the required IAM roles
   2. Configure the Streamlit utility.
   3. Run the Streamlit utility.

Conditions

It’s best to have the next conditions:

Provision the sources for IAM Id Middle, Amazon Redshift, and Okta

On this part, we stroll by the steps to provision the sources for IAM Id Middle, Amazon Redshift, and Okta.

Allow IAM Id Middle and configure Okta because the IdP

Full the next steps to allow IAM Id Middle and configure Okta because the IdP to handle person authentication and group provisioning:

1. Create the next customers and teams in Okta:
   1. Ethan International with e-mail [email protected], in group exec-global
   2. Frank Amer with e-mail [email protected], in group amer-sales
   3. Alex Emea with e-mail [email protected], in group emea-sales
   4. Ming Apac with e-mail [email protected], in group apac-sales

2. Create an IAM Id Middle occasion within the AWS Area the place Amazon Redshift goes to be deployed. A corporation occasion sort is really useful.
3. Configure Okta because the identification supply and allow automated person and group provisioning. The customers and teams can be pushed to IAM Id Middle utilizing SCIM protocol.

The next screenshot reveals the customers synced in IAM Id Middle utilizing SCIM protocol.

Create an Okta utility

Full the next steps to create an Okta utility to authenticate customers accessing the Streamlit utility:

1. Create an OIDC utility in Okta.
   1. Copy and save the shopper ID and shopper secret wanted later for the Streamlit utility and the IAM Id Middle utility to attach utilizing the Redshift Knowledge API.
   2. Generate the shopper secret and set sign-in redirect URL and sign-out URL to http://localhost:8501 (we are going to host the Streamlit utility domestically on port 8501).
   3. Beneath Assignments, Managed entry, grant entry to everybody.
2. Create an OIDC IdP on IAM the console. The next screenshot reveals an IdP created on the IAM console.

Arrange an Amazon Redshift IAM Id Middle connection utility

Full the next steps to create an Amazon Redshift IAM Id Middle connection utility to allow trusted identification propagation for safe authentication:

1. On the Amazon Redshift console, select IAM Id Middle connection within the navigation pane.
2. Select Create utility.
3. Title the appliance redshift-data-api-okta-app.
4. Observe down the IdP namespace. The default worth AWSIDC is used for this put up.
5. Within the IAM position for IAM Id Middle entry part, it’s essential present an IAM position. You’ll be able to go to the IAM console and create an IAM position referred to as RedshiftOktaRole with the next coverage and belief relationship. RedshiftOktaRole is utilized by the Amazon Redshift IAM Id Middle connection utility to handle and work together with IAM Id Middle.
   1. The coverage hooked up to the position wants the next permissions:

      {
        "Model": "2012-10-17",
        "Assertion": [
          {
            "Effect": "Allow",
            "Action": [
              "sso:DescribeApplication",
              "sso:DescribeInstance"
            ],
            "Useful resource": [
              "arn:aws:sso:::instance/",
              "arn:aws:sso:::application//*"
            ]
          }
        ]
      }

   2. The position makes use of the next belief relationship:

      {
        "Model": "2012-10-17",
        "Assertion": [
          {
            "Effect": "Allow",
            "Principal": {
              "Service": [
                "redshift.amazonaws.com",
                "redshift-serverless.amazonaws.com"
              ]
            },
            "Motion": [
              "sts:AssumeRole",
              "sts:SetContext"
            ]
          }
        ]
      }

6. Go away Trusted Id propagation part unchanged, then select Subsequent. You could have the choice to decide on AWS Lake Formation or Amazon S3 Entry Grants to be used circumstances like utilizing Amazon Redshift Spectrum to question exterior tables in Lake Formation. In our use case, we solely use Amazon Redshift native tables so we don’t select both.
7. Within the Configure shopper connections that use third-party IdPs part, select No.
8. Overview and select Create utility.
9. When the appliance is created, navigate to your IAM Id Middle connection redshift-data-api-okta-app and select Assign so as to add the teams that had been synced in IAM Id Middle utilizing SCIM protocol from Okta.

We’ll allow trusted identification propagation and third-party IdP (Okta) on the shopper managed utility for the Redshift Knowledge API in a later step as a substitute of configuring it within the Amazon Redshift connection utility.

The next screenshot reveals the IAM Id Middle connection utility created on the Amazon Redshift console.

The next screenshot reveals teams assigned to the Amazon Redshift IAM Id Middle connection for the managed utility.

Provision a Redshift Serverless workgroup

Full the next steps to create a Redshift Serverless workgroup. For extra particulars, seek advice from Making a workgroup with a namespace.

1. On the Amazon Redshift console, navigate to the Redshift Serverless dashboard.
2. Select Create workgroup.
3. Enter a reputation in your workgroup (for instance, redshift-tip-enabled).
4. Change the Base capability to eight RPU within the Efficiency and value management
5. You’ll be able to configure community and safety primarily based in your digital personal cloud (VPC) and subnet you wish to create the workgroup.
6. Within the Namespace part, create a brand new namespace in your workgroup. (For instance, redshift-tip-enabled-namespace).
7. Within the Database title and password part, choose Customise admin person credentials and set the admin person title and create a password. Observe them down to make use of in a later step to configure RBAC in Amazon Redshift.
8. Within the Id Middle connections part, select Allow for the cluster choice and choose the Amazon Redshift IAM Id Middle utility created within the earlier step (redshift-data-api-okta-app).
9. Affiliate an IAM position with the workgroup that has the next insurance policies hooked up. Make it the default position to make use of.
   1. AmazonS3ReadOnlyAccess
   2. AmazonRedshiftDataFullAccess
   3. AmazonRedshiftQueryEditorV2ReadSharing
10. Go away different settings as default and select Subsequent.
11. Overview the settings and create the workgroup.

Wait till the workgroup is on the market earlier than persevering with to the following steps.

Create the tables and configure RBAC throughout the Redshift Serverless workgroup

Subsequent, you utilize the Amazon Redshift Question Editor V2 on the Amazon Redshift console to hook up with the workgroup you simply created. You create the tables and configure the Amazon Redshift roles akin to Okta teams for the teams in IAM Id Middle and use the RBAC coverage to grant customers privileges to view knowledge just for their areas. Full the next steps:

1. On the Amazon Redshift console, open the Question Editor V2.
2. Select the choices menu (three dots) subsequent to the Redshift workgroup title and select Edit connection.
   
3. Choose Different methods to attach and use the database person title and password to attach.
4. Within the question editor, run the next code to create the gross sales desk and cargo the info from Amazon Easy Storage Service (Amazon S3):

   # Create the desk
   CREATE TABLE IF NOT EXISTS public.sales_data (
       SKU VARCHAR(50),
       Product_Name VARCHAR(255),
       Class VARCHAR(100),
       Amount INT,
       Sales_Price DECIMAL(10,2),
       Timestamp TIMESTAMP,
       Metropolis VARCHAR(100),
       Region_Code VARCHAR(10),
       Nation VARCHAR(10),
       Latitude DECIMAL(10,6),
       Longitude DECIMAL(10,6),
       Inhabitants INT,
       Elevation INT,
       Timezone VARCHAR(50)
   );
   
   # Load knowledge from S3 to desk
   COPY public.sales_data
   FROM 's3://redshift-blogs/redshift-data-api-idc/sales_data.csv'
   IAM_ROLE default
   CSV
   IGNOREHEADER 1
   DELIMITER ','
   TIMEFORMAT 'auto';
   
   # Create Redshift roles for the teams in IDC, the position format is Namespace:IDCGroupName
   CREATE ROLE "AWSIDC:amer-sales";
   CREATE ROLE "AWSIDC:emea-sales";
   CREATE ROLE "AWSIDC:apac-sales";
   CREATE ROLE "AWSIDC:exec-global";
   
   --Create RLS coverage
   CREATE RLS POLICY eu_region_filter
   WITH (timezone VARCHAR(50))
   USING (timezone LIKE 'Europe%');
   
   CREATE RLS POLICY apac_region_filter
   WITH (timezone VARCHAR(50))
   USING (timezone LIKE 'Asia%');
   
   CREATE RLS POLICY amer_region_filter
   WITH (timezone VARCHAR(50))
   USING (timezone LIKE 'America%');
   
   --Connect coverage
   ATTACH RLS POLICY eu_region_filter ON sales_data TO ROLE "AWSIDC:emea-sales";
   ATTACH RLS POLICY apac_region_filter ON sales_data TO ROLE "AWSIDC:apac-sales";
   ATTACH RLS POLICY amer_region_filter ON sales_data TO ROLE "AWSIDC:amer-sales";
   
   --Activate RLS on desk
   ALTER TABLE public.sales_data ROW LEVEL SECURITY ON;
   GRANT IGNORE RLS TO ROLE "AWSIDC:exec-global";

IAM Id Middle will map the teams into the Redshift roles within the format of Namespace:IDCGroupName. Subsequently, create the position title as AWSIDC:emea-sales and so forth to match them with Okta group names synced in IAM Id Middle. The customers can be created robotically throughout the teams as they log in utilizing SSO into Amazon Redshift.

Obtain, configure, and run the Streamlit utility

On this part, we stroll by the steps to obtain, configure, and run the Streamlit utility.

Create a buyer managed utility in IAM Id Middle for the Redshift Knowledge API shopper

So as to begin a trusted identification propagation workflow and permit Amazon Redshift to make authorization choices primarily based on the customers and teams from IAM Id Middle (provisioned from the exterior IdP), you want an identity-enhanced IAM position session.

This requires a few IAM roles and a buyer managed utility in IAM Id Middle to deal with the belief relationship between the exterior IdP and IAM Id Middle and management entry for the Redshift Knowledge API shopper, on this case, the Streamlit utility.

First, you create two IAM roles, then you definately create a buyer managed utility for the Streamlit utility. Full the next steps:

1. Create a short lived IAM position (we named it IDCBridgeRole) to change the token with IAM Id Middle (assuming you don’t have an current IAM identification to make use of). This position can be assumed by the Streamlit utility with AssumeRoleWithWebIdentity to get a short lived set of position credentials to name the CreateTokenWithIAM and AssumeRole APIs to get the identity-enhanced position session.
   1. Connect the next coverage the position:

      {
          "Model": "2012-10-17",
          "Assertion": [
              {
                  "Effect": "Allow",
                  "Action": "sso-oauth:CreateTokenWithIAM",
                  "Resource": "*"
              },
              {
                  "Effect": "Allow",
                  "Action": "sts:SetContext",
                  "Resource": "*"
              },
              {
                  "Effect": "Allow",
                  "Action": "sts:AssumeRole",
                  "Resource": "*"
              }
          ]
      }

   2. Within the belief relationship, present your AWS account ID and IdP’s URL. The trusted principal to make use of is the Amazon Useful resource Title (ARN) of oidc-provider you created earlier.

      {
          "Model": "2012-10-17",
          "Assertion": [
              {
                  "Effect": "Allow",
                  "Principal": {
                      "Federated": "arn:aws:iam:::oidc-provider/"
                  },
                  "Action": "sts:AssumeRoleWithWebIdentity"
              }
          ]
      }

2. Create an IAM position with permissions to entry the Redshift Knowledge API (we named it RedshiftDataAPIClientRole). This position can be assumed by the Streamlit utility with the improved identities from IAM Id Middle after which used to authenticate requests to the Redshift Knowledge API.
   1. Connect the AmazonRedshiftDataFullAccess AWS managed coverage. AWS recommends utilizing the precept of least privilege in your IAM coverage.
   2. Prohibit the belief relationship to the IDCBridgeRole ARN created within the earlier step), and supply your AWS account ID:

      {
          "Model": "2012-10-17",
          "Assertion": [
              {
                  "Sid": "Statement1",
                  "Effect": "Allow",
                  "Principal": {
                      "AWS": "arn:aws:iam:::role/IDCBridgeRole"
                  },
                  "Action": [
                      "sts:AssumeRole",
                      "sts:SetContext"
                  ]
              }
          ]
      }

Now you may create the shopper managed utility.

3. On the IAM Id Middle console, select Purposes within the navigation pane.
4. Select Add utility.
5. Select I’ve an utility I wish to setup, choose the OAuth 2.0 utility sort, and select Subsequent.
6. Enter a reputation for the appliance, for instance, RedshiftStreamlitDemo.
7. In Consumer and group task technique, select Don’t require task. This implies all of the customers provisioned in IAM Id Middle from Okta can use their Okta credentials to sign up to the Streamlit utility. You’ll be able to alternatively choose the Require assignments choice and decide the customers and teams you wish to enable entry to the appliance.
8. Within the AWS entry portal part, select Not seen, then select Subsequent.
9. Within the Authentication with trusted token issuer part, choose Create trusted token issuer, then enter the Okta issuer URL and enter a reputation for the trusted token issuer.
10. Within the map attribute, use the default e-mail to e-mail mapping between the exterior IdP attribute and IAM Id Middle attribute, then create the trusted token issuer.
11. Choose the trusted token issuer you simply created.
12. Within the Aud declare part, use the shopper ID of the Okta utility you famous earlier, then select Subsequent.
13. Within the Specify utility credentials part, select Edit the appliance coverage and use the next coverage:

    {
      "Model": "2012-10-17",
      "Assertion": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "redshift-data.amazonaws.com"
          },
          "Action": "sso-oauth:*",
          "Resource": "*"
        }
      ]
    }

14. Select Submit.
    

After the appliance is created, you may view it in on the IAM Id Middle.

15. Select Purposes within the navigation pane, and find the Buyer managed purposes tab.
    
    

16. Select the appliance to navigate to the appliance particulars web page.
17. Within the Trusted purposes for identification propagation part, select Specify trusted purposes and choose the setup sort as Particular person purposes and specify entry, then select Subsequent.
18. Select Amazon Redshift because the service, then select Subsequent.
19. Within the Utility that may obtain requests part, select the Amazon Redshift IAM Id Middle utility you created, then select Subsequent.
20. Within the Entry Scopes to use part, test the redshift:join
21. Overview after which select Belief utility.
    

Configure and run the Streamlit utility

Now that you’ve got the roles and the shopper managed utility in IAM Id Middle, you may create an identity-enhanced IAM position session, which is probably the most vital step to allow trusted identification propagation. Following steps present an summary of Streamlit utility code to create the identity-enhanced IAM position session.

1. Authenticate with and retrieve the id_token from the exterior IdP (Okta).
2. Name CreateTokenWithIAM utilizing the exterior IdP issued id_token to acquire an IAM Id Middle issued id_token.
3. Use AssumeRoleWithWebIdentity to acquire momentary IAM credentials (by assuming IDCBridgeRole, defined later).
4. Extract the sts:identity_context from the IAM Id Middle issued id_token.
5. Assume the position RedshiftDataAPIClientRole with the AssumeRole API and insert the sts:identity_context to acquire the identity-enhanced IAM position session credentials.

Now you should use these credentials to make requests to the Redshift Knowledge API, and Amazon Redshift will be capable to use the identification context for authorization choices.

At this level, you must have all of the required sources for creating the Streamlit utility. Full the next steps to check the Streamlit utility:

1. Obtain the Streamlit utility code and modify the configuration part of the code primarily based on the sources provisioned earlier:

# TIP Token change configuration
AWS_REGION = "" # us-east-1
TOKEN_EXCHANGE_APP_ARN = "" # The ARN of the IDC customer-managed-App created earlier
TOKEN_GRANT_TYPE = "urn:ietf:params:oauth:grant-type:jwt-bearer" # mounted worth, please do not change
TEMP_ROLE_ARN = "" # The position created on this step for customers to imagine with AssumeRoleWithWebIdentity(IDCBridgeRole)
ENHANCED_ROLE_ARN = "" # The position created on this step for customers to imagine for the Id-enhanced position session with IAM Id Middle(RedshiftDataAPIClientRole)
IDENHANCED_ROLE_SESSION_NAME = "rs-idc-tip-session" # Use any title for the session 
ROLE_DURATION_SECS = 3600  # 1 hour

# Okta OAuth configuration, substitute with your individual Okta Area
OKTA_DOMAIN = ""
AUTHORIZE_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/authorize"
TOKEN_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/token"
REFRESH_TOKEN_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/token"
REVOKE_TOKEN_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/revoke"
LOGOUT_URL = f"https://{OKTA_DOMAIN}/oauth2/v1/logout"
CLIENT_ID = "" # The shopper id of the Okta app created for the Streamlit app in 2.
CLIENT_SECRET = "" # The shopper id of the Okta app created for the Streamlit app in 2.
REDIRECT_URI = "http://localhost:8501" # That is for dev/take a look at function solely
SCOPE = "openid profile e-mail" # Please don't change
WORKGROUP_NAME = "" #The title of the created Redshift Workgroup
DATABASE = "dev" # The database set for the Workgroup

We advocate internet hosting this utility on an Amazon Elastic Compute Cloud (Amazon EC2) occasion for manufacturing use circumstances, and utilizing AWS Secrets and techniques Supervisor for delicate info just like the CLIENT_ID and CLIENT_SECRET offered as configuration parameters within the code for simplicity.

For this instance, we use the Okta group URL (/oauth2/v1/). You should use the shopper authorization servers as properly, for instance, the default authorization server, however be certain all URLs are utilizing the identical authorization server. Check with Authorization servers for extra details about authorization servers in Okta.

After you modify the script for the Streamlit utility, you may run it utilizing a Python digital atmosphere.

2. Create a Python digital atmosphere. The applying has been examined efficiently with variations v3.12.8 and v3.12.2.

You should set up the next packages, that are required libraries for the Streamlit utility code you downloaded in your digital atmosphere:

• streamlit
• streamlit_oauth
• boto3
• pyjwt
• pydeck
• pandas

3. You’ll be able to set up these libraries instantly utilizing the next command with the necessities file:

   pip set up -r https://redshift-blogs.s3.us-east-1.amazonaws.com/redshift-data-api-idc/necessities.txt

4. Check the Streamlit utility within the Python digital atmosphere with the next command:

   streamlit run /path/to/st_app.py

5. Log in with the person [email protected] from the apac-sales group.
   

The identity-enhanced position session credentials will show on the highest of the web page after profitable authentication with Okta.

For the APAC area supervisor, you must solely see the info from the international locations within the Asia-Pacific area primarily based on the row-level safety filter you configured earlier.

6. Sign off and log again in with the worldwide government person, [email protected] from the exec-global

It’s best to see the info in all areas.

You’ll be able to strive different regional customers’ logins and you must see solely the info within the area they belong to.

Trusted identification propagation deep dive

On this part, you stroll by the Python code of the Streamlit utility and clarify how trusted identification propagation works. The next is an evidence of key elements of the appliance code.

important()

The important() operate of the Streamlit utility implements the previous steps to get the identity-enhanced IAM position session utilizing the get_id_enhanded_session() operate, which wraps the login to get the identity-enhanced position session credentials:

def important():
    # Create OAuth2Component occasion
    oauth2 = OAuth2Component(
        CLIENT_ID, 
        CLIENT_SECRET, 
        AUTHORIZE_URL, 
        TOKEN_URL, 
        REFRESH_TOKEN_URL, 
        REVOKE_TOKEN_URL)
    
    # Different setup code omitted
    
    # Deal with OAuth authentication with Okta
    if not st.session_state.is_authenticated or is_token_expired():
        # Present the login button if not authenticated
        st.title("Login to the Demo app")
        end result = oauth2.authorize_button("Login with Okta", REDIRECT_URI, SCOPE)
        if end result and "token" in end result:
            # Save the token in session state and mark the person as authenticated
            st.session_state.token = end result.get("token")
            st.session_state.user_email = get_user_email_from_token(st.session_state.token.get("id_token"))
            st.session_state.aws_creds = get_id_enhanced_session(st.session_state.token.get("id_token"))
            st.session_state.is_authenticated = True
            st.rerun()
    else:
        
        st.json(st.session_state.aws_creds)
        st.title("Complete Gross sales by Metropolis")
    
        if not is_token_expired():
            # Use the improved credentials to create the Redshift shopper
            redshift_client = boto3.shopper("redshift-data", region_name=AWS_REGION,
                                        aws_access_key_id=st.session_state.aws_creds['AccessKeyId'],
                                        aws_secret_access_key=st.session_state.aws_creds['SecretAccessKey'],
                                        aws_session_token=st.session_state.aws_creds['SessionToken'])
        else:
            st.error("Session expired. Please re-authenticate.")
            logout()
            
    # extra code for question execution and knowledge visualizetion omitted

We use the Streamlit st.session_state offered by Streamlit to retailer vital session states, together with the authentication standing in addition to further info like person info and the AWS identity-enhanced position session credentials.

get_id_enhanced_session()

The get_id_enhanced_session() operate code has three steps:

1. We use the id_token (variable title: jwt_token) from Okta in JWT format to name the AssumeRoleWithWebIdentity API to imagine the position IDCBridgeRole. It’s because the person doesn’t have any AWS credentials to work together with the IAM Id Middle API. In case you plan to host this utility in an AWS atmosphere with an IAM position obtainable, for instance, on an EC2 occasion, you should use the position related to Amazon EC2 to make the decision to the IAM Id Middle APIs with out creating IDCBridgeRole, however be certain the EC2 position has the required permissions we specified for IDCBridgeRole.
2. After now we have the credentials of the momentary position, we use them to make a name to the CreateTokenWithIAM API of IAM Id Middle. This API handles the change of tokens by taking within the id_token from Okta and returning an IAM Id Middle issued token, which can be used later to get the identity-enhanced position session. For extra info, seek advice from the CreateTokenWithIAM API reference.
3. Lastly, we extract the sts:identity_context from the IAM Id Middle issued id_token and move it to the AWS Safety Token Service (AWS STS) AssumeRole That is accomplished by together with the sts:identity_context within the ContextAssertion parameter inside ProvidedContexts, together with ProviderArn set to arn:aws:iam::aws:contextProvider/IdentityCenter.

def get_id_enhanced_session(jwt_token):
    """
    Obtains an identity-enhanced session by assuming a short lived IAM position,
    making a token with IAM, and assuming an enhanced position session.
    
    Args:
        jwt_token (str): The JWT id token from the identification supplier.
    
    Returns:
        dict or None: The improved session credentials if profitable, in any other case None.
    """
    logging.data("Beginning identity-enhanced session course of.")

    # Step 1: Assume a short lived IAM position with the offered JWT token
    temp_credentials = assume_role_with_web_identity(jwt_token)
    if not temp_credentials:
        logging.error("Did not assume position with internet identification.")
        return None

    # Step 2: Use the momentary credentials to create a token with IAM
    id_token = create_token_with_iam(jwt_token, temp_credentials)
    if not id_token:
        logging.error("Did not create ID token with IAM.")
        return None

    # Step 3: Use the ID token to imagine an enhanced position session
    enhanced_creds = assume_enhanced_role_session(id_token, temp_credentials)
    if not enhanced_creds:
        logging.error("Did not assume enhanced position session.")
        return None

    logging.data("Efficiently obtained identity-enhanced session credentials.")
    return enhanced_creds

assume_role_with_web_identity()

The assume_role_with_web_identity() operate code is as follows. We initialize the STS shopper, decode the JWT token, after which assume the position with the net identification.

def assume_role_with_web_identity(jwt_token):
    """
    Assumes an IAM position utilizing an online identification token and returns the momentary credentials.

    Args:
        jwt_token (str): The JWT token for authentication, sometimes issued by an exterior identification supplier.

    Returns:
        dict: Non permanent IAM credentials (Entry Key, Secret Key, Session Token) or None if an error happens.
    """
    strive:
        # Initialize the STS shopper
        sts_client = boto3.shopper('sts', region_name=AWS_REGION)
        
        # Decode the JWT token with out verifying signature (for debugging functions)
        decoded_jwt = jwt.decode(jwt_token, choices={"verify_signature": False})
        logging.debug(f"Decoded JWT Token: {decoded_jwt}")

        # Put together the request for AssumeRoleWithWebIdentity
        assume_role_request = {
            'RoleArn': TEMP_ROLE_ARN,
            'RoleSessionName': 'WebIdentitySession',
            'WebIdentityToken': jwt_token,
            'DurationSeconds': DURATION_SECS  # 1 hour
        }

        # Name the AssumeRoleWithWebIdentity API
        assume_role_response = sts_client.assume_role_with_web_identity(**assume_role_request)
        
        # Extract the momentary credentials from the response
        temp_credentials = assume_role_response['Credentials']
        logging.data("Non permanent credentials efficiently obtained.")
        
        # Return the momentary credentials
        return temp_credentials

    besides ClientError as e:
        logging.error(f"Error calling AssumeRoleWithWebIdentity: {e}")
        return None
    besides jwt.ExpiredSignatureError:
        logging.error("JWT token has expired.")
        return None
    besides jwt.DecodeError:
        logging.error("Error decoding JWT token.")
        return None
    besides Exception as e:
        logging.error(f"Sudden error: {e}")
        return None

create_token_with_iam()

The create_token_with_iam() operate code is known as to get the id_token from IAM Id Middle. The jwt_token is the id_token in JWT format issued by Okta; the id_token is the IAM Id Middle issued id_token.

def create_token_with_iam(jwt_token, temp_credentials):
    """
    Creates an IAM token utilizing the offered JWT token and momentary credentials.

    Args:
        jwt_token (str): The JWT token to change for an IAM token.
        temp_credentials (dict): Non permanent AWS credentials for assuming the position.
    
    Returns:
        str or None: The IAM token if profitable, in any other case None.
    """
    logging.data("Beginning token creation course of with IAM.")
    
    # Initialize the SSO OIDC shopper with momentary credentials
    strive:
        sso_oidc_client = boto3.shopper(
            'sso-oidc', 
            region_name=AWS_REGION, 
            aws_access_key_id=temp_credentials['AccessKeyId'],
            aws_secret_access_key=temp_credentials['SecretAccessKey'],
            aws_session_token=temp_credentials['SessionToken']
        )
    besides Exception as e:
        logging.error(f"Error initializing SSO OIDC shopper: {e}")
        return None

    # Put together the request for CreateTokenWithIAM
    token_request = {
        'clientId': TOKEN_EXCHANGE_APP_ARN,
        'grantType': TOKEN_GRANT_TYPE,
        'assertion': jwt_token
    }

    # Name the CreateTokenWithIAM API
    strive:
        token_result = sso_oidc_client.create_token_with_iam(**token_request)
        id_token = token_result['idToken']
        logging.data(f"Efficiently obtained ID Token: {id_token}")
        return id_token
    besides ClientError as e:
        logging.error(f"Error calling CreateTokenWithIAM API: {e}")
        return None
    besides KeyError as e:
        logging.error(f"Lacking anticipated discipline in response: {e}")
        return None

Within the CreateTokenWithIAM name, we move the next parameters:

• clientId – The ARN of the IAM Id Middle utility for the Redshift Knowledge API shopper
• grantType – urn:ietf:params:oauth:grant-type:jwt-bearer
• assertion – The id_token (jwt_token) issued by Okta

The idToken issued by IAM Id Middle is returned.

assume_enhanced_role_session()

The assume_enhanced_role_session() operate makes use of the ID token to imagine an identity-enhanced position session:

def assume_enhanced_role_session(id_token, temp_credentials):
    """
    Assumes an identity-enhanced IAM position session utilizing the offered ID token and momentary credentials.

    Args:
        id_token (str): The ID token containing the identification context.
        temp_credentials (dict): Non permanent AWS credentials for assuming the position.

    Returns:
        dict or None: The credentials for the identity-enhanced IAM position session, or None on failure.
    """
    logging.data("Extracting identification context from ID token.")
    identity_context = extract_identity_context_from_id_token(id_token)

    if not identity_context:
        logging.error("Did not extract identification context from ID token.")
        return None

    strive:
        # Initialize STS shopper with momentary credentials
        sts_client = boto3.shopper(
            'sts',
            region_name=AWS_REGION,
            aws_access_key_id=temp_credentials['AccessKeyId'],
            aws_secret_access_key=temp_credentials['SecretAccessKey'],
            aws_session_token=temp_credentials['SessionToken']
        )

        # Put together AssumeRole request with identification context
        assume_role_request = {
            'RoleArn': ENHANCED_ROLE_ARN,
            'RoleSessionName': IDENHANCED_ROLE_SESSION_NAME,
            'DurationSeconds': ROLE_DURATION_SECS,
            'ProvidedContexts': [{
                'ContextAssertion': identity_context,
                'ProviderArn': "arn:aws:iam::aws:contextProvider/IdentityCenter"
            }]
        }

        # Name the AssumeRole API
        logging.data("Calling STS AssumeRole for identity-enhanced session.")
        assume_role_response = sts_client.assume_role(**assume_role_request)

        enhanced_role_credentials = assume_role_response['Credentials']
        logging.data("Efficiently assumed enhanced position.")
        
        return enhanced_role_credentials

    besides ClientError as e:
        logging.error(f"Error calling AssumeRole: {e}")
        return None

extract_identity_context_from_id_token()

The extract_identity_context_from_id_token() operate extracts the sts:identity_context:

def extract_identity_context_from_id_token(id_token):
    """
    Extracts the identification context from a decoded JWT token.

    Args:
        id_token (str): The JWT token containing identification context.

    Returns:
        dict or None: The extracted identification context if obtainable, in any other case None.
    """
    logging.data("Decoding ID token to extract identification context.")

    strive:
        # Decode the JWT token (with out signature verification)
        decoded_jwt = jwt.decode(id_token, choices={"verify_signature": False})

        logging.debug(f"Decoded JWT Claims: {decoded_jwt}")

        # Extract the identification context from the token
        for key in ('sts:identity_context', 'sts:audit_context'):
            if key in decoded_jwt:
                return decoded_jwt[key]

        logging.warning("No legitimate identification context discovered within the token.")
        return None

    besides Exception as e:
        logging.error(f"Error decoding JWT: {e}")
        return None

Now you’ve got the identity-enhanced position session credentials to name the Amazon Redshift Knowledge API.

execute_statement() and fetch_results()

The execute_statement() and fetch_results() features display the way to run Redshift queries and retrieve question outcomes with trusted identification propagation for visualization:

def execute_statement(sql, redshift_client):
    """
    Executes a SQL assertion on Amazon Redshift utilizing the offered Redshift Knowledge API shopper.

    Args:
        sql (str): The SQL question to execute.
        redshift_client (boto3.shopper): The Redshift Knowledge API shopper.

    Returns:
        str: The execution ID of the assertion.

    Raises:
        ClientError: If an error happens throughout execution.
    """
    strive:
        response = redshift_client.execute_statement(
            WorkgroupName=WORKGROUP_NAME,
            Database=DATABASE,
            Sql=sql 
        )
        return response["Id"]
    
    besides ClientError as e:
        error_code = e.response.get('Error', {}).get('Code', '')
        
        if error_code == 'ExpiredTokenException':
            logging.error("Session expired. Logging out...")
            logout()
        else:
            logging.error(f"Error executing assertion: {e}")
            elevate
            
def fetch_results(statement_id, redshift_client):
    """
    Fetches question outcomes from the Redshift Knowledge API.

    Args:
        statement_id (str): The execution ID of the assertion.
        redshift_client (boto3.shopper): The Redshift Knowledge API shopper.

    Returns:
        listing: An inventory of data from the question end result.
    """
    strive:
        response = redshift_client.get_statement_result(Id=statement_id)
        return response.get("Information", [])
    
    besides ClientError as e:
        logging.error(f"Error fetching question outcomes: {e}")
        elevate

Conclusion

On this put up, we confirmed the way to create a third-party utility backed by analytics insights arriving from Amazon Redshift securely utilizing OIDC. With Redshift Knowledge API assist of IAM Id Middle integration, you may hook up with Amazon Redshift utilizing SSO from the IdP of your alternative. You’ll be able to lengthen this technique to authenticate different AWS providers that assist trusted identification propagation, resembling Amazon Athena and Amazon QuickSight, enabling fine-grained entry management for IAM Id Middle customers and teams throughout your AWS ecosystem. We encourage you to arrange your utility utilizing IAM Id Middle integration and unify your entry management instantly out of your IdP throughout all IAM Id Middle supported AWS providers.

For extra info on AWS providers and purposes that assist trusted identification propagation, seek advice from Trusted identification propagation overview.

Concerning the Authors

Songzhi Liu is a Principal Massive Knowledge Architect with the AWS Id Options staff. On this position, he collaborates carefully with AWS clients and cross-functional groups to design and implement scalable knowledge architectures, specializing in integrating huge knowledge and machine studying options to boost identification consciousness throughout the AWS ecosystem.

Rohit Vashishtha is a Senior Analytics Specialist Options Architect at AWS primarily based in Dallas, Texas. He has over 19 years of expertise architecting, constructing, main, and sustaining huge knowledge platforms. Rohit helps clients modernize their analytic workloads utilizing the breadth of AWS providers and ensures that clients get one of the best worth/efficiency with utmost safety and knowledge governance.

Fei Peng is a Senior Software program Growth Engineer working within the Amazon Redshift staff, the place he leads the event of Redshift Knowledge API, enabling seamless and scalable entry to cloud knowledge warehouses.

Yanzhu Ji is a Product Supervisor within the Amazon Redshift staff. She has expertise in product imaginative and prescient and technique in industry-leading knowledge merchandise and platforms. She has excellent talent in constructing substantial software program merchandise utilizing internet improvement, system design, database, and distributed programming strategies. In her private life, Yanzhu likes portray, images, and enjoying tennis.