CDK Entest

Introduction#

GitHub this note shows managing data access via IAM. Before Lake Formation released in 2019, we need to configure data access via IAM Policy. For example, if there is Data Scientist (IAM user), to enable the DS to query tables in Glue Catalog, we need to configure

Grant access to the underlying data in S3 via IAM policy
Grant access to tables in Glue Catalog via IAM policy
Create an IAM user for a Data Scientist
Create an IAM role for Glue interactive session
Control access to Athena Workgroup via IAM

IAM User#

const secret = new aws_secretsmanager.Secret(this, `${props.userName}Secret`, {
  secretName: `${props.userName}Secret`,
  generateSecretString: {
    secretStringTemplate: JSON.stringify({ userName: props.userName }),
    generateStringKey: "password",
  },
});

const user = new aws_iam.User(this, `${props.userName}IAMUSER`, {
  userName: props.userName,
  password: secret.secretValueFromJson("password"),
  passwordResetRequired: false,
});

Grant Permissions#

Option 1. Grant the DS to access all data

user.addManagedPolicy(
  aws_iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonAthenaFullAccess")
);

Option 2. Least priviledge so the DS only can access requested tables. For Glue, please note that

All operations performed on a Data Catalog resource require permission on the resource and all the ancestors of that resource. For example, to create a partition for a table requires permission on the table, database, and catalog where the table is located. The following example shows the permission required to create partitions on table PrivateTable in database PrivateDatabase in the Data Catalog.

Specify the permission to access tables in Glue catalog

new aws_iam.PolicyStatement({
        actions: [
          "glue:CreateDatabase",
          "glue:DeleteDatabase",
          "glue:GetDatabase",
          "glue:GetDatabases",
          "glue:UpdateDatabase",
          "glue:CreateTable",
          "glue:DeleteTable",
          "glue:BatchDeleteTable",
          "glue:UpdateTable",
          "glue:GetTable",
          "glue:GetTables",
          "glue:BatchCreatePartition",
          "glue:CreatePartition",
          "glue:DeletePartition",
          "glue:BatchDeletePartition",
          "glue:UpdatePartition",
          "glue:GetPartition",
          "glue:GetPartitions",
          "glue:BatchGetPartition",
        ],
        effect: Effect.ALLOW,
        resources: [
          `arn:aws:glue:${this.region}:*:table/${props.databaseName}/*`,
          `arn:aws:glue:${this.region}:*:database/${props.databaseName}*`,
          `arn:aws:glue:${this.region}:*:*catalog`,
        ],
      }),

The full IAM policy attached to the DS IAM user

const policy = new aws_iam.Policy(
  this,
  "LeastPriviledgePolicyForDataScientist",
  {
    policyName: "LeastPriviledgePolicyForDataScientist",
    statements: [
      // athena
      new aws_iam.PolicyStatement({
        actions: ["athena:*"],
        effect: Effect.ALLOW,
        // resources: ["*"],
        resources: [
          `arn:aws:athena:${this.region}:${this.account}:workgroup/${props.athenaWorkgroupName}`,
        ],
      }),
      new aws_iam.PolicyStatement({
        actions: [
          "athena:ListEngineVersions",
          "athena:ListWorkGroups",
          "athena:ListDataCatalogs",
          "athena:ListDatabases",
          "athena:GetDatabase",
          "athena:ListTableMetadata",
          "athena:GetTableMetadata",
        ],
        effect: Effect.ALLOW,
        resources: ["*"],
      }),
      // access s3
      new aws_iam.PolicyStatement({
        actions: [
          "s3:GetBucketLocation",
          "s3:GetObject",
          "s3:ListBucket",
          "s3:ListBucketMultipartUploads",
          "s3:ListMultipartUploadParts",
          "s3:AbortMultipartUpload",
          "s3:CreateBucket",
          "s3:PutObject",
          "s3:PutBucketPublicAccessBlock",
        ],
        effect: Effect.ALLOW,
        resources: [
          props.athenaResultBucketArn,
          `${props.athenaResultBucketArn}/*`,
          props.sourceBucketArn,
          `${props.sourceBucketArn}/*`,
        ],
      }),
      // access glue catalog
      new aws_iam.PolicyStatement({
        actions: [
          "glue:CreateDatabase",
          "glue:DeleteDatabase",
          "glue:GetDatabase",
          "glue:GetDatabases",
          "glue:UpdateDatabase",
          "glue:CreateTable",
          "glue:DeleteTable",
          "glue:BatchDeleteTable",
          "glue:UpdateTable",
          "glue:GetTable",
          "glue:GetTables",
          "glue:BatchCreatePartition",
          "glue:CreatePartition",
          "glue:DeletePartition",
          "glue:BatchDeletePartition",
          "glue:UpdatePartition",
          "glue:GetPartition",
          "glue:GetPartitions",
          "glue:BatchGetPartition",
        ],
        effect: Effect.ALLOW,
        resources: [
          `arn:aws:glue:${this.region}:*:table/${props.databaseName}/*`,
          `arn:aws:glue:${this.region}:*:database/${props.databaseName}*`,
          `arn:aws:glue:${this.region}:*:*catalog`,
        ],
      }),
      // access lakeformation
      // new aws_iam.PolicyStatement({
      //   actions: ["lakeformation:GetDataAccess"],
      //   effect: Effect.ALLOW,
      //   resources: ["*"],
      // }),
    ],
}

Glue Role#

Create an IAM Role for Glue
Create a Glue Notebook or interactive session
Read data from Glue catalog, S3
GlueServiceRoleNotebook name convention and iam:PassRole

const role = new aws_iam.Role(this, `GlueRoleFor-${props.pipelineName}`, {
  roleName: `GlueRoleFor-${props.pipelineName}`,
  assumedBy: new aws_iam.ServicePrincipal("glue.amazonaws.com"),
});

role.addManagedPolicy(
  aws_iam.ManagedPolicy.fromAwsManagedPolicyName(
    "service-role/AWSGlueServiceRole"
  )
);

role.addManagedPolicy(
  aws_iam.ManagedPolicy.fromAwsManagedPolicyName("CloudWatchAgentServerPolicy")
);

Then attach the same policy bove LeastPriviledgePolicyForDataScientist to the Gule role. The notebook need to pass role to the execution session, so there are to options

Explicit sepcify iam:PassRole in the policy below
Follow the role name convetion such as AWSGlueServiceRoleNotebook

For Role name, enter a name for your role; for example, AWSGlueServiceRoleDefault. Create the role with the name prefixed with the string AWSGlueServiceRole to allow the role to be passed from console users to the service. AWS Glue provided policies expect IAM service roles to begin with AWSGlueServiceRole. Otherwise, you must add a policy to allow your users the iam:PassRole permission for IAM roles to match your naming convention. Choose Create Role.

const policy = new aws_iam.Policy(
  this,
  "LeastPriviledgePolicyForGlueNotebookRole",
  {
    policyName: "LeastPriviledgePolicyForGlueNotebookRole",
    statements: [
      // pass iam role
      new aws_iam.PolicyStatement({
        actions: ["iam:PassRole", "iam:GetRole"],
        effect: Effect.ALLOW,
        resources: ["*"],
      }),
      // athena
      new aws_iam.PolicyStatement({
        actions: ["athena:*"],
        effect: Effect.ALLOW,
        resources: ["*"],
      }),
      // access s3
      new aws_iam.PolicyStatement({
        actions: ["s3:*"],
        effect: Effect.ALLOW,
        resources: [
          props.athenaResultBucketArn,
          `${props.athenaResultBucketArn}/*`,
          props.sourceBucketArn,
          `${props.sourceBucketArn}/*`,
        ],
      }),
      // access glue catalog
      new aws_iam.PolicyStatement({
        actions: ["glue:*"],
        effect: Effect.ALLOW,
        resources: [
          `arn:aws:glue:${this.region}:*:table/${props.databaseName}/*`,
          `arn:aws:glue:${this.region}:*:database/${props.databaseName}*`,
          `arn:aws:glue:${this.region}:*:*catalog`,
        ],
      }),
    ],
  }
);

then attach the policy to the Glue role

policy.attachToUser(user);

IAM User#

Grant Permissions#

Glue Role#

Reference#