Multi-account codepipeline
Contents
Intro
In this note I will put some light on the basics of multi-account CI/CD pipelines setup in AWS. The constraints are the following:
- The dev teams should work independently and have the full control of their own code
- The enterprise has the defined guardrails and policies for IAM management
- The neterprise has the library of ready to use modules for infra deployment In this note I will only touch the aspecs of passing the artifacts and pipeline setting-up. CD tools and practices will probably come in future.
Design
The source repo for app resides in SRC Account, CI tools are deployed in CI Account, deployment targets are DEV and PROD accounts. The CodePipeline will have Source step that takes the code from SRC Account, the CI stage that will run build / test procedures and publish stage that will deploy hte artifacts to CodeArtifact repo.
sequenceDiagram
participant CI Account
participant Source Account
participant DEV Account
participant PROD Account
CI Account ->> Source Account: Get App SRC
Note over CI Account: Get DevOps SRC
Note over CI Account: Build BOM
Note over CI Account: Run syntetic tests
Note over CI Account: Publish artifacts
CI Account ->> DEV Account: Deploy DEV
CI Account ->> DEV Account: Run Functional Tests
Note over CI Account: Promote Artifacts
Note over Source Account: Merge Pull Request
CI Account ->> PROD Account: Deploy PROD
CI Account ->> Source Account: Tag Commit
Multi account problem statement
AWS CodePipeline can doesn't have easy way to build multi-account steps. This is possible, but requires some not evident preparation steps. In general, you will need to:
- Create S3 bucket that will be used for codepipeline artifacts storage
- Assign proper cross-account policy to the bucket, to allow other accounts use the bucket
- Share the KMS key to encrypt / decrypt operations during pipeline run and artifacts storing
- Create the set of IAM roles with proper policies
General CodePipeline Flow
Codepipepline has defined permissions flow, where pipeleline and actions can assume IAM Roles.
flowchart LR
A("`**Pipeline**`") ==> B("`**Source** Action`")
A ==Use Pipeline Role===> C("`Build Action`")
B ==> D("`Assumes Role
in **SRC** Acount
`")
D ==> F("`**SRC**`")
F==> G[["`SRC to S3 Artifact`"]]
C ==Get Artifacts===> G
C ==Call Build Executor===>H[["`Make build`"]]
Besides of action role and pipeline role, you would probably need to define service roles for action executors (e.g.: CodeBuild). All the roles should at least have the acccess to KMS key and pipeline artifacts bucket.
CI Account
The CI account will be the heart of the whole project. The majority of components should be deployed here. From the very beginning we will need to deploy the Pipeline IAM Role. I used the one with basic codepipeline policy. In addition, we need to add permissions to AssumeRole in Source Account. We will have several Source accounts, so you will need to put wiledcard in your policy (see the example in CF template below).
The KMS key should be used by all of the accounts in setup to share the artifacts.
The full CloudFormation template to build basic components in CI account:
AWSTemplateFormatVersion: "2010-09-09"
Parameters:
CPipelineArtifacts:
Type: 'AWS::S3::Bucket'
DeletionPolicy: Retain
Properties:
BucketName: org-name-codepipeline-artifacts
PublicAccessBlockConfiguration:
BlockPublicAcls: True
BlockPublicPolicy: False
IgnorePublicAcls: True
RestrictPublicBuckets: True
CPipelineArtifactsPolicy:
Type: 'AWS::S3::BucketPolicy'
Properties:
Bucket: CPipelineArtifacts
PolicyDocument:
Version:
Statement:
- Sid: DenyUnEncryptedObjectUploads
Effect: Deny
Principal: '*'
Action: 's3:PutObject'
Resource:
- ''
- -
- CPipelineArtifacts
- Arn
- /*
Condition:
StringNotEquals:
's3:x-amz-server-side-encryption': 'aws:kms'
- Sid: DenyInsecureConnections
Effect: Deny
Principal: '*'
Action: 's3:*'
Resource:
- ''
- -
- CPipelineArtifacts
- Arn
- /*
Condition:
Bool:
'aws:SecureTransport': false
CPipelineKMSKey:
Type: 'AWS::KMS::Key'
Properties:
Description: KMS key for Codepipeline
EnableKeyRotation: true
PendingWindowInDays: 14
KeyPolicy:
Version:
Id: cip-codepipeline-key
Statement:
- Sid: Allow administration of the key
Effect: Allow
Principal:
AWS: 'arn:aws:iam::${AWS::AccountId}:root'
Action:
- 'kms:Create*'
- 'kms:Describe*'
- 'kms:Enable*'
- 'kms:List*'
- 'kms:Put*'
- 'kms:Update*'
- 'kms:Revoke*'
- 'kms:Disable*'
- 'kms:Get*'
- 'kms:Delete*'
- 'kms:ScheduleKeyDeletion'
- 'kms:CancelKeyDeletion'
Resource: '*'
Condition:
ArnLike:
"aws:PrincipalArn": "arn:aws:iam::${AWS::AccountId}:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_AdministratorAccess_*"
- Sid: Lambda access to KMS Key
Effect: Allow
Principal:
AWS: ListenerLambdaRole.Arn
Action:
- kms:PutKeyPolicy
- kms:GetKeyPolicy
Resource: '*'
CodePipelineServiceRole:
Type: 'AWS::IAM::Role'
Properties:
RoleName: org-name-CodePipeline-Role
AssumeRolePolicyDocument:
Version:
Statement:
- Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action: 'sts:AssumeRole'
Path: /
Policies:
- PolicyName: AssumeRole
PolicyDocument:
Version:
Statement:
- Effect: Allow
Action:
- 'sts:AssumeRole'
Resource: 'arn:aws:iam::*:role/org-name-ChildRole'
- PolicyName: S3Access
PolicyDocument:
Version:
Statement:
- Effect: Allow
Action:
- 's3:*'
Resource:
- ''
- -
- CPipelineArtifacts
- Arn
- /*
- PolicyName: org-name-CodePipeline-Policy
PolicyDocument:
Version:
Statement:
- Effect: Allow
Action:
- 'codecommit:CancelUploadArchive'
- 'codecommit:GetBranch'
- 'codecommit:GetCommit'
- 'codecommit:GetUploadArchiveStatus'
- 'codecommit:UploadArchive'
Resource: '*'
- Effect: Allow
Action:
- 'codedeploy:CreateDeployment'
- 'codedeploy:GetApplicationRevision'
- 'codedeploy:GetDeployment'
- 'codedeploy:GetDeploymentConfig'
- 'codedeploy:RegisterApplicationRevision'
Resource: '*'
- Effect: Allow
Action:
- 'codebuild:BatchGetBuilds'
- 'codebuild:StartBuild'
Resource: '*'
- Effect: Allow
Action:
- 'devicefarm:ListProjects'
- 'devicefarm:ListDevicePools'
- 'devicefarm:GetRun'
- 'devicefarm:GetUpload'
- 'devicefarm:CreateUpload'
- 'devicefarm:ScheduleRun'
Resource: '*'
- Effect: Allow
Action:
- 'lambda:InvokeFunction'
- 'lambda:ListFunctions'
Resource: '*'
- Effect: Allow
Action:
- 'iam:PassRole'
Resource: '*'
- Effect: Allow
Action:
- 'elasticbeanstalk:*'
- 'ec2:*'
- 'elasticloadbalancing:*'
- 'autoscaling:*'
- 'cloudwatch:*'
- 's3:*'
- 'sns:*'
- 'cloudformation:*'
- 'rds:*'
- 'sqs:*'
- 'ecs:*'
Resource: '*'
- Effect: Allow
Action:
- "kms:Encrypt"
- "kms:Decrypt"
- "kms:ReEncrypt*"
- "kms:GenerateDataKey*"
- "kms:DescribeKey"
Resource: CPipelineKMSKey.Arn
SRC Account
In SRC account we should have the actual code repository, and IAM role, that is allowed to make repo actions and have access to S3 bucket with artifacts.
AWSTemplateFormatVersion: "2010-09-09"
Description: Base tmeplate for CI SRC account creation.
Parameters:
paramKMSKey:
Type: String
Default: ${KMS_KEY_ARN}
pRepoList:
Type: CommaDelimitedList
Resources:
CIPSrcRole:
Type: AWS::IAM::Role
Properties:
RoleName: "ChildRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
AWS: arn:aws:iam::${CI_ACCOUNT_ID}:role/CodePipeline-Role
Action:
- sts:AssumeRole
Policies:
- PolicyName: "KMSKey-Usage"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "kms:Encrypt"
- "kms:Decrypt"
- "kms:ReEncrypt*"
- "kms:GenerateDataKey*"
- "kms:DescribeKey"
Resource: paramKMSKey
- PolicyName: "CodeCommit-Access"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "codecommit:Get*"
- "codecommit:List*"
- "codecommit:Describe*"
- "codecommit:UploadArchive"
Resource: pRepoList
- PolicyName: "Artifact-S3-Access"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "s3:*"
Resource:
- arn:aws:s3:::codepipeline-artifacts
- arn:aws:s3:::codepipeline-artifacts/*
Pay attention, that in pRepoList parameter the ARNs of CodeCommit repos defined. In addition to this, make sure that you have resource policies on your artifact bucket and KMS key, that allows the access from SRC account Role. I have written the custom resource for this task, that I will share later.
DEV and PROD accounts
Actually theese are the target accounts, that will have the same roles. The setup is the same as for SRC account, the only difference here – is in permissions for ChildRole.
Pipeline Setup
Once you have all IAM roles, reource policies and everything else ready – let's configure the pipeline. There is no way to configure it through UI, so you should use API. In my case, I used CloudFormation.
Below is the part of CloudFormation template that build Pipeline with steps and Roles.
AppPipeline:
Type: 'AWS::CodePipeline::Pipeline'
Properties:
Name: 'AWS::StackName'
PipelineType: V2
RoleArn: '{{resolve:ssm:/pipeline-params/pipeline-role-arn}}'
ArtifactStore:
Type: S3
EncryptionKey:
Id: '{{resolve:ssm:/pipeline-params/kms-key-arn}}'
Type: "KMS"
Location: '{{resolve:ssm:/pipeline-params/s3-artifact-store}}'
Stages:
- Name: Source
Actions:
- Name: AppSRC
RoleArn: arn:aws:iam::${pSRCAcc}:role/ChildRole
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
OutputArtifacts:
- Name: SourceOutput
Configuration:
BranchName: pSourcebranch
RepositoryName: pSourceRepo
PollForSourceChanges: true
RunOrder: 1
- Name: DevOpsSRC
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
OutputArtifacts:
- Name: DevOpsCode
Configuration:
BranchName: main
RepositoryName: '{{resolve:ssm:/pipeline-params/devops-repo-name}}'
PollForSourceChanges: false
RunOrder: 2
- Name: CI
Actions:
- Name: CI-Process
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
InputArtifacts:
- Name: SourceOutput
- Name: DevOpsCode
Configuration:
ProjectName: CICodeBuildProject
PrimarySource: SourceOutput
RunOrder: 1
As you can see, we have 2 source actions here:
- AppSRC
- DevOpsSRC
The AppSRC gets the code from the external account, using Role in child account.
Wrapping Up
To make the things simple, you will need to build the following infra:
- Pipeline Role in your CI account with at least theese permissions:
- Assume Role in each of child accounts (SRC and Destination)
- Access to KMS Key and S3 bucket
- Role for each of the Actions, that are running in extrenal accounts. The following permissions required:
- Assume Role Policy allowing this role to be assumed by Pipeline Role
- Access to KMS Key and S3 bucket in CI Account
- Resource policies for Arifact S3 bucket and KMS Key allowing access for each of the roles
- Update the pipeline to use IAM roles in child accounts