632ac9e328
- Python CLI tool for rolling updates of CL-AppPipe-* and CL-SvcPipe-* stacks - Async update engine with configurable concurrency (asyncio.Semaphore) - Exponential backoff retry for API throttling - Dry-run mode for safe preview - IAM permission pre-validation - Comprehensive test suite (80 tests: 11 property-based + 69 unit) - Full spec documentation (requirements, design, tasks)
92 lines
7 KiB
Markdown
92 lines
7 KiB
Markdown
# Requirements Document
|
|
|
|
## Introduction
|
|
|
|
This feature provides a one-click mechanism to update 22+ CloudFormation stacks in an audit account. All stacks follow the `CL-AppPipe-*` naming convention (e.g., `CL-AppPipe-9894aa72`, `CL-AppPipe-ca6dca90`) and use the same nested template URL (`https://s3.amazonaws.com/solutions-reference/centralized-logging-with-opensearch/latest/AppLogS3Buffer.template`) from the Centralized Logging with OpenSearch solution (SO8025-s3b). The number of stacks grows over time as new pipelines are added. When a new version of that template becomes available, the operator needs a single action to trigger a rolling update across every stack without manually updating each one.
|
|
|
|
## Glossary
|
|
|
|
- **Stack_Updater**: The system that orchestrates the discovery and update of all target CloudFormation stacks in the audit account.
|
|
- **Target_Stack**: A CloudFormation stack whose name starts with the `CL-AppPipe-` prefix, belonging to the Centralized Logging with OpenSearch solution for log ingestion pipelines.
|
|
- **Stack_Name_Prefix**: The string `CL-AppPipe-` used to identify Target_Stacks by their CloudFormation stack name.
|
|
- **Nested_Template_URL**: The S3 URL (`https://s3.amazonaws.com/solutions-reference/centralized-logging-with-opensearch/latest/AppLogS3Buffer.template`) that always resolves to the latest version of the pipeline template.
|
|
- **Solution_ID**: The identifier `SO8025-s3b` for the Centralized Logging with OpenSearch AWS Solution that provisions the Target_Stacks.
|
|
- **Update_Run**: A single execution of the Stack_Updater that attempts to update all Target_Stacks.
|
|
- **Operator**: The person who triggers and monitors the update process.
|
|
- **Concurrency_Limit**: The maximum number of CloudFormation stack updates that the Stack_Updater executes in parallel during an Update_Run.
|
|
|
|
## Requirements
|
|
|
|
### Requirement 1: Discover Target Stacks
|
|
|
|
**User Story:** As an Operator, I want the system to automatically discover all stacks matching the pipeline naming convention, so that I do not have to maintain a manual list and newly added stacks are picked up automatically.
|
|
|
|
#### Acceptance Criteria
|
|
|
|
1. WHEN an Update_Run is triggered, THE Stack_Updater SHALL discover Target_Stacks by listing all CloudFormation stacks whose name starts with the Stack_Name_Prefix `CL-AppPipe-`.
|
|
2. THE Stack_Updater SHALL perform discovery dynamically on each Update_Run so that stacks added after the previous run are included automatically.
|
|
3. WHEN discovery completes, THE Stack_Updater SHALL report the total number of Target_Stacks found before proceeding with updates.
|
|
4. IF no Target_Stacks are found, THEN THE Stack_Updater SHALL log a warning message and terminate the Update_Run without error.
|
|
|
|
### Requirement 2: One-Click Trigger
|
|
|
|
**User Story:** As an Operator, I want to trigger the update of all Target_Stacks with a single action, so that I do not have to update each stack individually.
|
|
|
|
#### Acceptance Criteria
|
|
|
|
1. THE Stack_Updater SHALL expose a single invocation endpoint that starts an Update_Run.
|
|
2. WHEN the Operator invokes the endpoint, THE Stack_Updater SHALL begin an Update_Run that updates all discovered Target_Stacks.
|
|
3. THE Stack_Updater SHALL accept an optional Concurrency_Limit parameter that controls how many stacks are updated in parallel.
|
|
4. IF no Concurrency_Limit is provided, THEN THE Stack_Updater SHALL default to updating 5 stacks in parallel.
|
|
|
|
### Requirement 3: Stack Update Execution
|
|
|
|
**User Story:** As an Operator, I want each stack to be updated using its existing parameters, so that the only change is the refreshed nested template.
|
|
|
|
#### Acceptance Criteria
|
|
|
|
1. WHEN updating a Target_Stack, THE Stack_Updater SHALL call the CloudFormation UpdateStack API using the existing parameter values of the Target_Stack.
|
|
2. WHEN updating a Target_Stack, THE Stack_Updater SHALL use the current Nested_Template_URL so that CloudFormation resolves the latest template version.
|
|
3. WHILE an Update_Run is in progress, THE Stack_Updater SHALL respect the Concurrency_Limit and wait for an in-progress update to complete before starting the next one.
|
|
4. IF CloudFormation returns a "No updates are to be performed" response for a Target_Stack, THEN THE Stack_Updater SHALL treat that Target_Stack as successfully updated and continue.
|
|
|
|
### Requirement 4: Error Handling and Resilience
|
|
|
|
**User Story:** As an Operator, I want the update process to continue even if individual stacks fail, so that one failure does not block the remaining stacks.
|
|
|
|
#### Acceptance Criteria
|
|
|
|
1. IF a Target_Stack update fails, THEN THE Stack_Updater SHALL log the stack name and error details and continue updating the remaining Target_Stacks.
|
|
2. IF a Target_Stack is in a non-updatable state (e.g., ROLLBACK_COMPLETE, UPDATE_ROLLBACK_IN_PROGRESS), THEN THE Stack_Updater SHALL skip that Target_Stack and log a warning.
|
|
3. IF the CloudFormation API returns a throttling error, THEN THE Stack_Updater SHALL retry the request using exponential backoff with a maximum of 3 retries.
|
|
4. IF all retries for a Target_Stack are exhausted, THEN THE Stack_Updater SHALL mark that Target_Stack as failed and continue with the remaining stacks.
|
|
|
|
### Requirement 5: Progress Reporting and Summary
|
|
|
|
**User Story:** As an Operator, I want to see the progress and final results of an Update_Run, so that I know which stacks succeeded and which failed.
|
|
|
|
#### Acceptance Criteria
|
|
|
|
1. WHILE an Update_Run is in progress, THE Stack_Updater SHALL log the status of each Target_Stack update as it completes (succeeded, failed, skipped, no-update-needed).
|
|
2. WHEN an Update_Run completes, THE Stack_Updater SHALL produce a summary report containing: total stacks found, stacks updated successfully, stacks skipped, stacks failed, and stacks with no update needed.
|
|
3. WHEN an Update_Run completes, THE Stack_Updater SHALL return a non-zero exit code if any Target_Stack update failed.
|
|
4. THE Stack_Updater SHALL log the start time and end time of each Update_Run.
|
|
|
|
### Requirement 6: Dry Run Mode
|
|
|
|
**User Story:** As an Operator, I want to preview which stacks would be updated without actually performing updates, so that I can verify the scope before committing.
|
|
|
|
#### Acceptance Criteria
|
|
|
|
1. THE Stack_Updater SHALL accept a dry-run flag as an input parameter.
|
|
2. WHEN the dry-run flag is set, THE Stack_Updater SHALL discover and list all Target_Stacks without performing any updates.
|
|
3. WHEN the dry-run flag is set, THE Stack_Updater SHALL output the name and current status of each Target_Stack that would be updated.
|
|
|
|
### Requirement 7: IAM Permission Validation
|
|
|
|
**User Story:** As an Operator, I want the system to verify it has the required permissions before starting updates, so that I catch permission issues early.
|
|
|
|
#### Acceptance Criteria
|
|
|
|
1. WHEN an Update_Run is triggered, THE Stack_Updater SHALL verify that the executing role has `cloudformation:DescribeStacks`, `cloudformation:UpdateStack`, and `cloudformation:ListStacks` permissions before proceeding.
|
|
2. IF the executing role lacks required permissions, THEN THE Stack_Updater SHALL report the missing permissions and terminate the Update_Run without attempting any updates.
|