- Python CLI tool for rolling updates of CL-AppPipe-* and CL-SvcPipe-* stacks - Async update engine with configurable concurrency (asyncio.Semaphore) - Exponential backoff retry for API throttling - Dry-run mode for safe preview - IAM permission pre-validation - Comprehensive test suite (80 tests: 11 property-based + 69 unit) - Full spec documentation (requirements, design, tasks)
183 lines
9.5 KiB
Markdown
183 lines
9.5 KiB
Markdown
# Implementation Plan: One-Click CloudFormation Stack Updater
|
|
|
|
## Overview
|
|
|
|
Implement a Python CLI tool that discovers and updates all `CL-AppPipe-*` CloudFormation stacks in an audit account. The implementation follows the pipeline architecture: Permission Validation → Stack Discovery → Stack Update Engine → Report Generator. Each task builds incrementally, wiring components together at the end.
|
|
|
|
## Tasks
|
|
|
|
- [x] 1. Set up project structure, configuration constants, and data models
|
|
- Create the package directory structure: `cfn_updater/` with `__init__.py`, `cli.py`, `permissions.py`, `discovery.py`, `updater.py`, `report.py`, `config.py`
|
|
- Create `tests/` directory with `__init__.py`, `conftest.py`
|
|
- Implement `config.py` with all configuration constants (`TEMPLATE_URL`, `DEFAULT_PREFIX`, `DEFAULT_CONCURRENCY`, `MAX_RETRIES`, `BASE_RETRY_DELAY`, `NON_UPDATABLE_STATUSES`)
|
|
- Implement data model dataclasses: `DiscoveredStack`, `StackUpdateResult`, `UpdateRunReport`
|
|
- Add `pyproject.toml` or `requirements.txt` with dependencies: `boto3`, `pytest`, `hypothesis`, `botocore`
|
|
- _Requirements: 1.1, 2.3, 2.4, 3.1, 5.2_
|
|
|
|
- [ ] 2. Implement Permission Validator
|
|
- [x] 2.1 Implement `validate_permissions()` in `permissions.py`
|
|
- Check `cloudformation:ListStacks`, `cloudformation:DescribeStacks`, and `cloudformation:UpdateStack` permissions via dry-run API calls
|
|
- Return a list of missing permission names; empty list means all OK
|
|
- _Requirements: 7.1, 7.2_
|
|
|
|
- [x] 2.2 Write property test for permission validation (Property 11)
|
|
- **Property 11: Permission validation correctness**
|
|
- For any subset of required permissions marked as missing, the validator returns exactly those missing permissions
|
|
- When any permissions are missing, no UpdateStack calls are made
|
|
- **Validates: Requirements 7.1, 7.2**
|
|
|
|
- [x] 2.3 Write unit tests for permission validation
|
|
- Test all permissions present (happy path)
|
|
- Test single missing permission
|
|
- Test all permissions missing
|
|
- Test boto3 error handling during validation
|
|
- _Requirements: 7.1, 7.2_
|
|
|
|
- [ ] 3. Implement Stack Discovery
|
|
- [x] 3.1 Implement `discover_stacks()` in `discovery.py`
|
|
- List all CloudFormation stacks using paginated `list_stacks` API calls
|
|
- Filter stacks by `Stack_Name_Prefix` (`CL-AppPipe-`)
|
|
- Mark each stack as updatable or not based on `NON_UPDATABLE_STATUSES`
|
|
- Return list of `DiscoveredStack` objects
|
|
- _Requirements: 1.1, 1.2, 1.3, 4.2_
|
|
|
|
- [x] 3.2 Write property test for discovery prefix filtering (Property 1)
|
|
- **Property 1: Discovery returns exactly prefix-matched stacks with correct count**
|
|
- For any list of stack names, discovery returns exactly those starting with the prefix
|
|
- Reported count equals the length of the filtered list
|
|
- **Validates: Requirements 1.1, 1.3**
|
|
|
|
- [x] 3.3 Write property test for non-updatable stack classification (Property 6)
|
|
- **Property 6: Non-updatable stacks are skipped**
|
|
- For any stack with a status in `NON_UPDATABLE_STATUSES`, `updatable` is `False`
|
|
- For any stack with a status not in `NON_UPDATABLE_STATUSES`, `updatable` is `True`
|
|
- **Validates: Requirements 4.2**
|
|
|
|
- [x] 3.4 Write unit tests for stack discovery
|
|
- Test empty stack list (Req 1.4)
|
|
- Test pagination across multiple pages
|
|
- Test mixed updatable and non-updatable stacks
|
|
- _Requirements: 1.1, 1.2, 1.3, 1.4, 4.2_
|
|
|
|
- [x] 4. Checkpoint - Ensure all tests pass
|
|
- Ensure all tests pass, ask the user if questions arise.
|
|
|
|
- [ ] 5. Implement Stack Update Engine
|
|
- [x] 5.1 Implement `update_stack()` coroutine in `updater.py`
|
|
- Fetch current stack parameters via `describe_stacks`
|
|
- Call `UpdateStack` with `UsePreviousValue=True` for all existing parameters and the configured `TEMPLATE_URL`
|
|
- Handle "No updates are to be performed" response as `no-update-needed`
|
|
- Implement exponential backoff retry for throttling errors (`Throttling`, `RequestLimitExceeded`)
|
|
- Record duration and return `StackUpdateResult`
|
|
- _Requirements: 3.1, 3.2, 3.4, 4.1, 4.3, 4.4_
|
|
|
|
- [x] 5.2 Implement `update_all_stacks()` coroutine in `updater.py`
|
|
- Use `asyncio.Semaphore` to bound concurrent updates to `Concurrency_Limit`
|
|
- Skip non-updatable stacks (record as `skipped`)
|
|
- Collect results for all stacks, including failures
|
|
- _Requirements: 2.2, 2.3, 3.3, 4.1, 4.2_
|
|
|
|
- [x] 5.3 Write property test for parameter preservation (Property 4)
|
|
- **Property 4: Update call preserves existing parameters and uses correct template URL**
|
|
- For any stack with any set of parameters, UpdateStack includes all parameter keys with `UsePreviousValue=True`
|
|
- `TemplateURL` equals the configured `TEMPLATE_URL`
|
|
- **Validates: Requirements 3.1, 3.2**
|
|
|
|
- [x] 5.4 Write property test for "no updates" handling (Property 5)
|
|
- **Property 5: "No updates" response maps to no-update-needed status**
|
|
- For any stack returning "No updates are to be performed", result status is `no-update-needed`
|
|
- **Validates: Requirements 3.4**
|
|
|
|
- [x] 5.5 Write property test for all updatable stacks attempted (Property 2)
|
|
- **Property 2: All updatable stacks are attempted**
|
|
- For any set of discovered stacks, exactly one result per updatable stack; no drops, no duplicates
|
|
- **Validates: Requirements 2.2**
|
|
|
|
- [x] 5.6 Write property test for concurrency limit invariant (Property 3)
|
|
- **Property 3: Concurrency limit invariant**
|
|
- For any positive concurrency limit and stack list, concurrent in-progress updates never exceed the limit
|
|
- **Validates: Requirements 2.3, 3.3**
|
|
|
|
- [x] 5.7 Write property test for fault isolation (Property 7)
|
|
- **Property 7: Fault isolation — failures do not block remaining stacks**
|
|
- For any N updatable stacks where K fail, results are produced for all N stacks
|
|
- **Validates: Requirements 4.1, 4.4**
|
|
|
|
- [x] 5.8 Write property test for exponential backoff (Property 8)
|
|
- **Property 8: Throttling triggers exponential backoff retries**
|
|
- For any stack receiving throttling errors, retries up to `MAX_RETRIES` times
|
|
- Delay between attempt i and i+1 is at least `BASE_RETRY_DELAY * 2^i` seconds
|
|
- **Validates: Requirements 4.3**
|
|
|
|
- [x] 5.9 Write unit tests for stack updater
|
|
- Test successful update flow
|
|
- Test "no updates" response handling
|
|
- Test throttling with retry and eventual success
|
|
- Test throttling with retry exhaustion
|
|
- Test non-updatable stack skipping
|
|
- Test concurrent updates with Stubber
|
|
- _Requirements: 3.1, 3.2, 3.4, 4.1, 4.2, 4.3, 4.4_
|
|
|
|
- [x] 6. Checkpoint - Ensure all tests pass
|
|
- Ensure all tests pass, ask the user if questions arise.
|
|
|
|
- [ ] 7. Implement Report Generator
|
|
- [x] 7.1 Implement `generate_report()` and `format_report()` in `report.py`
|
|
- Aggregate `StackUpdateResult` list into `UpdateRunReport` with correct counts
|
|
- Format report as human-readable console output with start/end times, per-stack results, and summary totals
|
|
- _Requirements: 5.1, 5.2, 5.4_
|
|
|
|
- [x] 7.2 Write property test for report aggregation (Property 9)
|
|
- **Property 9: Report aggregation and exit code correctness**
|
|
- For any list of `StackUpdateResult`, report counts match actual counts per status
|
|
- Exit code is non-zero if and only if `failed > 0`
|
|
- **Validates: Requirements 5.2, 5.3**
|
|
|
|
- [x] 7.3 Write unit tests for report generator
|
|
- Test all-success scenario
|
|
- Test mixed results scenario
|
|
- Test empty results list
|
|
- Test format output contains expected fields
|
|
- _Requirements: 5.1, 5.2, 5.3, 5.4_
|
|
|
|
- [ ] 8. Implement Dry-Run Mode
|
|
- [x] 8.1 Add dry-run logic to the pipeline
|
|
- When `--dry-run` is set, discover stacks and output name + status for each, but make zero UpdateStack calls
|
|
- _Requirements: 6.1, 6.2, 6.3_
|
|
|
|
- [x] 8.2 Write property test for dry-run (Property 10)
|
|
- **Property 10: Dry-run performs no updates and lists all discovered stacks**
|
|
- For any set of discovered stacks with dry-run enabled, zero UpdateStack calls are made
|
|
- Output contains name and status of every discovered stack
|
|
- **Validates: Requirements 6.2, 6.3**
|
|
|
|
- [x] 8.3 Write unit tests for dry-run mode
|
|
- Test dry-run outputs all stack names and statuses
|
|
- Test dry-run makes no API update calls
|
|
- _Requirements: 6.1, 6.2, 6.3_
|
|
|
|
- [ ] 9. Implement CLI Entry Point and Wire Components Together
|
|
- [x] 9.1 Implement `main()` in `cli.py` with argument parsing
|
|
- Parse `--prefix`, `--concurrency`, `--dry-run`, `--region` flags using `argparse`
|
|
- Orchestrate the full pipeline: validate permissions → discover stacks → update (or dry-run) → generate report
|
|
- Return exit code 0 (success/no stacks/dry-run), 1 (any failure), or 2 (permission failure)
|
|
- Add `if __name__ == "__main__"` block
|
|
- _Requirements: 1.3, 1.4, 2.1, 2.2, 2.3, 2.4, 5.1, 5.2, 5.3, 5.4, 6.1, 7.1, 7.2_
|
|
|
|
- [x] 9.2 Write unit tests for CLI argument parsing and pipeline orchestration
|
|
- Test default argument values
|
|
- Test custom argument values
|
|
- Test invalid arguments produce non-zero exit
|
|
- Test end-to-end pipeline with mocked components
|
|
- _Requirements: 2.1, 2.3, 2.4_
|
|
|
|
- [x] 10. Final checkpoint - Ensure all tests pass
|
|
- Ensure all tests pass, ask the user if questions arise.
|
|
|
|
## Notes
|
|
|
|
- Tasks marked with `*` are optional and can be skipped for faster MVP
|
|
- Each task references specific requirements for traceability
|
|
- Property tests use `hypothesis` with a minimum of 100 iterations per property
|
|
- AWS API calls are mocked using `botocore.stub.Stubber` in all tests
|
|
- Checkpoints ensure incremental validation between major phases
|