- Python CLI tool for rolling updates of CL-AppPipe-* and CL-SvcPipe-* stacks - Async update engine with configurable concurrency (asyncio.Semaphore) - Exponential backoff retry for API throttling - Dry-run mode for safe preview - IAM permission pre-validation - Comprehensive test suite (80 tests: 11 property-based + 69 unit) - Full spec documentation (requirements, design, tasks)
185 lines
6.2 KiB
Markdown
185 lines
6.2 KiB
Markdown
# One-Click CloudFormation Stack Updater
|
|
|
|
A Python CLI tool that discovers and updates all `CL-AppPipe-*` and `CL-SvcPipe-*` CloudFormation stacks in an AWS audit account. When a new version of the Centralized Logging with OpenSearch nested templates becomes available, this tool triggers a rolling update across every matching stack with a single command.
|
|
|
|
## Supported Stack Types
|
|
|
|
| Prefix | Solution ID | Template |
|
|
|--------|-------------|----------|
|
|
| `CL-AppPipe-*` | SO8025-s3b | `AppLogS3Buffer.template` |
|
|
| `CL-SvcPipe-*` | SO8025-s3 | `S3AccessLog.template` |
|
|
|
|
Each prefix is automatically mapped to its correct template URL. Running without `--prefix` updates both types.
|
|
|
|
## What It Does
|
|
|
|
1. Validates IAM permissions before starting
|
|
2. Discovers all stacks matching the `CL-AppPipe-` prefix
|
|
3. Updates each stack using its existing parameters (only the template URL is refreshed)
|
|
4. Runs updates concurrently with configurable parallelism
|
|
5. Produces a summary report showing succeeded, failed, skipped, and no-update-needed counts
|
|
|
|
## Prerequisites
|
|
|
|
- Python 3.10+
|
|
- AWS CLI configured with a profile that has access to the audit account
|
|
- Required IAM permissions:
|
|
- `cloudformation:ListStacks`
|
|
- `cloudformation:DescribeStacks`
|
|
- `cloudformation:UpdateStack`
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
# Install dependencies
|
|
pip install boto3 botocore
|
|
|
|
# For development (tests)
|
|
pip install pytest hypothesis
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Dry Run (preview which stacks would be updated)
|
|
|
|
```bash
|
|
# Preview all stack types (CL-AppPipe-* and CL-SvcPipe-*)
|
|
py -m cfn_updater.cli --profile audit --dry-run
|
|
|
|
# Preview only CL-AppPipe-* stacks
|
|
py -m cfn_updater.cli --profile audit --prefix "CL-AppPipe-" --dry-run
|
|
|
|
# Preview only CL-SvcPipe-* stacks
|
|
py -m cfn_updater.cli --profile audit --prefix "CL-SvcPipe-" --dry-run
|
|
```
|
|
|
|
### Run the Update
|
|
|
|
```bash
|
|
# Update all stack types
|
|
py -m cfn_updater.cli --profile audit
|
|
|
|
# Update only CL-AppPipe-* stacks
|
|
py -m cfn_updater.cli --profile audit --prefix "CL-AppPipe-"
|
|
|
|
# Update only CL-SvcPipe-* stacks
|
|
py -m cfn_updater.cli --profile audit --prefix "CL-SvcPipe-"
|
|
```
|
|
|
|
### CLI Flags
|
|
|
|
| Flag | Type | Default | Description |
|
|
|------|------|---------|-------------|
|
|
| `--profile` | string | None | AWS profile name (e.g. `audit`) |
|
|
| `--region` | string | SDK default | AWS region override |
|
|
| `--prefix` | string | all configured | Stack name prefix to match (omit to update all types) |
|
|
| `--concurrency` | int | `5` | Max parallel stack updates |
|
|
| `--dry-run` | flag | `False` | Preview mode — lists stacks without updating |
|
|
|
|
### Examples
|
|
|
|
```bash
|
|
# Dry run all stack types with audit profile
|
|
py -m cfn_updater.cli --profile audit --dry-run
|
|
|
|
# Update only CL-SvcPipe-* stacks, 3 at a time
|
|
py -m cfn_updater.cli --profile audit --prefix "CL-SvcPipe-" --concurrency 3
|
|
|
|
# Update all stacks in a specific region
|
|
py -m cfn_updater.cli --profile audit --region us-east-1
|
|
|
|
# Update only CL-AppPipe-* stacks
|
|
py -m cfn_updater.cli --profile audit --prefix "CL-AppPipe-"
|
|
```
|
|
|
|
## Exit Codes
|
|
|
|
| Code | Meaning |
|
|
|------|---------|
|
|
| `0` | All stacks updated successfully, no stacks found, or dry-run |
|
|
| `1` | One or more stacks failed to update |
|
|
| `2` | Permission validation failed |
|
|
|
|
## Configuration
|
|
|
|
Default values are in `cfn_updater/config.py`:
|
|
|
|
| Constant | Value | Description |
|
|
|----------|-------|-------------|
|
|
| `TEMPLATE_URL` | `https://s3.amazonaws.com/.../AppLogS3Buffer.template` | Default template URL (CL-AppPipe-*) |
|
|
| `STACK_PROFILES` | `{"CL-AppPipe-": "...AppLogS3Buffer.template", "CL-SvcPipe-": "...S3AccessLog.template"}` | Prefix-to-template mapping |
|
|
| `DEFAULT_PREFIX` | `CL-AppPipe-` | Legacy default prefix |
|
|
| `DEFAULT_CONCURRENCY` | `5` | Max parallel updates |
|
|
| `MAX_RETRIES` | `3` | Retry attempts on throttling |
|
|
| `BASE_RETRY_DELAY` | `1.0` seconds | Base delay for exponential backoff |
|
|
|
|
## Error Handling
|
|
|
|
- **Throttling**: Automatically retries with exponential backoff (1s, 2s, 4s) up to 3 times
|
|
- **Non-updatable stacks**: Stacks in states like `ROLLBACK_COMPLETE` or `DELETE_IN_PROGRESS` are skipped
|
|
- **"No updates needed"**: Treated as success when the stack is already on the latest template
|
|
- **Individual failures**: One stack failing does not block the rest — all stacks are attempted
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
cfn_updater/
|
|
├── __init__.py # Package exports
|
|
├── cli.py # CLI entry point and pipeline orchestration
|
|
├── config.py # Configuration constants
|
|
├── discovery.py # Stack discovery (prefix filtering, pagination)
|
|
├── models.py # Data models (DiscoveredStack, StackUpdateResult, UpdateRunReport)
|
|
├── permissions.py # IAM permission validation
|
|
├── report.py # Report generation and formatting
|
|
└── updater.py # Stack update engine (async, concurrency, retry)
|
|
|
|
tests/
|
|
├── test_cli.py # CLI argument parsing and pipeline tests
|
|
├── test_discovery.py # Stack discovery property + unit tests
|
|
├── test_dry_run.py # Dry-run property + unit tests
|
|
├── test_permissions.py # Permission validation property + unit tests
|
|
├── test_report.py # Report aggregation property + unit tests
|
|
└── test_updater.py # Update engine property + unit tests
|
|
```
|
|
|
|
## Running Tests
|
|
|
|
```bash
|
|
# Run all tests
|
|
py -m pytest tests/ -v
|
|
|
|
# Run a specific test file
|
|
py -m pytest tests/test_updater.py -v
|
|
|
|
# Run with short output
|
|
py -m pytest tests/
|
|
```
|
|
|
|
The test suite includes 80 tests: 11 property-based tests (using Hypothesis) and 69 unit tests. All AWS API calls are mocked — no real AWS credentials needed for testing.
|
|
|
|
## Sample Output
|
|
|
|
```
|
|
Discovered 22 stack(s).
|
|
============================================================
|
|
CloudFormation Stack Update Report
|
|
============================================================
|
|
Start Time : 2026-04-02T02:20:41.247416+00:00
|
|
End Time : 2026-04-02T02:20:54.620248+00:00
|
|
Total Found: 22
|
|
|
|
Per-Stack Results:
|
|
------------------------------------------------------------
|
|
CL-AppPipe-9894aa72: succeeded (0.4s)
|
|
CL-AppPipe-ca6dca90: succeeded (0.6s)
|
|
CL-AppPipe-8521cc5e: no-update-needed (0.5s)
|
|
...
|
|
|
|
Summary:
|
|
------------------------------------------------------------
|
|
Succeeded : 20
|
|
Failed : 0
|
|
Skipped : 1
|
|
No Update Needed: 1
|
|
============================================================
|
|
```
|