Docs Rebuilder
Purpose
The docs-rebuilder is a Lambda function that orchestrates the complete documentation rebuild pipeline. When triggered (by webhooks or manually), it aggregates documentation from all repositories, builds the static site with MkDocs, and deploys to S3.
How It Works
The rebuilder executes a three-stage pipeline:
- Aggregate - Runs the aggregator to pull
/docsfolders from all GitHub repositories to/tmp/docs-build/docs - Build - Runs MkDocs to generate the static website to
/tmp/docs-build/site - Deploy - Uploads all files to S3 with correct MIME types and invalidates the CloudFront cache
Architecture
Runtime: Custom Docker container with: - Node.js 20 - Runs the aggregator and handler - Python 3.11 - Runs MkDocs and custom plugins - Git - Required for sparse checkout operations
Resources: - 2GB memory - 10GB ephemeral storage (needed for cloning multiple repositories) - 15-minute timeout
Structure
The Docker image packages three components:
/opt/aggregator- Pre-built TypeScript aggregator with dependencies/opt/mkdocs- MkDocs installation with custom plugins (lenient-builder)/var/task/handler.js- Main Lambda handler that orchestrates the pipeline
The handler is a Node.js script that:
- Fetches GitHub credentials from Secrets Manager
- Executes the aggregator via child_process
- Runs mkdocs build to generate the site
- Uploads to S3 using AWS SDK v3 with proper MIME types
- Creates CloudFront invalidation for immediate cache refresh
Local Testing
The handler supports local execution for testing:
./docs-rebuilder/test-local.sh
This runs the full pipeline locally without deploying to AWS.
Monitoring
Logs are available in CloudWatch Logs (DataDog instrumentation not yet supported for Docker Lambdas).
Log Group: /aws/lambda/AllDocsStack-DocsRebuilder* in us-east-1