Skip to content

Monitoring and Operations

Important: Region

All resources are deployed to us-east-1 (required by the StaticWebsite CDN construct). Make sure to select the correct region when viewing logs or invoking Lambdas in the AWS Console.

Logs

Webhook Handler Lambda

Logs are available in DataDog:

https://app.datadoghq.eu/logs?query=service%3Adocs-builder

The webhook handler is instrumented with DataDog APM, providing structured logs and traces.

Docs Rebuilder Lambda

The rebuilder is a Docker Lambda and DataDog instrumentation is not yet supported. Use CloudWatch Logs instead:

Log Group: /aws/lambda/AllDocsStack-DocsRebuilder*

Quick access via npm script:

# View recent logs
pnpm logs

# Follow logs in real-time
pnpm logs -- --follow

# Follow logs from the last 10 minutes
pnpm logs -- --follow --since 10m

# Filter logs for errors
pnpm logs -- --filter-pattern "ERROR"

Or use AWS CLI directly:

# Tail recent logs
aws logs tail --region us-east-1 --since 30m --follow \
  /aws/lambda/AllDocsStack-DocsRebuilderBA7BC04B-<hash>

# Or find the exact log group name
aws logs describe-log-groups \
  --region us-east-1 \
  --log-group-name-prefix "/aws/lambda/AllDocsStack-DocsRebuilder"

Alternatively, view logs in the AWS Console (ensure you're in us-east-1).

Triggering Content Rebuilds

Automatic Triggers

Content rebuilds happen automatically when: - Code is pushed to a repository's /docs folder on the default branch - A new repository with /docs is added (via GitHub App installation event)

Manual Trigger (AWS Console)

To manually trigger a rebuild:

  1. Go to Lambda Console
  2. Select the region: us-east-1
  3. Find the function: AllDocsStack-DocsRebuilder...
  4. Click the Test tab
  5. Create a test event (payload can be empty: {})
  6. Click Test to invoke

The rebuild takes 5-15 minutes depending on the number of repositories.

Manual Trigger (CLI)

Quick trigger via npm script:

pnpm rebuild-docs

Or use AWS CLI directly:

aws lambda invoke \
  --region us-east-1 \
  --function-name $(aws cloudformation describe-stacks \
    --region us-east-1 \
    --stack-name AllDocsStack \
    --query 'Stacks[0].Outputs[?OutputKey==`DocsRebuilderFunctionName`].OutputValue' \
    --output text) \
  --payload '{}' \
  /tmp/rebuild-result.json

cat /tmp/rebuild-result.json

CloudFront Cache Invalidation

The rebuilder Lambda automatically invalidates the CloudFront cache after uploading new content. However, if you need to manually invalidate:

# Get distribution ID
DIST_ID=$(aws cloudformation describe-stacks \
  --region us-east-1 \
  --stack-name AllDocsStack \
  --query 'Stacks[0].Outputs[?OutputKey==`DistributionId`].OutputValue' \
  --output text)

# Invalidate all paths
aws cloudfront create-invalidation \
  --distribution-id $DIST_ID \
  --paths "/*"

Cache invalidation takes a few minutes to propagate globally.

Common Issues

Changes not appearing on the site

  1. Check CloudWatch logs for the rebuilder Lambda to verify it completed successfully
  2. Verify CloudFront cache was invalidated (check logs for "Invalidation created")
  3. Try a hard refresh in your browser (Cmd+Shift+R or Ctrl+Shift+R)

Webhook not triggering rebuild

  1. Check DataDog logs for the webhook handler
  2. Verify the webhook signature is valid (check for "Invalid signature" errors)
  3. Confirm the push was to the default branch and modified /docs
  4. Check that the webhook handler successfully invoked the rebuilder

Rebuilder Lambda timeout

The rebuilder has a 15-minute timeout. If repositories are growing too large: 1. Check CloudWatch logs for which repo is slow 2. Consider optimizing sparse checkout depth 3. May need to increase Lambda memory (improves CPU allocation)