Compute logs not visible in Dagster Cloud UI showing "No Log file Available"

Problem Description

  AWS hybrid agents successfully capture compute logs in S3 buckets, but the logs are not visible in the Dagster Cloud UI. Users see the message "No Log file Available" even though logs are being written to S3 correctly.

Symptoms

• Message "No Log file Available" appears in the UI

• Both stdout and stderr tabs appear empty

• Log links at the bottom start with /tmp/ instead of s3:/

• Logs are successfully written to S3 bucket but not displayed in UI

Root Cause

  The compute log manager is not configured correctly in the deployment configuration. Specifically, the configuration key uses computeLogs instead of the correct compute_logs format.

Solution

  Change computeLogs to compute_logs in your deployment configuration

Step-by-Step Resolution

1. Locate your deployment configuration (typically in Helm chart values)

2. Update the configuration key from computeLogs to compute_logs:

compute_logs: # <- here
  enabled: true
  custom:
    module: dagster_aws.s3.compute_log_manager
    class: S3ComputeLogManager
    config:
      show_url_only: false
      bucket: your-s3-bucket-name
      region: us-east-1
      prefix: "your-deployment-name"
      use_ssl: true
      verify: true
      verify_cert_path: "/etc/ssl/certs/ca-certificates.crt"

3. Deploy the updated configuration

4. Verify that new runs show logs in both stdout and stderr tabs

5. Check that log links now start with s3:/ instead of /tmp/

Alternative Solutions (if applicable)

  Ensure your S3 bucket is in the same region as your Dagster Cloud organization and verify IAM permissions allow Dagster Cloud to read from S3

Prevention

  Always use the correct configuration key format compute_logs when setting up S3 compute log managers. Double-check your deployment configuration against the official documentation before deploying.

Related Documentation

• S3ComputeLogManager Documentation