Troubleshooting Sigma API Rate Limits and Timeouts in Dagster

When working with the dagster-sigma integration, you may encounter timeout errors or connection issues, particularly when loading workbooks with the Sigma API. These issues are often related to Sigma's API rate limits.

Common error messages you might see include:

aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://api.sigmacomputing.com/v2/workbooks/...

To address these timeout issues, you can try the following steps:

1. Increase the local startup timeout in your Dagster configuration:

   code_servers:
     local_startup_timeout: 600

2. Optimize your Sigma asset loading by minimizing unnecessary data fetching:

   sigma_assets = load_sigma_asset_specs_with_retry(
       organization=sigma_organization,
       fetch_lineage_data=False,
       fetch_column_data=False,
   )

If you continue to experience rate limit issues, consider using Dagster's newer CI-focused APIs for loading from Sigma, which are designed to avoid these rate limits. These APIs can be found in newer versions of the dagster-sigma integration.

For more information about Dagster's Sigma integration, refer to the Sigma integration documentation.