dbt Models Not Appearing in Dagster UI After Scaffolding

Problem Description

  After scaffolding a dbt component in Dagster, dbt models are not visible when running dg list defs or in the Dagster UI. This occurs even when dbt configuration appears correct and dbt debug passes all checks.

Symptoms

• No dbt models appear when running dg list defs

• dbt models are not visible in the Dagster UI

• dbt debug returns all checks passed

• dbt project and profiles YAML files are correctly configured

Root Cause

  The definitions.py file is not properly configured to recursively load dbt components from the project structure, preventing Dagster from discovering and registering the dbt models.

Solution

  Update your definitions.py file to use recursive loading with load_from_defs_folder

Step-by-Step Resolution

1. First, run diagnostic commands to verify your configuration:

   dg check defs

2. Ensure your dbt manifest exists by running:

   dbt deps
   dbt parse

3. Update your definitions.py file to use recursive loading:

   @definitions
   def defs():
       return load_from_defs_folder(project_root=Path(__file__).parent.parent)

4. Verify your defs.yaml configuration uses the correct format:

   type: dagster_dbt.DbtProjectComponent
   attributes:
     project:
       project_dir: "{{ project_root }}/dbt"  # Use forward slashes
       profiles_dir: "{{ project_root }}"

5. Run dg list defs again to verify your dbt models are now visible

Alternative Solutions (if applicable)

  If the above doesn't work, ensure your dbt project structure matches the expected layout and that all required dependencies are installed with compatible versions.

Prevention

  When scaffolding dbt components, always configure the definitions.py file to use recursive loading to ensure all components are properly discovered and registered by Dagster.