Preventing Old Partitions from Executing When Reactivating AutomationCondition Sensors

Last updated: November 6, 2025

When reactivating a deactivated AutomationCondition sensor after an extended period, you may want to prevent execution of assets for partitions that were added while the sensor was inactive, and only execute for new partitions going forward.

Problem Description

After deactivating an AutomationCondition sensor for an extended period and then reactivating it, the sensor triggers execution for all partitions that were added during the inactive period. This can result in thousands of unwanted asset materializations, especially problematic for resource-intensive jobs like GPU workloads.

Symptoms

  • Thousands of partitions queue for execution immediately upon sensor reactivation

  • All partitions created during the sensor's inactive period are triggered

  • Resource-intensive downstream assets (like GPU jobs) execute unexpectedly

  • Error: AttributeError: type object 'AutomationCondition has no attribute 'updated_since_cron' when using incorrect API methods

Root Cause

The default AutomationCondition.eager() condition triggers execution for all available partitions when reactivated. Simple conditions like AutomationCondition.any_deps_updated() & ~AutomationCondition.initial_evaluation() still don't account for the time gap during which the sensor was inactive.

Solution

Use a time-aware automation condition that only triggers for newly updated dependencies going forward.

Step-by-Step Resolution

  1. Before reactivating, preview what assets would be triggered by clicking into the "target" section of your automation condition sensor in the Dagster UI

  2. Replace your automation condition with a time-aware condition that prevents execution of old partitions:

automation_condition = (
    AutomationCondition.any_deps_match(
        AutomationCondition.newly_updated()
        .since(AutomationCondition.cron_tick_passed("0 0 * * *"))
        & ~AutomationCondition.executed_with_root_target()
    )
    .newly_true()
    & ~AutomationCondition.in_progress()
    & AutomationCondition.in_latest_time_window()
)

  1. Test the condition with a small subset of assets first before applying to all downstream assets

  2. Monitor the behavior for a day or two to ensure it's working as expected

  3. Apply to all downstream assets once verified

Alternative Solutions

For more complex scenarios, you can create a custom AutomationCondition class that filters partitions based on specific date ranges:

class RecentPartitionsOnlyCondition(AutomationCondition):
    """Only allow materialization of partitions from the last 7 days."""
    def evaluate(self, context: AutomationContext) -> AutomationResult:
        from datetime import datetime, timedelta
        if not context.partitions_def:
            # Not partitioned, evaluate normally
            return AutomationResult(context, context.candidate_subset)
        
        # Get all candidate partition keys
        all_partitions = context.candidate_subset.expensively_compute_partition_keys()
        
        # Filter to recent partitions (assuming date-based partitions)
        cutoff = CUTOFF_DATETIME
        recent_partitions = {
            pk for pk in all_partitions
            if datetime.fromisoformat(pk) >= cutoff
        }
        
        # Create subset with only recent partitions
        true_subset = context.candidate_subset.compute_intersection_with_partition_keys(
            recent_partitions
        )
        return AutomationResult(context, true_subset=true_subset)

Prevention

When planning to deactivate sensors for extended periods, consider using time-aware automation conditions from the start. Always test automation condition changes in a development environment before applying to production systems with resource-intensive workloads.

Related Documentation