AdapterCapability и BehaviorFlag

Не каждый warehouse поддерживает каждую dbt feature. AdapterCapability позволяет adapter’у декларировать что он поддерживает — dbt-core тогда graceful опускает или адаптирует поведение.

BehaviorFlag — более новый mechanism (dbt-adapters 1.5+) для opt-in / opt-out новых behaviors. Allows gradual rollout features без breaking existing adapters.

В этом уроке — оба mechanism с examples.

AdapterCapability

# dbt-adapters/dbt/adapters/capability.py
from enum import Enum

class Capability(str, Enum):
    SchemaMetadataByRelations = 'SchemaMetadataByRelations'
    TableLastModifiedMetadata = 'TableLastModifiedMetadata'
    TableLastModifiedMetadataBatch = 'TableLastModifiedMetadataBatch'
    # ... другие

Capability — enum значений, идентифицирующих специфические features.

Support levels:

from enum import Enum

class Support(str, Enum):
    Unknown = 'Unknown'           # default — unspecified
    Unsupported = 'Unsupported'   # adapter explicitly не supports
    NotImplemented = 'NotImplemented'   # might be added later
    Full = 'Full'                  # fully supported
    Versioned = 'Versioned'         # supported в specific warehouse versions

Declaration в adapter:

from dbt.adapters.capability import (
    Capability, CapabilityDict, CapabilitySupport, Support
)

class MyAdapter(SQLAdapter):
    @classmethod
    def capabilities(cls) -> CapabilityDict:
        return CapabilityDict({
            Capability.SchemaMetadataByRelations: CapabilitySupport(support=Support.Full),
            Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.NotImplemented),
            Capability.TableLastModifiedMetadataBatch: CapabilitySupport(support=Support.Unsupported),
        })

dbt-core checks capabilities перед использованием feature:

# В dbt-core internals
if adapter.has_feature(Capability.TableLastModifiedMetadata):
    # Use the feature
    last_modified = adapter.get_table_last_modified(relation)
else:
    # Skip / use fallback
    last_modified = None

Common capabilities

SchemaMetadataByRelations

Что это: ability fetch metadata for multiple relations в одном query (vs one-at-a-time).

Why useful: для dbt docs generate на больших projects — batch fetch metadata.

Adapters that support:

• Snowflake: Full (DESCRIBE TABLES IN SCHEMA)
• BigQuery: Full
• Postgres: Full (information_schema queries)

Adapters that don’t:

• Спецификal proprietary warehouses

Implementation:

class MyAdapter(SQLAdapter):
    @classmethod
    def capabilities(cls) -> CapabilityDict:
        return CapabilityDict({
            Capability.SchemaMetadataByRelations: CapabilitySupport(support=Support.Full),
        })
    
    def get_relations_in_schema(self, schema_relation):
        # Fetch all relations metadata in one query
        ...

TableLastModifiedMetadata

Что это: query когда relation was last modified.

Why useful: source freshness checks.

Adapters that support:

• Snowflake: information_schema.tables.LAST_ALTERED
• BigQuery: TABLES.last_modified_time
• Postgres: pg_stat_user_tables.last_analyze

Implementation:

@classmethod
def capabilities(cls):
    return CapabilityDict({
        Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.Full),
    })

def calculate_freshness_from_metadata(self, source):
    # Query metadata for last_modified
    ...

TableLastModifiedMetadataBatch

Что это: batch version — fetch last_modified для multiple relations в одном query.

Why useful: scale.

How dbt-core uses capabilities

# Pseudocode из dbt-core
def check_source_freshness(adapter, source):
    if adapter.has_feature(Capability.TableLastModifiedMetadata):
        # Fast path — query metadata
        last_modified = adapter.get_metadata_freshness(source)
        return compare_to_threshold(last_modified, source.freshness)
    else:
        # Slow path — execute custom query
        last_modified = adapter.calculate_freshness_with_sql(source)
        return compare_to_threshold(last_modified, source.freshness)

dbt-core auto-detects available capabilities и chooses optimal path.

Gracefully degrade: adapter не имеющий capability — feature missing, не error.

BehaviorFlag (dbt-adapters 1.5+)

BehaviorFlag — newer mechanism для behavior changes. Different from capabilities — for opt-in / opt-out new behaviors.

# dbt-adapters/dbt/adapters/behavior.py
from dbt_common.behavior_flags import Behavior


class MyAdapterBehavior(Behavior):
    flags = [
        {
            'name': 'enable_new_caching',
            'default': False,
            'description': 'Enable new caching mechanism (faster but uses more memory)',
            'docs_url': 'https://docs.getdbt.com/...',
        },
        {
            'name': 'strict_quoting',
            'default': True,
            'description': 'Strictly enforce quoting policy',
        },
    ]

Use в adapter:

class MyAdapter(SQLAdapter):
    def list_relations_without_caching(self, schema_relation):
        if self.behavior.enable_new_caching.no_warn:
            return self._list_relations_with_new_cache(schema_relation)
        else:
            return self._list_relations_default(schema_relation)

Use в profiles.yml (or dbt_project.yml):

flags:
  enable_new_caching: True

User opts-in to new behavior.

When to use BehaviorFlag

Use case 1 — Breaking change rollout:

You want to change behavior, but не break existing users. Solution: feature flag.

# Old default behavior
def get_columns(self, relation):
    return self._old_implementation(relation)

# Add flag
def get_columns(self, relation):
    if self.behavior.use_new_columns_method.no_warn:
        return self._new_implementation(relation)
    else:
        return self._old_implementation(relation)

Initially default: False. Users opt-in.

After period — change default к True. Users who didn’t opt-in get warning.

Eventually — remove flag, only new behavior remains.

Gradual migration без big-bang breaking.

Use case 2 — Performance optimization:

New behavior faster but riskier. Some workloads not ready.

flags = [
    {
        'name': 'batch_metadata_fetch',
        'default': False,
        'description': 'Batch metadata queries (10x faster but uses more memory)',
    },
]

Performance-conscious users opt-in. Conservative users stay на default.

Use case 3 — Experimental features:

New feature not yet stable. Hide behind flag while developing.

flags = [
    {
        'name': 'enable_experimental_iceberg',
        'default': False,
        'description': 'Enable experimental Iceberg materialization (alpha)',
    },
]

Brave users test. Most stay safe.

BehaviorFlag vs AdapterCapability

AdapterCapability = static — adapter says ‘I support X’.

BehaviorFlag = dynamic — user says ‘I want behavior Y’.

Examples в популярных adapters

dbt-postgres

class PostgresAdapter(SQLAdapter):
    @classmethod
    def capabilities(cls):
        return CapabilityDict({
            Capability.SchemaMetadataByRelations: CapabilitySupport(support=Support.Full),
            Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.Full),
        })

Postgres supports both — full.

dbt-snowflake

class SnowflakeAdapter(SQLAdapter):
    @classmethod
    def capabilities(cls):
        return CapabilityDict({
            Capability.SchemaMetadataByRelations: CapabilitySupport(support=Support.Full),
            Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.Full),
            Capability.TableLastModifiedMetadataBatch: CapabilitySupport(support=Support.Full),
        })

Snowflake supports batch metadata fetching — full.

dbt-bigquery

class BigQueryAdapter(BaseAdapter):
    @classmethod
    def capabilities(cls):
        return CapabilityDict({
            Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.Full),
        })

Different feature set due to BigQuery’s nature (no traditional information_schema).

dbt-spark

class SparkAdapter(BaseAdapter):
    @classmethod
    def capabilities(cls):
        return CapabilityDict({
            # Limited capabilities — Spark different
            Capability.SchemaMetadataByRelations: CapabilitySupport(support=Support.Unsupported),
        })

Spark uses DataFrame API differently — many SQL-features don’t apply.

Adapter version evolution

Sometimes adapter adds new capability в new version:

class MyAdapter(SQLAdapter):
    @classmethod
    def capabilities(cls):
        # Version 1.0
        if cls.adapter_version < '1.5':
            return CapabilityDict({
                Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.NotImplemented),
            })
        # Version 1.5+
        return CapabilityDict({
            Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.Full),
        })

Versioned capabilities — older versions admit don’t support, newer do.

BehaviorFlag in practice

# dbt-myadapter/dbt/adapters/myadapter/impl.py
from dbt_common.behavior_flags import Behavior


class MyAdapterBehavior(Behavior):
    flags = [
        {
            'name': 'use_iceberg_default',
            'default': False,
            'description': (
                'Use Iceberg as default materialization для tables. '
                'Otherwise uses native table format. '
                'Requires Iceberg catalog setup.'
            ),
            'docs_url': 'https://docs.getdbt.com/docs/myadapter/iceberg',
        },
    ]


class MyAdapter(SQLAdapter):
    BEHAVIORS = MyAdapterBehavior
    
    def default_materialization(self):
        if self.behavior.use_iceberg_default.no_warn:
            return 'iceberg'
        return 'table'

User opts in:

# dbt_project.yml
flags:
  use_iceberg_default: True

Or per-project:

models:
  +use_iceberg_default: True

After opt-in — все tables materialized as Iceberg.

Use BehaviorFlag для backwards compat

class MyAdapterBehavior(Behavior):
    flags = [
        {
            'name': 'enable_new_quote_policy',
            'default': False,
            'description': 'Use new quoting policy that handles edge cases better',
        },
    ]


class MyAdapter(SQLAdapter):
    def get_relation_class(self):
        if self.behavior.enable_new_quote_policy.no_warn:
            return MyNewQuotingRelation
        return MyLegacyRelation

Phase 1 (release X.0):

• Add flag, default: False
• Old behavior maintained для everyone
• Document new option

Phase 2 (release X.5):

• Change default к True
• Warning if user explicitly sets к False (‘deprecated’)

Phase 3 (release X+1.0):

• Remove flag
• Only new behavior

Gradual migration. Users have time to adapt.

Попробуй сам

1. В Python REPL:

from dbt.adapters.capability import Capability, CapabilitySupport, Support, CapabilityDict

caps = CapabilityDict({
    Capability.SchemaMetadataByRelations: CapabilitySupport(support=Support.Full),
    Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.NotImplemented),
})

print(caps.get(Capability.SchemaMetadataByRelations))
# CapabilitySupport(support=Support.Full)

print(caps.get(Capability.TableLastModifiedMetadata))
# CapabilitySupport(support=Support.NotImplemented)

2. В custom adapter define capabilities:

class MyAdapter(SQLAdapter):
    @classmethod
    def capabilities(cls):
        return CapabilityDict({
            Capability.SchemaMetadataByRelations: CapabilitySupport(support=Support.Full),
        })

3. Test через dbt:

dbt --debug debug
# Look for capability negotiation logs

4. For BehaviorFlag — добавьте flag в profiles.yml:

flags:
  my_custom_flag: True

Run dbt. Verify behavior changes.

Ключевые выводы

1. AdapterCapability — declarative: adapter says ‘I support X’. dbt-core uses to choose code paths.

2. Common capabilities: SchemaMetadataByRelations, TableLastModifiedMetadata, TableLastModifiedMetadataBatch.

3. Support levels: Unknown, Unsupported, NotImplemented, Full, Versioned.

4. BehaviorFlag (dbt-adapters 1.5+) — opt-in/opt-out behaviors. For rolling out changes без breaking existing.

5. AdapterCapability vs BehaviorFlag:

   • Capability: static, adapter-set, declares support
   • BehaviorFlag: dynamic, user-set, opt-in changes

6. Gradual rollout pattern: добавить flag (default False) -> change default к True -> remove flag. Years of phased migration.

7. Production-grade adapter — use both правильно. Capabilities для feature declaration, BehaviorFlag для controlled rollout.