dbt-tests-adapter suite, packaging, Trusted Adapter Program

Заключительный урок модуля — comprehensive review of adapter testing, packaging, и пути к Trusted Adapter Program (TAP). Это production discipline для serious adapter projects.

dbt-tests-adapter package

Что это: Pip package от dbt Labs с base test classes. Used для adapter conformance testing.

Install:

pip install dbt-tests-adapter

Source: https://github.com/dbt-labs/dbt-adapters/tree/main/dbt-tests-adapter

Содержит ~50 test classes covering:

Basic operations

• BaseSimpleMaterializations — view, table, ephemeral
• BaseEphemeral — ephemeral materialization edge cases
• BaseValidateConnection — connection validation

Generic tests

• BaseGenericTests — unique, not_null, accepted_values, relationships
• BaseSingularTests — singular SQL tests

Materializations

• BaseIncremental — basic incremental
• BaseIncrementalUniqueKey — incremental с unique_key
• BaseIncrementalOnSchemaChange — schema evolution
• BaseIncrementalNotSchemaChange — vs schema change

Snapshots

• BaseSnapshots — timestamp strategy
• BaseSnapshotCheckCols — check strategy
• BaseSnapshotTimestamps — timestamp edge cases
• BaseSnapshotHardDeletes — hard_deletes config

Seeds

• BaseSeeds — basic seeds
• BaseSeedColumnQuoting — column name quoting
• BaseSeedTimezone — timezone handling

Concurrency

• BaseConcurrency — parallel runs

Schema management

• BaseConstraintsRuntimeDdlEnforcement — constraints
• BaseConstraintsRollback — rollback on constraint violation
• BaseModelContractsConstraints — contracts

Docs

• BaseDocsGenerate — docs generation
• BaseDocsGenerateBase — base
• BasePersistDocs — persist docs to warehouse

Exposures

• BaseExposures — exposures работают

Sources

• BaseSourceFreshness — source freshness

Grants

• BaseGrants — basic grants
• BaseIncrementalGrants — grants on incremental
• BaseSnapshotGrants — grants on snapshots
• BaseSeedGrants — grants on seeds

Operations

• BaseRunOperation — run-operation command
• BaseDbtClone — dbt clone command (1.7+)

How to integrate

В scaffold, tests/functional/adapter/test_basic.py:

from dbt.tests.adapter.basic.test_base import BaseSimpleMaterializations
from dbt.tests.adapter.basic.test_singular_tests import BaseSingularTests
from dbt.tests.adapter.basic.test_singular_tests_ephemeral import BaseSingularTestsEphemeral
from dbt.tests.adapter.basic.test_empty import BaseEmpty
from dbt.tests.adapter.basic.test_ephemeral import BaseEphemeral
from dbt.tests.adapter.basic.test_incremental import BaseIncremental
from dbt.tests.adapter.basic.test_generic_tests import BaseGenericTests
from dbt.tests.adapter.basic.test_snapshot_check_cols import BaseSnapshotCheckCols
from dbt.tests.adapter.basic.test_snapshot_timestamp import BaseSnapshotTimestamp
from dbt.tests.adapter.basic.test_adapter_methods import BaseAdapterMethod
from dbt.tests.adapter.basic.test_docs_generate import BaseDocsGenerate, BaseDocsGenerateReferences


class TestSimpleMaterializationsMyAdapter(BaseSimpleMaterializations):
    pass

class TestSingularTestsMyAdapter(BaseSingularTests):
    pass

class TestSingularTestsEphemeralMyAdapter(BaseSingularTestsEphemeral):
    pass

class TestEmptyMyAdapter(BaseEmpty):
    pass

class TestEphemeralMyAdapter(BaseEphemeral):
    pass

class TestIncrementalMyAdapter(BaseIncremental):
    pass

class TestGenericTestsMyAdapter(BaseGenericTests):
    pass

class TestSnapshotCheckColsMyAdapter(BaseSnapshotCheckCols):
    pass

class TestSnapshotTimestampMyAdapter(BaseSnapshotTimestamp):
    pass

class TestBaseAdapterMethodMyAdapter(BaseAdapterMethod):
    pass

class TestDocsGenerateMyAdapter(BaseDocsGenerate):
    pass

class TestDocsGenReferencesMyAdapter(BaseDocsGenerateReferences):
    pass

Каждый class — это inheritance. Tests run automatically.

Run:

pytest tests/functional/adapter/test_basic.py -v

Customization когда tests don’t fit

Sometimes base tests assume specific behavior. Override через fixtures:

class TestSimpleMaterializationsMyAdapter(BaseSimpleMaterializations):
    @pytest.fixture(scope='class')
    def project_config_update(self):
        return {
            'models': {
                '+materialized': 'view',
            }
        }
    
    @pytest.fixture(scope='class')
    def expected_relation_types(self):
        # Override expected relation types if your adapter differs
        return {
            'view_model': 'VIEW',
            'table_model': 'BASE TABLE',
        }

Other fixtures можно override:

• project_files — model SQL files
• seed_files — CSV files
• models — model configs
• snapshots — snapshot configs
• tests — test SQL

Opt-out approach

If warehouse genuinely doesn’t support feature, skip test:

class TestSnapshotsMyAdapter(BaseSnapshotCheckCols):
    @pytest.mark.skip(reason='MyAdapter does not support check_cols strategy')
    def test_snapshot_check_cols(self, project):
        pass

Document opt-outs в README. Reasonable opt-outs accepted by TAP review.

Setup для адаптер тестов

Conftest.py (test setup):

# tests/conftest.py
import pytest

pytest_plugins = ['dbt.tests.fixtures.project']


def pytest_addoption(parser):
    parser.addoption('--profile', action='store', default='myadapter', type=str)


@pytest.fixture(scope='class')
def dbt_profile_target():
    return {
        'type': 'myadapter',
        'host': os.environ.get('DBT_TEST_HOST', 'localhost'),
        'user': os.environ.get('DBT_TEST_USER', 'root'),
        'password': os.environ.get('DBT_TEST_PASSWORD', ''),
        'database': os.environ.get('DBT_TEST_DATABASE', 'test'),
        'schema': os.environ.get('DBT_TEST_SCHEMA', 'public'),
    }

Environment variables:

# .env (not committed)
DBT_TEST_HOST=localhost
DBT_TEST_USER=test_user
DBT_TEST_PASSWORD=test_pass
DBT_TEST_DATABASE=test_db
DBT_TEST_SCHEMA=test_schema

Local run:

source .env
pytest tests/functional/adapter/ -v

CI integration

GitHub Actions (генерирован cookiecutter):

# .github/workflows/test.yml
name: Adapter Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    
    services:
      myadapter:
        image: myadapter/myadapter-ce:latest
        ports:
          - 5432:5432
        env:
          POSTGRES_PASSWORD: test
    
    strategy:
      matrix:
        python-version: ['3.9', '3.10', '3.11', '3.12']
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}
      
      - name: Install
        run: |
          pip install -e .
          pip install -r dev-requirements.txt
      
      - name: Run tests
        run: pytest tests/functional/adapter/ -v
        env:
          DBT_TEST_HOST: localhost
          DBT_TEST_USER: test
          DBT_TEST_PASSWORD: test
          DBT_TEST_DATABASE: postgres

Packaging для distribution

setup.py (для PyPI release):

from setuptools import setup, find_namespace_packages

VERSION = '1.0.0'

setup(
    name='dbt-myadapter',
    version=VERSION,
    description='A dbt adapter for MyWarehouse',
    long_description=open('README.md').read(),
    long_description_content_type='text/markdown',
    author='Your Name',
    author_email='[email protected]',
    license='Apache-2.0',
    url='https://github.com/your-org/dbt-myadapter',
    
    packages=find_namespace_packages(include=['dbt.*']),
    package_data={
        'dbt': [
            'include/myadapter/dbt_project.yml',
            'include/myadapter/sample_profiles.yml',
            'include/myadapter/profile_template.yml',
            'include/myadapter/macros/*.sql',
            'include/myadapter/macros/**/*.sql',
        ],
    },
    
    install_requires=[
        'dbt-adapters>=1.7,<2.0',
        'dbt-common>=1.10,<2.0',
        'dbt-core>=1.8,<2.0',
        'pymysql>=1.0',
    ],
    
    python_requires='>=3.9',
    
    classifiers=[
        'Development Status :: 5 - Production/Stable',
        'License :: OSI Approved :: Apache Software License',
        'Operating System :: Microsoft :: Windows',
        'Operating System :: MacOS :: MacOS X',
        'Operating System :: POSIX :: Linux',
        'Programming Language :: Python :: 3.9',
        'Programming Language :: Python :: 3.10',
        'Programming Language :: Python :: 3.11',
        'Programming Language :: Python :: 3.12',
    ],
)

Build и upload:

python setup.py sdist bdist_wheel
twine upload dist/*

Now installable: pip install dbt-myadapter.

AdapterPlugin discovery

# dbt/adapters/myadapter/__init__.py
from dbt.adapters.myadapter.connections import MyConnectionManager, MyCredentials
from dbt.adapters.myadapter.impl import MyAdapter
from dbt.adapters.base import AdapterPlugin
from dbt.include import myadapter

Plugin = AdapterPlugin(
    adapter=MyAdapter,
    credentials=MyCredentials,
    include_path=myadapter.PACKAGE_PATH,
)

How dbt-core discovers:

1. User installs pip install dbt-myadapter.
2. Pip places package в site-packages: site-packages/dbt/adapters/myadapter/.
3. dbt-core scans dbt.adapters.* namespace.
4. Finds Plugin attribute в __init__.py.
5. Registers in FACTORY.plugins['myadapter'] = Plugin.
6. User’s profiles.yml type: myadapter -> finds Plugin -> instantiates adapter.

Без AdapterPlugin export — adapter не найден.

Trusted Adapter Program (TAP)

What it is: dbt Labs’ program признающий production-ready adapters.

Benefits:

• Listed на docs.getdbt.com/docs/trusted-adapters
• Discoverability через dbt Cloud Marketplace
• Community trust
• Endorsement by dbt Labs

Requirements:

1. All dbt-tests-adapter pass (or documented opt-outs)
2. Public GitHub repo с CI/CD
3. PyPI package with regular releases
4. Active maintainership:
   • Responsive к issues
   • Regular releases following dbt-core versions
   • Public roadmap
5. Documentation:
   • README aligned с getdbt.com style
   • Migration guides
   • Examples
   • Troubleshooting
6. Security review:
   • No hardcoded secrets
   • Sensitive data sanitization
   • Authentication best practices

Application process:

1. Self-assessment — все criteria met?
2. Submit application к dbt Labs (typically через GitHub issue или docs)
3. Review — dbt Labs evaluates (1-2 months)
4. Feedback — usually requested improvements
5. Iterate — address feedback
6. Approval — adapter listed
7. Maintenance — continue maintenance per program

Timeline: 9-12 months from start to TAP listing.

Examples Trusted Adapters

Current Trusted Adapters (2026):

• dbt-labs maintained (Tier 1):

  • dbt-postgres, dbt-redshift, dbt-snowflake, dbt-bigquery, dbt-spark, dbt-duckdb

• Community Trusted (Tier 2):

  • dbt-databricks (Databricks)
  • dbt-trino (Trino / Starburst)
  • dbt-clickhouse (ClickHouse)
  • dbt-doris (Doris)
  • dbt-firebolt (Firebolt)
  • dbt-impala (Impala)
  • dbt-rockset (Rockset)
  • dbt-singlestore (SingleStore)
  • dbt-synapse (Azure Synapse)
  • dbt-vertica (Vertica)
  • dbt-greenplum (Greenplum)
  • dbt-iotdb (Apache IoTDB)
  • dbt-materialize (Materialize)

• Community (not Trusted):

  • Various proprietary, newer adapters
  • Used at your own risk

Maintenance commitments

After TAP listing:

Release cadence:

• Follow dbt-core releases (minor version)
• Patch releases для bug fixes
• Major versions для breaking changes (with deprecation period)

Issue management:

• Acknowledge within 1 week
• Triage и label
• Critical security issues — fast turnaround

Community engagement:

• Slack channel (dbt-community)
• GitHub discussions
• Annual contributor meetings

Documentation:

• Keep up-to-date с dbt-core changes
• Update for new warehouse features
• Migration guides

This is long-term commitment. Adapters не one-shot projects.

Examples: dbt-snowflake structure

dbt-snowflake/
├── setup.py
├── pyproject.toml
├── README.md
├── LICENSE.md
├── CHANGELOG.md
├── .github/
│   └── workflows/
├── dbt/
│   ├── adapters/
│   │   └── snowflake/
│   │       ├── __init__.py
│   │       ├── __version__.py
│   │       ├── connections.py
│   │       ├── impl.py
│   │       ├── relation.py
│   │       ├── column.py
│   │       └── ...
│   └── include/
│       └── snowflake/
│           ├── dbt_project.yml
│           ├── profile_template.yml
│           ├── sample_profiles.yml
│           └── macros/
└── tests/
    ├── functional/
    ├── unit/
    └── integration/

Standard structure across all Trusted Adapters.

Попробуй сам

1. Install dbt-tests-adapter:

pip install dbt-tests-adapter

2. Look at source:

find $(pip show dbt-tests-adapter | grep Location | cut -d' ' -f2)/dbt/tests/adapter -name "*.py" | head -20

3. Read one test class — test_base.py:

cat $(pip show dbt-tests-adapter | grep Location | cut -d' ' -f2)/dbt/tests/adapter/basic/test_base.py

Understand structure: setup fixtures, model/seed files, test methods.

4. In your adapter scaffold, add test_basic.py с base classes.

5. Run:

pytest tests/functional/adapter/ -v

Verbose output — see passing/failing tests.

6. Iterate based on failures.

Ключевые выводы

1. dbt-tests-adapter — official conformance test suite. ~50 base test classes.

2. Integration через inheritance: class TestX(BaseY): pass. No test code from вас.

3. Customization через fixture overrides когда warehouse differs.

4. Opt-outs для truly unsupported features (с documentation).

5. Packaging: setup.py с namespace package + package_data для macros.

6. AdapterPlugin в __init__.py — discovery point.

7. Trusted Adapter Program — pathway для production credibility. 9-12 months timeline. Requires full tests pass, maintenance commitment, documentation, security review.

8. Trusted Adapters listed: dbt-postgres, dbt-snowflake, dbt-bigquery, dbt-duckdb, dbt-databricks, dbt-trino, etc.

9. Maintenance ongoing: follow dbt-core releases, respond к issues, community engagement.

10. Adapter development = months/years project. Plan accordingly.