Learning Platform
Глоссарий Troubleshooting
Урок 06.09 · 90 мин
Продвинутый
LabControls MatrixDriver EarningsCDE-SWR-003dbt + GX Core stubSOX-grade evidence

Введение

8 уроков M5 покрыли control taxonomy, ITGC, application controls, 3-level structure, DQ dimensions, reconciliation, SoD, lineage. Lab — practical synthesis: вы как SwiftRide CDO Office T+6M строите полный controls matrix для одного material CDE — CDE-SWR-003 driver_earnings_ledger (M1.7 weighted score 4.50; multi-regulator SOX + GDPR + IRS 1099 + labor; прецедент SwiftPay 2024 $2.3M DACH incident).

Lab — doc-centric (обязательный). Output — controls matrix Markdown / YAML с минимум 12 контролей; objective + activity + evidence per control; ownership + frequency + regulation refs. Opt-in tooling lab — dbt + GX Core 1.17.1 stub с 3-5 expectations live runnable.

Inputs

CDE-SWR-003 driver_earnings_ledger

Per M4.8 lab — полный registry entry. Ключевые поля для M5.9:

  • business_definition — Daily driver earnings ledger с gross_earnings, commission, net_payout, currency_code per driver per day. Прямой feed SwiftPay payouts (Aurora) → IRS 1099-NEC export ежегодно.
  • applicable_regulations — SOX 404 (financial reporting), GDPR Art. 30 (PII), AMLR (transaction monitoring), IRS 1099-NEC (tax reporting), labor regulation (timely payment).
  • criticality_score — 4.50 (financial 4, regulatory 5, operational 4, reputational 5; per M1.7 weighted average).
  • quality_tolerance — null_rate ≤0.01%; reconciliation_delta ≤0.05%; pipeline_freshness ≤6h SLA.
  • retention — 7y immutable SOX-grade.

Risk register (M2.6)

5 идентифицированных рисков на CDE-SWR-003 (per M2.7 lab):

  1. R-DE-001 — Commission calculation error (formula bug) → material misstatement.
  2. R-DE-002 — Late-arriving trips skew daily aggregates → cut-off error.
  3. R-DE-003 — Unauthorised change в commission_rules engine → fraud / error.
  4. R-DE-004 — Snowflake-Aurora drift из-за schema migration → silent corruption.
  5. R-DE-005 — Privileged access misuse (engineer читает PII или модифицирует значения) → GDPR breach + integrity issue.

Lab workflow

Workflow M5 Lab — 6 шагов к controls matrix

Sequential декомпозиция; input от M4 registry + M2 risk register; output controls matrix YAML + self-check.

1. ObjectivesШаг 1 — выводим 1-3 control objectives из записей risk register
2. ActivitiesШаг 2 — specifyirовать 1-2 activities per objective (всего минимум 12); каждый verb-first + frequency + actor + outcome chain
3. EvidenceШаг 3 — определить evidence per activity (6 атрибутов: timestamp, immutable, signed, outcome, retention, queryable)
4. OwnersШаг 4 — назначить owners per SoD 4-actor model + M4.4 RACI
5. Cross-refsШаг 5 — cross-references risk + regulation + lineage + forward dependencies к M6-M9
6. Self-checkШаг 6 — self-check против 4 категорий критериев

Шаги выполнения

Шаг 1 — Define control objectives (1-3 per CDE) per M5.4

Группируем risks в objectives. Каждый objective сформулирован как желаемое состояние outcome.

control_objectives:
  - id: OBJ-CDE-SWR-003-01
    title: "Accuracy of commission calculation"
    description: "Driver earnings commission calculation produces accurate values per documented formula; any systematic error detected within 24 hours; delta vs source-of-truth ≤0.05%."
    risk_refs: [R-DE-001, R-DE-004]
    regulation_refs: [SOX 404, BCBS 239 Principle 3]

  - id: OBJ-CDE-SWR-003-02
    title: "Integrity of commission rules"
    description: "Changes to commission_rules engine require 4-eyes peer review + Finance Lead approval + signed deployment; no direct UI modification."
    risk_refs: [R-DE-003]
    regulation_refs: [SOX 404 ITGC change management, PCAOB AS 2201 ¶.47]

  - id: OBJ-CDE-SWR-003-03
    title: "Completeness of earnings ledger"
    description: "All eligible completed trips included in daily aggregate; no silent drops; row count vs expected (active drivers × period) within ±5%; late-arriving trips handled via adjustment journal."
    risk_refs: [R-DE-002]
    regulation_refs: [BCBS 239 Principle 4, IRS 1099 completeness]

  - id: OBJ-CDE-SWR-003-04
    title: "Timeliness + access integrity"
    description: "Pipeline completes within T+1 06:00 UTC SLA; access restricted via Snowflake RBAC + JIT; no privileged access without 2-person Saviynt PIM approval; PII fields masked except authorised."
    risk_refs: [R-DE-005]
    regulation_refs: [BCBS 239 Principle 5, GDPR Art. 30, PCI-DSS Req. 8.4]

Шаг 2 — Specify control activities (1-2 per objective, всего минимум 12) per M5.1 + M5.3 + M5.4

Каждый activity сформулирован с verb-first + frequency + actor + outcome chain.

control_activities:
  # OBJ-01 — Accuracy
  - id: CTL-CDE-SWR-003-001
    objective_id: OBJ-CDE-SWR-003-01
    title: "GE Core expectation suite — earnings range checks"
    type: preventive-automated-business  # M5.1 cube cell
    activity: "Hourly GE Core 1.17.1 expectation suite на fct_driver_earnings.gross_earnings_usd; expectations: > 0, < $50K per row, per-BU mean within trailing 30-day baseline ± 3 SD. Fail → PagerDuty Sev-2 + halt downstream consumers."
    frequency: hourly
    actor: airflow_runner_service_account
    owner: data_platform_lead
    tooling: GE_Core_1.17.1
    evidence: ge_data_docs_html + validation_result_json → S3 object lock 7y

  - id: CTL-CDE-SWR-003-002
    objective_id: OBJ-CDE-SWR-003-01
    title: "Daily cross-system reconciliation — Snowflake vs Aurora"
    type: detective-automated-business
    activity: "Daily Airflow DAG T+1 06:00 UTC: reconcile snowflake.fct_driver_earnings vs aurora.swiftpay.payouts; count match (±2 records) + sum match (delta ≤0.05%); close-out 04:00 UTC; late events → adjustment journal. Fail → PagerDuty Sev-1 + ServiceNow Change ticket + auto-trigger reprocess DAG."
    frequency: daily
    actor: airflow_runner_service_account
    owner: data_platform_lead
    tooling: dbt_audit_helper + Python
    evidence: reconciliation_log_json HMAC-signed → S3 object lock 7y; queryable audit.evidence_index

  - id: CTL-CDE-SWR-003-003
    objective_id: OBJ-CDE-SWR-003-01
    title: "Formula parity check — independent Python recomputation"
    type: detective-automated-business
    activity: "Daily Airflow task — 1000 random fct_driver_earnings rows перевычисляются через отдельную Python implementation (canonical formula stored в commission_rules.yaml); 0 mismatches tolerated. Mismatch → ServiceNow ticket; Finance Lead investigation."
    frequency: daily
    actor: airflow_runner_service_account
    owner: data_quality_lead
    tooling: custom_python
    evidence: parity_check_report_json + python_codepath_sha → S3 object lock 7y

  # OBJ-02 — Integrity
  - id: CTL-CDE-SWR-003-004
    objective_id: OBJ-CDE-SWR-003-02
    title: "CODEOWNERS + branch protection на commission_rules"
    type: preventive-automated-it
    activity: "GitHub branch protection на main repo: PR modifying /dbt/models/marts/driver_earnings/commission_rules.sql требует Finance Lead approval + Data Platform Lead approval + signed commits + dbt build/tests passing. No bypass даже для admins (force-push disabled)."
    frequency: per_change
    actor: github_action + codeowners
    owner: cto_office
    tooling: github_codeowners
    evidence: github_pr_commit_sha + signed_commit + codeowners_approval_log → mirror archive S3 7y

  - id: CTL-CDE-SWR-003-005
    objective_id: OBJ-CDE-SWR-003-02
    title: "Lineage-guarded schema migration"
    type: preventive-automated-business
    activity: "PR modifying commission_rules schema триггерит OpenLineage event → Marquez идентифицирует downstream CDE-marked artefacts → CI lineage analysis posts PR comment с impact requirements → Data Owners (Sami SwiftPay, Carlos Finance, Tax Compliance Lead) review + sign → CODEOWNERS approval required for merge."
    frequency: per_change
    actor: ci_pipeline + marquez
    owner: data_platform_lead
    tooling: OpenLineage_1.46 + Marquez_0.51
    evidence: openlineage_event_json + marquez_export + impact_analysis_document + signed_attestations → S3 object lock 7y

  - id: CTL-CDE-SWR-003-006
    objective_id: OBJ-CDE-SWR-003-02
    title: "dbt contract enforcement"
    type: preventive-automated-business
    activity: "dbt 1.9 contract на fct_driver_earnings columns + types + non-null constraints; CI запускает dbt build --select state:modified --defer-state prod; contract violation (column drop, type narrowing, non-null relaxation) fails build."
    frequency: per_change
    actor: ci_pipeline
    owner: data_platform_lead
    tooling: dbt_1.9
    evidence: dbt_manifest_json + run_results_json + ci_log → S3 object lock 7y

  # OBJ-03 — Completeness
  - id: CTL-CDE-SWR-003-007
    objective_id: OBJ-CDE-SWR-003-03
    title: "Daily completeness check + reprocess trigger"
    type: detective-automated-business
    activity: "Daily Airflow task T+1 06:00 UTC: count fct_driver_earnings rows для prior period vs expected (active_drivers_count × period × 1.0 ±5%); out-of-range → ServiceNow ticket Sev-2 + auto-trigger reprocess_driver_earnings(date_range) DAG."
    frequency: daily
    actor: airflow_runner
    owner: data_platform_lead
    tooling: airflow + dbt
    evidence: completeness_check_result_json + active_drivers_reference_snapshot → S3 object lock 7y

  - id: CTL-CDE-SWR-003-008
    objective_id: OBJ-CDE-SWR-003-03
    title: "Late-event adjustment journal"
    type: detective-manual-business
    activity: "Weekly Data Quality engineer reviews swiftride_prod.recon_adjustments table; latency >24h events flagged; root cause investigation; AML Compliance Lead notified if pattern suggests upstream system issue."
    frequency: weekly
    actor: dq_engineer
    owner: data_quality_lead
    tooling: snowflake_query + servicenow
    evidence: adjustment_review_signed_off + investigation_ticket_trail → S3 7y

  - id: CTL-CDE-SWR-003-009
    objective_id: OBJ-CDE-SWR-003-03
    title: "Reprocessing DAG — idempotent corrective"
    type: corrective-automated-business
    activity: "Triggered by CTL-007 или CTL-002 fail: Airflow reprocess_driver_earnings(date_range) DAG; idempotent (re-runnable без double-counting); produces restated values + reconciliation post-recompute; publishes restatement event на Kafka."
    frequency: on_demand
    actor: airflow_runner
    owner: data_platform_lead
    tooling: airflow + dbt
    evidence: reprocess_run_log + restated_values + post_recompute_reconciliation → S3 7y

  # OBJ-04 — Timeliness + access
  - id: CTL-CDE-SWR-003-010
    objective_id: OBJ-CDE-SWR-003-04
    title: "Pipeline SLA monitoring"
    type: detective-automated-business
    activity: "Airflow DAG SLA monitoring: fct_driver_earnings completes by T+1 06:00 UTC; breach → PagerDuty Sev-2 + Datadog timeseries dashboard + monthly trending review by Data Platform Lead."
    frequency: daily
    actor: airflow_sla_monitor + datadog
    owner: data_platform_lead
    tooling: airflow + datadog + pagerduty
    evidence: airflow_run_log + sla_breach_events + pagerduty_incident → S3 7y

  - id: CTL-CDE-SWR-003-011
    objective_id: OBJ-CDE-SWR-003-04
    title: "Snowflake RBAC + JIT access"
    type: preventive-automated-it
    activity: "Snowflake RBAC: ROLE_DATA_PLATFORM_ENGINEER read all, write dev_* only; ROLE_AIRFLOW_RUNNER_PROD writes production CDE schemas (service account only); ACCOUNTADMIN 3 named accounts с Saviynt PIM 2-person JIT approval; access_history → S3 7y."
    frequency: continuous
    actor: snowflake_rbac + saviynt_pim
    owner: security_lead
    tooling: snowflake + okta + saviynt
    evidence: snowflake_access_history + okta_login_logs + saviynt_pim_approvals → S3 7y

  - id: CTL-CDE-SWR-003-012
    objective_id: OBJ-CDE-SWR-003-04
    title: "Quarterly user-access review (UAR)"
    type: detective-manual-it
    activity: "Quarterly Saviynt scheduled campaign — Data Platform Lead reviews accounts с access к fct_driver_earnings; certifies необходимость; revokes stale; exceptions ticketed Jira; signed campaign report archived."
    frequency: quarterly
    actor: data_platform_lead + security_lead
    owner: security_lead
    tooling: saviynt_campaigns
    evidence: signed_uar_report + jira_remediation_tickets → S3 7y

Всего: 12 controls. Coverage:

Тип ячейки кубаCount
Preventive × Automated × IT1
Preventive × Automated × Business3
Preventive × Manual × IT0
Detective × Automated × Business4
Detective × Manual × IT1
Detective × Manual × Business1
Corrective × Automated × Business1

Defense-in-depth — 4 preventive + 6 detective + 1 corrective; mix manual/automated; IT + business mix. Single-cell concentration отсутствует.

Шаг 3 — Define evidence requirements per control per M5.4

Per control row выше — evidence column уже заполнен. Проверяем каждое:

  • Timestamp captured (да, JSON includes).
  • Immutable storage (S3 object lock compliance mode 7y).
  • Signed / authenticated (HMAC-SHA256 для service accounts; OIDC identity для human attestation; gpg-signed commits).
  • Outcome captured (pass / fail с полным state — input values, threshold, computed delta).
  • Retention 7y SOX (multi-regulator — побеждает most-restrictive, все 7y здесь).
  • Queryable (Snowflake audit.evidence_index pointers к S3 keys).

Шаг 4 — Assign owners per M5.7 SoD + M4.4 RACI

Per control owner column заполнен. Валидация SoD:

  • Initiator (CTL-004, CTL-006, CTL-005, CTL-009) — engineer Anna Data Platform.
  • Approver (CTL-004 CODEOWNERS) — Carlos Finance Lead + Priya Data Platform Lead.
  • Executor (CTL-001, CTL-002, CTL-003, CTL-007, CTL-009, CTL-010, CTL-011) — Airflow runner service account (signed code).
  • Monitor (CTL-008, CTL-010, CTL-012) — Yuki DQ rotation + Sven Security Lead.
  • Attestor (квартально per M7) — Sami SwiftPay Steward + CDO Office.

5 distinct individuals across 4 teams (Data Platform, Finance Data Office, Data Quality, Security, SwiftPay business). 4-actor model удовлетворена.

Шаг 5 — Cross-references к risk + regulation + lineage

Каждая строка control ссылается:

  • risk_id (M2 risk register).
  • regulation_refs (M3 regulatory matrix).
  • evidence_storage_pointer (S3 bucket + Snowflake audit.evidence_index).
  • attestation_cadence (M7 attestation cycle).

Шаг 6 — Self-check

Запускаем критерии ниже; документируем gaps + remediation plan; submit с template.

Критерии self-check

Completeness

  • Минимум 12 controls defined (цель 12-16 для high-criticality CDE).
  • 1-3 control objectives покрывающих все material risks.
  • 1-2 control activities per objective (всего минимум 12).
  • Каждый control имеет: objective_id, activity (verb-first + frequency + actor + outcome chain), type (3-axis cube cell), owner, evidence requirement, regulation_refs.
  • Defense-in-depth: минимум 1 preventive + 1 detective + 1 corrective.
  • Coverage всех 6 DQ dimensions (completeness, accuracy, consistency, timeliness, uniqueness, validity) либо задокументированный gap rationale.

Defensibility

  • Можно walk Big 4 reviewer через каждый control: objective → activity → evidence?
  • Можно defend rationale defense-in-depth (что НЕ покрыто single-control failure)?
  • Можно cite специфическое regulatory requirement per objective?
  • Можно demonstrate SoD 4-actor model для material changes?
  • Можно реконструировать evidence trail 7 лет назад?

Cross-references

  • Risk register refs (M2.6) — каждое objective links к ≥1 идентифицированному risk.
  • Regulation refs (M3 + M4.5 registry entries) — каждое objective links к ≥1 regulatory citation.
  • Lineage refs (M4.2 + M5.8) — schema migration controls reference Marquez + OpenLineage events.
  • Forward refs к M7 evidence pipeline + M8 operating cadence.

Tooling realism

  • Tools cited match SwiftRide T+9M state (M4.6 catalog).
  • Никакие vendor-specific magical features не предполагаются.
  • Costs (audit-test cost + operational cost) учтены реалистично.

Opt-in tooling lab — dbt + GX Core 1.17.1 stub

Для мотивированных студентов. Пропустить, если doc-centric output достаточен.

dbt + GX Core stubvdbt-core 1.9.x + GX Core 1.17.12026-05

Mini dbt project + GE Core stub:

Setup

mkdir swiftride-controls-lab && cd swiftride-controls-lab
python -m venv venv && source venv/bin/activate
pip install dbt-core==1.9.* dbt-snowflake==1.9.* great-expectations==1.17.1
dbt init swiftride_controls
cd swiftride_controls

dbt model + contract — fct_driver_earnings

-- models/marts/driver_earnings/fct_driver_earnings.sql
{{ config(
    materialized='table',
    contract={"enforced": true}
) }}

SELECT
  driver_id,
  payout_date,
  SUM(fare_usd * (1.0 - commission_pct)) AS gross_earnings_usd,
  CURRENT_TIMESTAMP() AS computed_at
FROM {{ ref('stg_trips') }} t
JOIN {{ ref('commission_rules') }} cr
  ON t.country_code = cr.country_code
  AND t.payout_date BETWEEN cr.effective_from AND cr.effective_to
GROUP BY driver_id, payout_date
# models/marts/driver_earnings/_schema.yml
version: 2
models:
  - name: fct_driver_earnings
    config:
      contract:
        enforced: true
    columns:
      - name: driver_id
        data_type: VARCHAR
        constraints:
          - type: not_null
      - name: payout_date
        data_type: DATE
        constraints:
          - type: not_null
      - name: gross_earnings_usd
        data_type: NUMERIC(18,4)
        constraints:
          - type: not_null
        tests:
          - dbt_expectations.expect_column_values_to_be_between:
              min_value: 0
              max_value: 50000
      - name: computed_at
        data_type: TIMESTAMP

GE Core 1.17.1 expectation suite

# ge/expectations/driver_earnings_suite.py
import great_expectations as gx

context = gx.get_context()
asset = context.sources.add_snowflake(
    "swiftride_prod",
    connection_string="snowflake://..."
).add_table_asset("fct_driver_earnings", "fct_driver_earnings")

batch_request = asset.build_batch_request()
context.add_or_update_expectation_suite("driver_earnings_suite")

validator = context.get_validator(
    batch_request=batch_request,
    expectation_suite_name="driver_earnings_suite"
)

validator.expect_column_values_to_not_be_null("driver_id")
validator.expect_column_values_to_not_be_null("payout_date")
validator.expect_column_values_to_be_between("gross_earnings_usd", min_value=0, max_value=50000)
validator.expect_column_value_lengths_to_be_between("driver_id", min_value=8, max_value=64)
validator.expect_column_values_to_not_be_null("gross_earnings_usd", mostly=0.9999)

validator.save_expectation_suite()
checkpoint = context.add_or_update_checkpoint(
    name="driver_earnings_checkpoint",
    validations=[{"batch_request": batch_request, "expectation_suite_name": "driver_earnings_suite"}],
)
checkpoint.run()

Run → Data Docs HTML generated → архив к S3 object lock (s3://swiftride-controls-evidence/cde-swr-003/ge/) 7y retention.

dbt audit_helper reconciliation stub

-- analyses/recon_snowflake_vs_aurora.sql
{{ audit_helper.compare_relations(
    a_relation=ref('fct_driver_earnings'),
    b_relation=source('aurora_swiftpay', 'payouts'),
    primary_key='payout_id',
    columns=['driver_id', 'gross_earnings_usd', 'payout_date'],
    summarize=true
) }}

dbt compile + run; output → target/compiled/... архив evidence.

Cleanup

deactivate && rm -rf venv

Tooling lab демонстрирует 3 controls live (CTL-001 GE expectation suite, CTL-002 reconciliation, CTL-006 dbt contract). Doc-centric lab покрывает все 12.

Submission template

# M5 Lab — SwiftRide CDE-SWR-003 Controls Matrix

**Студент:** [ваше имя]
**Дата:** [дата]
**Метод:** Doc-centric (обязательный) + opt-in dbt+GX Core tooling [yes/no]

## Inputs used
- CDE-SWR-003 registry entry from M4.8 lab
- Risk register entries R-DE-001 through R-DE-005 from M2.7 lab
- Regulatory matrix cell-level from M3.10

## Output
- Минимум 12 controls (цель 12-16) defined в YAML format
- 1-3 control objectives + 1-2 activities per objective
- Каждый control с: objective_id, activity, type (3-axis cube), owner, evidence, regulation_refs

## Coverage analysis
| Тип ячейки куба | Count |
|---|---|
| Preventive × Automated × IT | [N] |
| Preventive × Automated × Business | [N] |
| ... (8 more cells)

## Defense-in-depth
- Preventive total: [N]
- Detective total: [N]
- Corrective total: [N]
- Coverage 6 DQ dimensions: [completeness yes/no | accuracy ... | uniqueness ...]

## Валидация SoD
- Initiator: [name + team]
- Approver: [name + team]
- Executor: [actor + team]
- Monitor: [name + team]
- Attestor: [name + team]

## Self-check verdict
- Completeness: [pass / partial / gaps]
- Defensibility: [pass / partial / gaps]
- Cross-references: [pass / partial / gaps]
- Tooling realism: [pass / partial / gaps]

## Gaps + remediation plan
| Gap | Remediation | Owner | Target |
|---|---|---|---|
| [example] DQ uniqueness dimension не covered | Add CTL-XXX uniqueness check на driver_id × payout_date | Data Quality Lead | T+1M |

## Forward dependencies
- M6 BIA: dependency на BIA mapping SwiftPay wallet (SwiftPay → driver_earnings → IRS export)
- M7 evidence pipeline: каждое evidence column в матрице — target M7 pipeline
- M8 operating cadence: квартальная attestation Q3-Q4 включает CDE-SWR-003

## Lessons learned
[Рефлексия — какие аспекты теории M5.1-M5.8 translated в difficulty in lab; что adjusted]

Типичные ошибки

Generic activity «daily reconciliation»

Симптом: activity column читается как «daily reconciliation» без specifics.

Почему плохо: auditor не может verify control execution; threshold, outcome chain, fail handling все неясны.

Fix: verb-first + frequency + actor + criteria + outcome chain. Пример: «Daily Airflow DAG T+1 06:00 UTC: reconcile snowflake.fct_driver_earnings vs aurora.swiftpay.payouts; sum match delta ≤0.05%; fail → PagerDuty Sev-1 + ServiceNow + reprocess DAG».

Single-cell concentration

Симптом: 10 из 12 controls — Detective × Automated × Business.

Почему плохо: single-cell coverage = single point of failure; over-reliance на detective; PCAOB 2024 spotlight top deficiency.

Fix: распределить по cube cells; defense-in-depth analysis pre-defined минимум 1 preventive + 1 detective + 1 corrective.

Evidence «check passed»

Симптом: evidence column читается «log entry pass/fail».

Почему плохо: не reconstructable; auditor не может verify, что actually controlled.

Fix: structured JSON с timestamp, control_id, input values, threshold, computed delta, version of rule, signed_by, signature_alg. SOX-grade.

Generic regulation refs

Симптом: regulation_refs: «SOX» (просто «SOX»).

Почему плохо: не specific enough; auditor не может trace cell-level applicability per M3.10.

Fix: specific article references. «SOX 404 + PCAOB AS 2201 ¶.47 + BCBS 239 Principle 3 + GDPR Art. 30».

Отсутствие валидации SoD

Симптом: owner column все = «Data Engineering team».

Почему плохо: SoD violation; нельзя demonstrate distinct individuals per role.

Fix: specific named roles (Data Platform Lead, Finance Lead, Data Quality Lead, Security Lead, Data Owner) per M5.7 4-actor model.

Нет corrective controls

Симптом: 12 controls, все preventive + detective; нет corrective.

Почему плохо: когда detective fires, что происходит дальше? Без corrective restoration ad-hoc.

Fix: минимум 1 corrective control (например, идемпотентный reprocessing DAG, restatement runbook) per material CDE.

Evidence в mutable storage

Симптом: «evidence stored в Snowflake table».

Почему плохо: mutable; DBA modify; не SOX-grade primary evidence.

Fix: S3 object lock compliance mode primary; Snowflake table только query index pointers.

Generic refs lineage

Симптом: «lineage tracked».

Почему плохо: не implementable; auditor asks «где» — нет ответа.

Fix: specific OpenLineage URI + Marquez run reference; nightly export к S3 для retention.

Quarterly UAR без evidence

Симптом: «UAR квартально».

Почему плохо: нет signed report, нет remediation tickets, нет архива.

Fix: Saviynt campaign + signed report + Jira tickets + S3 archive 7y.

Забыли map к M6 BIA + M7 evidence + M8 operating

Симптом: controls list без forward dependencies.

Почему плохо: integration с downstream modules missing; controls catalog не actionable.

Fix: forward refs к M6 (BIA dependency mapping), M7 (target pipeline сбора evidence), M8 (operating cadence + attestation cycle).

Bridge к M7 + M8

Output этого lab → M7 (evidence + attestation):

  • Каждое evidence column в матрице — target M7 pipeline;
  • 12 controls × evidence streams → ~12 evidence pipelines к build;
  • Квартальная attestation cycle aggregates evidence per CDE-SWR-003 для Sami (Steward) + CDO Office sign-off.

Output этого lab → M8 (operating model):

  • Embedded SDLC gates (CTL-004 CODEOWNERS, CTL-005 lineage-guarded, CTL-006 dbt contract) — первое место в SDLC, где controls fire;
  • Quarterly UAR (CTL-012) — M8 operating cadence;
  • Maturity Level 3 embedded SDLC = controls catalog operational без daily intervention CDO Office.

Output этого lab → M9 capstone:

  • Synthesis across M5-M8 controls + evidence + attestation; цель — Big 4 external audit unqualified opinion SwiftRide T+18M.

Резюме

  • Цель output lab: 12-16 controls для CDE-SWR-003 driver_earnings_ledger; 4 control objectives × ~3 activities each; defense-in-depth (минимум 4 preventive + минимум 6 detective + минимум 1 corrective).
  • Каждый control: objective_id, activity (verb-first + frequency + actor + outcome chain), 3-axis cube cell, owner (M5.7 SoD validation), evidence requirement (S3 object lock 7y), regulation_refs (specific articles), cross-refs (risk register + lineage).
  • 12 controls в шаблоне synthesise M5.1-M5.8: control taxonomy (M5.1) ✓; ITGC (M5.2 — CTL-011 RBAC, CTL-012 UAR, CTL-004 CODEOWNERS) ✓; application controls (M5.3 — CTL-001 GE, CTL-007 completeness) ✓; objective/activity/evidence (M5.4) ✓; DQ dimensions (M5.5 — multiple cells) ✓; reconciliation (M5.6 — CTL-002) ✓; SoD (M5.7) ✓; lineage-as-control (M5.8 — CTL-005) ✓.
  • Типичные ошибки — generic activities, single-cell concentration, evidence «pass/fail», generic regulation refs, отсутствие валидации SoD, нет corrective control, evidence в mutable storage, generic refs lineage, UAR без evidence, забыли forward dependencies.
  • Bridge: M7 (evidence pipelines), M8 (operating cadence), M9 capstone (цель Big 4 audit unqualified opinion).
  • Lab — центральный артефакт M5; без controls matrix baseline M7 evidence pipelines не могут anchor, M8 operating cadence не может calendar.

Следующий модуль (M6): Business Impact Analysis (BIA) + BCP / DRP — для CDE programme. M6 строит BIA mapping для SwiftPay wallet (overlaps CDE-SWR-003 driver_earnings + CDE-SWR-005 aml_alerts); recovery objectives RPO/RTO per CDE; resilience design per DORA.

Great Expectations — автоматизация control evidence KPIs эффективности DG-программы

Проверьте понимание

Результат: 0 из 0
Аналитический
Вопрос 1 из 4. Student submits M5.9 lab — controls matrix для CDE-SWR-003 с 14 controls. Coverage breakdown: 12 detective × automated × business + 1 preventive × automated × IT + 1 detective × manual × IT. Defense-in-depth gaps?

Закончили урок?

Отметьте его как пройденный, чтобы отслеживать свой прогресс

Войдите чтобы оценить урок

Прогресс модуля
0 из 9