Data Engineer

Posted: 2026-09-22

Data Engineer

Job requirements Key Responsibilities Logic Discovery and Requirements Analysis

- Work with pricing analysts to identify custom definitions, calculations, transformations, dependencies, assumptions, edge cases, and ambiguities currently implemented in the Gold layer.

- Reverse-engineer analyst-written SQL, spreadsheet logic, and ad hoc calculations to define intended business behaviour.

- Translate legacy logic into clear technical requirements and migration plans.

Migration to the Silver Layer

- Re-implement approved pricing definitions as governed, reusable, documented dbt models in the Silver layer.

- Follow data modelling, naming, testing, version control, and code review standards.

- Build maintainable, scalable, auditable models independent of undocumented analyst knowledge.

- Apply dbt tests for uniqueness, not-null, accepted values, relationships, and business rules.

Parity Validation and Quality Assurance

- Compare migrated Silver-layer results with existing Gold-layer outputs and resolve differences.

- Validate results across representative periods, segments, boundary conditions, and known exceptions.

- Confirm migrations only after parity review and pricing analyst approval.

Analyst Collaboration and Sign-Off

- Partner with pricing analysts to clarify calculations, confirm intended behaviour, and align on expected results.

- Lead walkthroughs and reviews; document analyst approval before retiring legacy definitions.

- Communicate risks, open questions, dependencies, and decisions to technical and business stakeholders.

Documentation and Lineage

- Document business meaning, calculation rules, assumptions, exceptions, source inputs, ownership, lineage, tests, and downstream consumers in dbt and Confluence.

- Ensure documentation enables future engineers and analysts to maintain logic without relying on the original analyst.

Gold-Layer Cleanup and Cutover

- Decommission ad hoc Gold-layer logic after the Silver replacement is validated, approved, adopted, and downstream dependencies are updated.