Lead Data Engineer

Licensed sponsor London Contract Posted 1 day ago

Role: Lead Data Engineer (Snowflake)

Location: London, UK

Job Type : Open for Contract (6 months)/FTC (6 months)/Perm

Work Mode: Hybrid (4 days per week from office)

Start Date: Only Immediate Joiners or candidates with max 2 weeks’ notice.

About the Role

We are looking for a Senior / Lead Data Engineer to join the engineering organisation a multi-year banking transformation programme delivering capabilities across Accounts (Real-Time Ledger), Payments, Foreign Exchange and Client Lifecycle Management, underpinned by an Enterprise Data Platform built on Snowflake.

This is a hands-on build role. Working closely with the Enterprise Data Architect, you will turn the target-state data architecture into a production platform: standing up the platform foundations, engineering data acquisition and transformation, and implementing canonical data models and data products. You will act as the engineering counterpart to the Data Architect – implementing the design, feeding back on buildability, and setting the engineering standards others follow. At Lead level, you will also shape and lead the data engineering team.

Experience & Profile

Senior: 10+ years in data engineering, including at least 3 years of hands-on Snowflake delivery in production. Lead: 12+ years, including leading a data engineering team or workstream
Strong track record in data warehousing and data modelling, with experience implementing models designed by architects as well as designing their own
Has stood up a new data platform or major platform workstream – environments, access control, automation – rather than only extending an existing one
Has built a dbt project from the ground up, including standards, testing and CI/CD
Has designed and built a metadata-driven ingestion framework that has been used to onboard multiple sources in production
Strong hands-on experience scheduling and orchestrating production pipelines natively in Snowflake with Tasks / task graphs and Dynamic Tables
Has designed and implemented data acquisition across heterogeneous sources, and can clearly explain the trade-offs between full, incremental and CDC approaches
Comfortable reading and implementing architecture artefacts – canonical models, mapping specifications, ADRs – and offering well-reasoned implementation feedback
Clear communicator, able to work with architects, application teams and business stakeholders

Primary Skills (Essential)

Candidates are expected to have strong, hands-on depth in all of the following:

SQL

Advanced, production-grade SQL – complex joins, window functions, set-based transformation, MERGE / upsert patterns, query profiling and performance tuning

Data Warehousing

Solid grounding in warehouse principles – layered / medallion architectures, ELT, incremental processing, slowly changing dimensions, history and audit, reconciliation, partitioning and performance

Data Modelling

Dimensional modelling (Kimball – facts, dimensions, conformed dimensions); implementing canonical / enterprise data models; normalised vs denormalised trade-offs; SCD Types 1/2; working from logical models and mapping specifications

Snowflake

Hands-on platform engineering – schema and layer design, RBAC, virtual warehouses, Snowpipe, Streams & Tasks, zero-copy cloning, Time Travel, masking and row access policies, resource monitors, cost optimisation

dbt

Building dbt projects from scratch – layering, incremental models, snapshots, macros, packages, tests, docs; dbt Projects on Snowflake and dbt Cloud

Data Acquisition & CDC

Designing full, incremental and CDC-based acquisition; log-based CDC tooling (e.g. Fivetran / HVR, Debezium, Kafka Connect, Snowflake Openflow); batch, micro-batch and event-driven ingestion

Managed ELT

Fivetran or comparable – connector configuration, schema drift, history mode, re-syncs, consumption cost management

Metadata-driven Ingestion

Designing and building configuration-led ingestion frameworks – metadata / control tables, parameterised and reusable load patterns, watermarking, audit, restartability and reprocessing

Snowflake Orchestration

Snowflake Tasks and task graphs (DAGs), Streams & Tasks patterns, Dynamic Tables (target lag, incremental vs full refresh), serverless tasks, alerts and notifications, monitoring of task history and cost

Engineering Practice

Python for data engineering and automation; Git; CI/CD (e.g. GitHub Actions,); infrastructure-as-code (e.g. Terraform, schemachange); automated testing

Snowflake AI

Good working knowledge of Snowflake CoCo (formerly Cortex Code) and Snowflake Cortex AI, and a disciplined approach to AI-assisted development in a regulated environment