Principal Data Engineer
The advert
As posted by Digital FuturesAbout Digital Futures
Digital Futures is an AI services and software company that specialises in accelerating AI transformation and enabling AI governance for enterprise and government.
The Role
Digital Futures is supporting a large, regulated organisation to build and operate trusted, scalable data assets that enable priority business use cases across its enterprise data platform. This is a hands-on engineering role, moving use cases from requirements into production-ready, reusable and governed data assets.
Working as part of a multidisciplinary team of data analysts, engineers and data scientists, you will design, build, test, deploy and support scalable data solutions, turning agreed requirements into robust data pipelines, curated datasets and data products. You will work closely with business stakeholders, product and delivery leads, data owners, architects and platform teams and, as a principal-level engineer, you will shape technical designs and set engineering standards for the team.
The role combines strong big-data and cloud engineering capability on GCP or AWS with a practical understanding of data modelling, data quality, Agile delivery and production-ready DevOps practices.
Key Responsibilities
Data engineering and solution delivery
- Design and build reliable, scalable and maintainable data pipelines, data products and processing solutions, leading solution designs that are secure, observable, reusable and aligned to enterprise standards.
- Translate requirements, source-to-target mappings and acceptance criteria into technical designs and production-ready implementations across batch and, where relevant, event-driven patterns.
- Support the full lifecycle of data assets, including development, testing, deployment, monitoring, incident resolution and continuous improvement.
Big-data and cloud platform engineering
- Build and maintain distributed data-processing solutions using Hadoop, Hive, HDFS, Apache Spark and Scala or Python.
- Develop and deploy data solutions on GCP (such as BigQuery, Dataflow, Dataproc, Pub/Sub and Cloud Composer) or AWS (such as Redshift, Glue, EMR, Kinesis and Managed Workflows for Apache Airflow), including migrating and modernising workloads onto the cloud.
- Select processing, storage and orchestration patterns based on data volume, latency, reliability, cost, security and operational requirements.
Data modelling, quality and controls
- Apply relational and non-relational data-modelling techniques, creating schemas, tables, views and partitions that support performance, usability and maintainability.
- Implement data-quality checks covering completeness, validity, uniqueness, consistency, timeliness, reconciliation and referential integrity.
- Ensure lineage, ownership, access, privacy, retention and permitted-use considerations are reflected and documented in technical delivery.
DevOps, testing and production readiness
- Implement automated unit, integration and data-quality testing for data-processing logic and platform components.
- Contribute to automated build, deployment and release processes using Git or GitHub and tools such as Jenkins and Ansible, applying peer review, environment promotion and rollback practices.
- Monitor pipeline health, data freshness and performance in production, maintaining runbooks and support procedures, and investigating code and data issues to identify and communicate root causes.
Agile collaboration and delivery
- Work within Scrum, Kanban or a comparable Agile model, using Jira to manage stories, dependencies, defects and delivery status.
- Break complex engineering work into deliverable increments, providing realistic estimates and technical sequencing and managing technical risks.
- Collaborate with data analysts, data scientists, delivery leads and business stakeholders, mentoring engineers and contributing to a collaborative engineering culture across global teams.
Experience and Profile
- Extensive experience in data engineering, big-data engineering, cloud data engineering or a related role, typically 5+ years.
- Strong hands-on experience with Hadoop, Hive, HDFS, Apache Spark, Scala, SQL and Airflow.
- Hands-on experience with data migration and processing on GCP or AWS, using services such as BigQuery, Dataflow, Dataproc and Pub/Sub, or Redshift, Glue, EMR and Kinesis.
- Proven ability to design and deploy large-scale distributed data-processing systems, defining architecturally sound solution designs and understanding the logical flow of solutions at code level.
- Experience delivering automated unit testing, integration testing and automated build and deployment processes, using Git or GitHub, Jenkins, Ansible and Jira.
- Understanding of big-data modelling using relational and non-relational approaches, including PostgreSQL or equivalent databases and enterprise data warehouses.
- Experience debugging code and data issues, analysing differences and communicating root causes and remediation options.
- Understanding of cloud design patterns, cloud security principles, observability and production support.
- Strong interpersonal skills, with the ability to work effectively with senior business stakeholders in team-based and global environments, and a proactive, learning mindset.
- Experience in financial services or another regulated environment, and a GCP or AWS certification, is desirable.
Benefits
We offer a competitive salary. Our benefits package is designed to support your wellbeing and development. This includes:
- Group Life Assurance
- Virtual GP and Online Physiotherapy
- Confidential Helpline/Employee Assistance Programme
- Wellbeing Studio & Digital Gym
- Discounted shopping cards, e-vouchers & cashback offers
- Percentage discounts & savings on brands online & in store
- Statutory pension contribution - salary exchange scheme
- 30 days annual leave, including UK Bank Holidays
Why Join Us
Digital Futures consultants deliver solutions across the full data and AI transformation lifecycle, from strategy, analytics and engineering through to adoption, governance and assurance.
We’re focused on creating opportunity in the age of AI. Joining Digital Futures means joining an organisation making a real impact in delivering an AI-first future, at whatever stage of your career.
Sponsor check
Digital Futures holds a Home Office sponsor licence (A-rated · Skilled Worker route), checked against the register published 19 Nov 2026. We read this advert: no sponsorship restrictions. Home Office FOI figures: 6 visa sponsorships in 2021–24.
How we check sponsorshipSponsor check
- Sponsor register
Digital Futures is on the Home Office register of licensed sponsors: A-rated, licensed since at least 2023.
Home Office register · published 19 Nov 2026
- Visa sponsorships
6 visa sponsorships in 2021–24 (Home Office FOI 2025/12841).
- From the advert
The advert doesn't mention sponsorship.
Read from the advert below
Facts from the Home Office register of licensed sponsors and Home Office FOI figures. We don't judge employers; check the advert and the register yourself.
Our estimate from the job title
- Likely occupation
- SOC 2134 · Programmers and software development professionals
- Going rate
- £54,700 a year (£28.05/h)estimate
- Visa route
- Likely eligible for Skilled Worker sponsorship
- Visa salary for this occupation
- £54,700 · new entrants £38,300estimate
- Typical UK pay
- £56,914 median full-time
The employer picks the real occupation code on the Certificate of Sponsorship. If it picks a different one, the visa salary changes. How we read this
Digital Futures
- Sponsor licence
- A-rated · Worker
- Visa sponsorships
- 6 · 2021–24
- Visa route
- Skilled Worker
- Industry
- Technology
- Location
- London
Digital Futures is a licensed UK visa sponsor (A-rated). Home Office figures released under the Freedom of Information Act show Digital Futures assigned 6 Certificates of Sponsorship in 2021–24 and 7 in 2025 for Skilled Worker and Global Business Mobility visas.
Similar sponsor-licensed roles
The closest matches at other employers with active sponsor licences.
Principal AI & Data Engineer EPAM Systems · London Sponsorable from abroad HT Principal Engineer HCL Technologies UK · United Kingdom Sponsorable from abroad
Principal Data Engineer Kingfisher · London Sponsorable from abroad
Principal Data Engineer Ensek · United Kingdom Sponsorable from abroad
Principal Software Engineer Global Payments UK · Belfast Sponsorable from abroad
Principal DevOps Engineer (AWS, Kubernetes, SaaS) Gearset · Cambridge Sponsorable from abroad O Principal Platform Software Engineer Oracle UK · London Sponsorable from abroad
Principal Data Architect Bupa · London Sponsorable from abroad New roles like this
Added in the last 48 hours by other sponsor-licensed employers.
Principal Data Engineer FUNDING CIRCLE · London Sponsorable from abroad DH Principal Infrastructure Engineer, BCDR Darktrace Holdings Limited · Cambridge Sponsorable from abroad
Senior Data Engineer Hack the Box · London Sponsorable from abroad
Senior Software Engineer - Data Engineering Moody's Investors Service · Manchester Sponsorable from abroad IB Data Engineer Investec Bank · United Kingdom Sponsorable from abroad HS Data Engineer HCC Service Company Inc · London Sponsorable from abroad UM Database Administrator Universal Music Operations Ltd · United Kingdom Sponsorable from abroad Roles like this at sponsors
Licensed UK visa sponsors
Principal AI & Data Engineer EPAM Systems · London Sponsorable from abroad HT Principal Engineer HCL Technologies UK · United Kingdom Sponsorable from abroad
Principal Data Engineer Kingfisher · London Sponsorable from abroad