Big Data Developer

Job Title: Junior Big Data Developer

Location: UK

Experience: 2-5 years

Education: Bachelors in Science (IT/Computer Science/Engineer)

Employment Type: Full-Time

UK based candidates only.

Job Summary

We are looking for a skilled Data Engineer to design, develop, and maintain scalable data pipelines and data infrastructure. The ideal candidate will have experience in data integration, ETL/ELT processes, data warehousing, and cloud-based data platforms to support analytics and business intelligence.

Key Responsibilities

• Design, develop, and maintain robust data pipelines for batch and real-time data processing.
• Build and optimize ETL/ELT workflows to extract, transform, and load data from multiple sources.
• Develop complex SQL queries and optimize database performance.
• Work with data warehouses, data lakes, and distributed data processing frameworks.
• Implement data quality checks, validation, monitoring, and error-handling mechanisms.
• Collaborate with data analysts, data scientists, BI developers, and business stakeholders.
• Ensure data security, governance, integrity, and compliance with organizational standards.
• Troubleshoot pipeline failures and improve data processing performance.
• Maintain technical documentation for data models, pipelines, and workflows.

Required Skills and Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 2–5 years of experience in data engineering or a related role.
• Strong proficiency in SQL and Python.
• Experience with ETL/ELT tools and data integration frameworks.
• Knowledge of relational and NoSQL databases.
• Experience with data warehousing concepts, dimensional modeling, and database optimization.
• Familiarity with Apache Spark, Apache Airflow, or similar data processing and orchestration tools.
• Experience with at least one cloud platform: AWS, Microsoft Azure, or Google Cloud Platform.
• Understanding of data pipeline monitoring, testing, and performance optimization.
• Strong analytical, problem-solving, and communication skills.

Preferred Qualifications

• Experience with Snowflake, Databricks, Amazon Redshift, Google BigQuery, or Azure Synapse Analytics.
• Knowledge of Kafka or other streaming technologies.
• Familiarity with Docker, Kubernetes, Git, and CI/CD pipelines.
• Experience with data lakehouse architecture and cloud-native data solutions.
• Knowledge of data governance, metadata management, and data cataloging.