Senior Data Process Engineer

Job Purpose

We are seeking a Senior Data Engineer to own and evolve our data platforms, driving the design and delivery of scalable, cloud-native data solutions. This role will oversee the development and optimisation of automated data workflows and reporting systems across the business, ensuring the reliability and growth of our data infrastructure.

The successful candidate will combine deep technical expertise in Python, SQL, and modern cloud technologies — particularly Microsoft Fabric — with leadership skills to mentor and guide other engineers. They will play a pivotal role in migrating legacy KNIME workflows into modern architectures, building robust lakehouse environments, and establishing best practices in automation and orchestration.

This is a strategic, hands-on role that blends technical leadership with platform ownership. It offers significant autonomy, direct impact on business-critical functions, and the opportunity to shape the future of data engineering within the organisation.

Main Duties and Responsibilities

• Own the design, build, and optimisation of cloud-based data lakehouse environments to support analytics and reporting at scale
• Lead the development and maintenance of Python-based ETL/ELT pipelines to monitor systems, ensure data quality, and proactively flag issues
• Oversee the design and management of Microsoft Fabric solutions, including Lakehouses, Pipelines, and Dataflows
• Translate high-level business requirements into scalable, actionable workflows and reporting solutions, working closely with Finance and other key stakeholders
• Drive the migration of legacy processes into modern Python and SQL-based architectures
• Establish and own automation/orchestration frameworks using tools such as Dagster, Apache Airflow, Prefect, or Azure Data Factory
• Coach, mentor, and guide junior data engineers, ensuring best practices, knowledge sharing, and skills development
• Take ownership of data engineering roadmaps, ensuring delivery, scalability, and alignment with business needs

Essential Skills

• Advanced Python skills for data transformation (e.g., pandas, polars, SQLAlchemy)
• Strong SQL expertise across varied data sources
• Proven track record in building, deploying, and maintaining complex ETL/ELT workflows at scale
• Experience designing and managing data workflows in financial reporting or other data-intensive contexts
• Demonstrated ability to lead projects, manage priorities, and provide technical guidance to team members
• Proactive ownership mindset with the ability to thrive in a fast-paced, evolving environment

Desirable Skills

• Hands-on experience with Microsoft Fabric (Lakehouses, Pipelines, Dataflows) and exposure to data lake architecture design
• Familiarity with orchestration tools (e.g., Dagster, Airflow, Prefect, Azure Data Factory)
• Exposure to containerisation and DevOps practices (e.g., Docker, Git, Bitbucket)
• Exposure to event streaming platforms like Kafka
• Experience with version control best practices (git, Bitbucket, CI/CD)
• Experience building RESTful APIs (FastAPI, Flask, Django)
• Experience leading migrations from legacy tools (e.g., KNIME, SAS) to modern, cloud-native solutions

Education & Experience

• Bachelor's or Master's degree in Engineering, Computer Science, Mathematics/Statistics, or equivalent industry experience
• Substantial experience (5+ years) in data engineering, data architecture, or similar roles, ideally with exposure to financial or data-intensive industries

Personal Qualities

• Can-do attitude — prepared to "go the extra mile"
• Lots of energy — a team player who can work off their own initiative
• Heightened sense of urgency
• Curious learner with an interest in professional development
• Actively demonstrates a passion for the business
• Strong leadership skills, with the ability to coach, mentor, and develop junior data engineers
• Excellent communication and collaboration skills, able to translate complex technical concepts for non-technical stakeholders and influence senior decision-makers
• Strategic mindset with the ability to balance hands-on delivery and long-term platform ownership
• Ambassador of corporate values