Lead Data Engineer

Apply for this job now

Job Description

Overview

The role will lead agile data engineering activities as a technical leader. You will take ownership of the implementation and operation of APIs and Integrations, to automate data transfers between systems and the aggregation of data in the RSSB data platform. You will understand the business and IT strategy to align technical solutions with outcomes.

Responsibilities

  • Lead complex data engineering projects as a technical leader.
  • Influence the information strategy and platform architecture to align technical solutions with outcomes.
  • Influence new Data Engineering tools, frameworks, standards, and processes to improve the productivity and quality of outputs for Data Engineers across the team.
  • Initiate the development and management of scalable RESTful APIs, machine learning endpoints and data integration pipelines.
  • Ensure that agreed standards and tools are applied to achieve well-engineered outcomes, and that own work and that of colleagues meets coding standards, through code review processes.
  • Ensure technical solutions support the accessibility, retrievability, security, quality, retention, and ethical handling of specific sets of data.
  • Design, code, test and document complex data engineering scripts and applications, applying software engineering best practices.
  • Develop both quick prototypes and high-quality products, judging which is needed.
  • Coach Data Engineers and undertake code reviews to ensure the team can continue to deliver sustainable solutions.

Qualifications

  • Experience of strategies and design approaches for APIs and data pipelines.
  • Experience developing scalable APIs in Python using frameworks such as Fast API or Flask.
  • Experience developing scalable data pipelines in Spark/Python using orchestrators such as Airflow.
  • Experience using SQL, Python, and other scripting/programming to solve a wide array of data challenges.
  • Experience working with a diverse range of data infrastructure and systems, such as PostgreSQL; SQL Server; Azure data lakes; Data Bricks; Kubernetes.
  • Effective analysis and documentation skills, with the ability to communicate complex technical information to both technical and non-technical audiences.
  • Effective collaboration skills to work with multiple teams across the organization
  • Experience exercising substantial personal responsibility and autonomy
  • Effective organisational skills and the ability to work independently and with others.
  • Ability to coach and mentor colleagues.

Early applications are encouraged, as we may close this vacancy before the advertised deadline if sufficient numbers of applications are received.

We value diversity and equal opportunities in employment and are committed to creating a workplace which is inclusive to everyone. As a member of the Disability Confident Scheme, we encourage candidates with disabilities who meet the minimum criteria, to apply for our jobs. If you have applied under the Disability Confident Scheme, please let us know in advance by emailing vacancies@RSSB.CO.UK

If you require any reasonable adjustments with respect to our selection process including information in an alternative format, please contact us at vacancies@rssb.co.uk

We understand the importance of work-life balance and we offer our staff the flexibility to work within our core hours and the option to vary their location between both the office and home. If you are looking for further flexibility, speak to us at interview stage so that we can consider your request.

We value our staff and we offer a competitive benefits package to ensure our staff can achieve their best throughout their journey with us. This includes 30 days annual leave (plus bank holidays); a holiday buy and sell scheme; private medical and dental cover; a season ticket loan and travel subsidy; access to a cycle to work scheme; volunteer leave; a performance related bonus and pension.