Senior Data Engineer

The Department for Business and Trade (DBT) has a clear mission - to grow the economy. Our role is to help businesses invest, grow and export to create jobs and opportunities right across the country. We do this in three ways. Firstly, we help to build a strong, competitive business environment, where consumers are protected and companies rewarded for treating their employees properly.Secondly, we open international markets and ensure resilient supply chains. This can be through Free Trade Agreements, trade facilitation and multilateral agreements.Finally, we work in partnership with businesses every day, providing advance, finance and deal-making support to those looking to start up, invest, export and grow.The Digital, Data and Technology (DDaT) directorate develops and operates tools and services to support us in this mission.
About the team
To enable the work of the department you will join our Digital Data and Technology (DDaT) team of over 200 colleagues working across the full spectrum of the DDaT specialisms, in a modern, user-centric, agile way. You will join the team that build and operate innovative tools and services such as great.gov.uk that support businesses in the UK and overseas. Our public-facing services are visited millions of times each year and the tools we develop for our staff, across the globe, help them be more efficient in their daily working lives. This is possible due to the state-of-the-art data infrastructure our teams build and maintain. We work on critical national infrastructure that powers trade at the UK border and innovative data projects looking at global supply chains and future ways of trading. We believe in creativity, curiosity, diversity of thought, entrepreneurship, delivery focus and collaboration. We embody the DIT values to be expert, enterprising, engaged and inclusive.
These values have helped us become a high performing team that continuously delivers high quality outputs., As a Senior Data Engineer, you will design, implement, test and operate our data products, pipelines and services. You will work in a cross-functional, agile, team to integrate them into wider systems and business processes., You will be:
+ Lead the design, build and test of data products and services using standard techniques and tools.
+ Lead delivery of data solutions which are scalable, resilient and future proof.
+ Have thorough understanding of common problems in databases, data processes, data products and services. You will understand and be able to apply typical solutions to those problems.
+ Follow internal and industry standards to design, code, test and document complex data solutions.
+ Undertake data profiling and source system analysis including techniques like reverse engineering.
+ Be a key contributor in an Agile and cross-functional team

Person specification
+ Agile methodologies. You can lead work following agile methodologies on data projects.
+ Data development process. You can lead design, build and test of data products based on feeds from multiple systems using a range of different storage technologies and/or access methods. You create repeatable, reusable products.
+ Data integration design. You know how to select and implement the appropriate technologies to deliver resilient, scalable and future-proofed data solutions.
+ Data modelling. You understand the concepts and principles of data modelling and can produce relevant data models across multiple subject areas. You know how to reverse-engineer data models from a live system. You understand industry-recognised data modelling patterns and standards and when to apply them. You can compare and align different data models.
+ Problem resolution (data). You know how to respond to problems in databases, data processes, data products and services as they occur. You can initiate actions, monitor services and identify trends to resolve problems. You can determine the appropriate remedy and assist with implementation of it as well as preventative measures.
+ Programming and build (data engineering). You know how to use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations. You can collaborate with others to review specifications where appropriate.
+ Technical understanding (data engineering). You understand core technical concepts like data normalisation, modelling, performance of data intensive solutions. You understand basic software development concepts.
It is desirable that you have:
+ Communication skills (data). You know about the need to translate technical concepts into non-technical language and understand what communication is required for internal and external stakeholders.
+ Cloud infrastructure. Experience with cloud infrastructure, especially AWS, and infrastructure as code tools, such as Terraform., Offers will be made in merit order based on location preferences. If you pass the bar at interview but are not the highest scoring you will be held on a 12-month reserve list in case a role becomes available. If you are judged a near miss at interview, you may be offered a post at the grade below the one you applied for.
This role requires SC clearance. DBT’s requirement for SC clearance is to have been present in the UK for at least 3 of the last 5 years. Failure to meet this requirement will result in your application being rejected and your offer will be withdrawn.
Checks will also be made against:
+ departmental or company records (personnel files, staff reports, sick leave reports and security records)
+ UK criminal records covering both spent and unspent criminal records
+ your credit and financial history with a credit reference agency
+ security services record
+ location details

While you will have opportunities to contribute to the wider solution to satisfy our user needs, the team will rely on your ability to lead development of data products, to optimise the code, and help with database management. You will ensure internal and industry standards are followed in delivery of reusable solutions. You will help more junior team members to gain your level of expertise, and ensure the teams are working to the standards set by the lead data engineer. You will be confident in using languages like SQL and Python, and have deeper understanding of how they work., At the interview stage for this role, you will be asked to demonstrate relevant Technical Skills and Behaviours from the Success Profiles framework. These are role specific and in line with the DDaT Capability Framework.
Technical Skills
+ Problem resolution (data)
+ Programming and build (data engineering)
+ Technical understanding
+ Testing
Behaviours
+ Managing a Quality Service
+ Communicating and Influencing
+ Developing Self and Others

+ Learning and development tailored to your role
+ An environment with flexible working options
+ A culture encouraging inclusion and diversity
+ A Civil Service pension with an employer contribution of 28.97%, This role can only be worked from within the UK, not overseas. If you are based in London, you will receive London weighting. DBT employees work in a hybrid pattern, spending 2-3 days a week (pro rata) in the office on average. Travel to your primary office location will not be paid for by DBT, but costs for travel to an office which is not your main location will be covered.
You can find out more about our office locations, how we calculate salaries, our diversity statement and reasonable adjustments, the Recruitment Principles, the Civil Service code and our complaints procedure on our website.
Find out more about life at DBT, our benefits and meet the team by watching our video or reading our blog!
Feedback will only be provided if you attend an interview or assessment.

Security
Successful candidates must undergo a criminal record check.
Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).See our vetting charter (opens in a new window).
People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements
This job is broadly open to the following groups:
+ UK nationals
+ nationals of the Republic of Ireland
+ nationals of Commonwealth countries who have the right to work in the UK
+ nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
+ nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
+ individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
+ Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service
Further information on nationality requirements (opens in a new window)

Working for the Civil Service
The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.
The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.