Skip to content
Michaela Lawrence edited this page Jul 11, 2024 · 4 revisions

Welcome to the DRAFT data engineering learning roadmap homepage

The pathway is currently still being developed and is still missing lots of learning resources. Any questions can be directed to the Head of Profession for Data Engineering and head of the Data Engineering Community of practice PUBLIC TEST. Or the capability team in DGO, DALI, specifically PUBLIC TEST.

These courses are just suggestions you don’t have to learn things from these resources. If you have alternatives that you would like to share please add them.

How to use this roadmap

Remember the 10:20:70 split

  • 10% Formal training e.g. structured learning, eLearning, classes.
  • 20% Relationships e.g. Coaching, feedback, performance discussions
  • 70% Job experience

The roadmap is structured according to the GDD role description for a data engineer. You first select a skill and then you can look through the different levels of the skill. The different levels of expertise:

  • 0 = No knowledge/Skills
  • 1 = Awareness: you can describe the fundamentals of the skill, and demonstrate basic knowledge of some of the skill's tools and techniques
  • 2 = Working: You can apply the skill with some support, and adopt the most appropriate tools and techniques
  • 3 = Practitioner: You can apply the skill without support, determine and use the most appropriate tools and techniques, and share knowledge and experience of the skill
  • 4 = Expert: You can: lead and guide a team or organisation in the skill's best practice, and teach the skill's advanced tools and techniques

It is not enough to just do the training courses - to be an expert for example, you would have to have led a team to complete a project where that skill was used. Working knowledge would require you to have used the skill in a work project.

Platforms

Some info here about specialising to a particular platform

Keeping a CDP log

some info here

Essential Skills

The list of essential skills for any data engineer

  • Communicating between the technical and non-technical
  • Data analysis and synthesis
  • Data development process
  • Data innovation
  • Data integration design
  • Data modelling
  • Metadata management
  • Problem resolution (data)
  • Programming and build (data engineering)
  • Technical understanding
  • Testing
  • Ways of working e.g. TDD, Agile etc (not included in the original GDD framework)

Behaviours

As well as skills demonstrating behaviours at appropriate levels is an essential part of progression, so we include descriptions and expectations in this learning roadmap

  • Making effective decisions
  • Working Together
  • Changing and improving
  • Delivering at Pace
  • Leadership
  • Communicating and Influencing