Data Engineer: Appendix
Glossary
We’re striving to use clear language, but there are a few words where it’s helpful to expound upon how we interpret them.
Craft Responsibility Definitions
The responsibilities defined below make up the foundation of the Data Engineer craft:
- Code fluency: The ability to read and write code fluently and well. Code for a data engineer is both software (Python, etc) and SQL
- Data and Software design: The ability to design software components with reasonable APIs and interaction patterns (writing good classes, modules, etc - building out a box in your architecture diagram) as well as intuitive data models which abstract the complexity of upstream data sources at an appropriate level of optimization for data user needs
- Architecture design: The ability to design systems of interacting components - e.g. a collection of interacting features or the architecture of a product, binary or significant service (what are the boxes in your architecture diagram, how do they interact). This is inclusive of the architecture of a data model and the data pipelines required to produce that data model
- Technical strategy: Ensuring the right long-term technical and data decisions are being made by the organization - knowing what systems and data models to build; making technical choices when there are not clear solutions. What standards apply to all the boxes in the organizations architecture diagrams?