Data engineering has grown rapidly in the past decade, leaving many
software engineers, data scientists, and analysts looking for a
comprehensive view of this practice. With this practical book, you'll
learn how to plan and build systems to serve the needs of your
organization and customers by evaluating the best technologies available
through the framework of the data engineering lifecycle.
Authors Joe Reis and Matt Housley walk you through the data engineering
lifecycle and show you how to stitch together a variety of cloud
technologies to serve the needs of downstream data consumers. You'll
understand how to apply the concepts of data generation, ingestion,
orchestration, transformation, storage, and governance that are critical
in any data environment regardless of the underlying technology.
This book will help you:
Get a concise overview of the entire data engineering landscape
Assess data engineering problems using an end-to-end framework of best
practices
Cut through marketing hype when choosing data technologies,
architecture, and processes
Use the data engineering lifecycle to design and build a robust
architecture
Incorporate data governance and security across the data engineering
lifecycle