Modern Data Engineering with Databricks
Build practical skills to design, develop and operate modern Data Engineering solutions using Databricks, Apache Spark, Delta Lake and Lakehouse architecture.
About this course
Course overview: Build practical skills to design, develop and operate modern Data Engineering solutions using Databricks, Apache Spark, Delta Lake and Lakehouse architecture. Module 1 — Modern Data Engineering Foundations Understand the modern Data Engineering ecosystem and responsibilities. Compare traditional Data Warehouses, Data Lakes and Lakehouse architectures. Understand batch, streaming, ETL and ELT patterns. Module 2 — Databricks & Lakehouse Fundamentals Understand the Databricks platform, workspaces, notebooks and compute. Learn the core principles of Lakehouse architecture. Explore how data, engineering, analytics and AI workloads come together. Module 3 — Apache Spark & PySpark Understand Spark architecture and distributed data processing. Work with DataFrames, transformations, actions and Spark SQL. Build practical transformations using PySpark. Module 4 — Delta Lake Understand Delta tables and the Delta Lake architecture. Work with ACID transactions, schema evolution and time travel. Learn practical techniques for maintaining reliable Lakehouse data. Module 5 — Building Data Pipelines Build ingestion and transformation pipelines in Databricks. Handle incremental loads, CDC and changing source data. Design reliable, maintainable and restartable pipelines. Module 6 — Medallion Architecture Understand Bronze, Silver and Gold data layers. Design transformations and data-quality rules between layers. Learn when Medallion Architecture works—and when simpler designs may be better. Module 7 — Data Quality, Governance & Unity Catalog Understand centralized data governance with Unity Catalog. Learn catalog, schema, table and access-control concepts. Apply data-quality, lineage and governance practices. Module 8 — Performance & Production Best Practices Understand partitioning, file sizes, caching and query optimization. Learn common Spark and Databricks performance problems. Design production-ready jobs with monitoring and operational considerations. Module 9 — Modern Data Engineering Project Design an end-to-end Lakehouse architecture. Ingest, transform and publish data through multiple layers. Apply quality, governance and performance practices to the solution. Module 10 — Interview & Career Preparation Practice Databricks, Spark and Data Engineering interview questions. Work through real-world architecture and troubleshooting scenarios. Assess your skills and identify areas requiring further practice.