Build production-grade deployment pipelines with Databricks Asset Bundles. Package your Project as Code!
Are you ready to put DevOps and CI/CD to work in your Databricks deployments?
What you’ll learn
- Package notebooks, jobs, and configurations as versioned code with Databricks Asset Bundles.
- Create automated CI/CD pipelines that deploy reliably from development to production.
- Build and distribute custom Python packages for use in your Databricks environment.
- Implement unit testing and validation for Databricks code.
- Set up GitHub Actions workflows for automated builds, tests, and deployments.
- Apply DevOps best practices to Databricks.
Course Content
- Introduction and Core Concepts –> 5 lectures • 17min.
- Environment Set Up –> 6 lectures • 13min.
- Databricks CLI –> 3 lectures • 23min.
- Getting Started with Databricks Asset Bundles –> 7 lectures • 43min.
- Set Up for Local Development –> 7 lectures • 22min.
- Citibike Project Set Up –> 2 lectures • 10min.
- Notebook Tasks and Variables with DAB –> 11 lectures • 1hr 17min.
- Python Script Tasks with DAB –> 2 lectures • 12min.
- Distributing your Reusable Code as a Wheel –> 4 lectures • 28min.
- Delta Live Table Pipelines with DAB –> 2 lectures • 13min.
- Databricks Asset Bundles – Summary of YAML Configuration Approach –> 1 lecture • 2min.
- Local SparkSession Development and Unit Testing –> 7 lectures • 32min.
- End to End CI/CD Workflows with DAB and GitHub –> 17 lectures • 1hr 49min.
- Congratulations –> 1 lecture • 1min.
Requirements
Are you ready to put DevOps and CI/CD to work in your Databricks deployments?
In this course, you’ll become an expert in Databricks Asset Bundles—the official “workspace-as-code” framework that brings true DevOps to your analytics platform. You’ll learn to bundle notebooks, jobs, pipelines, cluster specs, infrastructure and workspace configurations into a single, versioned package—and then automate its validation, testing, and multi-stage deployment through CI/CD pipelines. No more one-off clicks or hidden drift—just repeatable, reliable releases.
High-Level Curriculum Overview
- Introduction & Core ConceptsGet oriented with Databricks Asset Bundles and CI/CD concepts. Review the course goals, the “infinite delivery loop,” and where to find code samples for each hands-on module.
- Environment & SetupProvision your Azure Databricks workspaces, configure VS Code, install the Databricks CLI, and prepare Databricks Connect for IDE-driven development.
- Asset Bundles FundamentalsLearn the core databricks bundles commands—init, validate, deploy, run, and destroy—and how to define, version, and manage your analytics project in databricks.yml.
- Local Development and Unit TestingIntegrate PyTest for unit and integration tests, run tests via CI or Databricks Connect, and generate coverage reports to enforce quality gates.Understand how to switch between local PySpark for rapid unit testing and Databricks Connect to execute and debug code on real clusters, ensuring parity between your IDE and the cloud.
- Hands-On ProjectsApply your knowledge in three practical hands-on projects:
- Notebook ETL pipelines (Bronze→Silver→Gold)
- Python script tasks and .whl-packaged jobs
- Delta Live Tables streaming pipelines
- Git Integration & CI/CD PipelinesOnboard your project to Git, adopt branch-based workflows, and author GitHub Actions or Azure Pipelines to automate builds, tests, staging (with approval), and production rollouts.
By the end of this course, you’ll have an automated end to end CI/CD process for your entire Databricks environment.