Introduction to dbt

Overview:

An introduction to dbt, its purpose, and its place in the modern data stack.

Topics to Cover:

• What is dbt and how does it fit in the data pipeline?
• Key features and benefits of dbt
• Understanding the dbt ecosystem: dbt CLI vs. dbt Cloud
• Introduction to cloud data warehouses and dbt’s integration with them

Overview:

Understanding the broader context of dbt in the cloud data stack and the roles involved in data transformation.

Topics to Cover:

• Overview of modern cloud data stack (data lakes, data warehouses, and data marts)
• Roles in data transformation: Data Engineer, Analyst, and Scientist
• The role of dbt in the cloud data stack
• Key cloud platforms: Snowflake, BigQuery, Redshift

Overview:

Learn how to use source() and ref() functions to create reusable and modular dbt models.

Topics to Cover:

• How to define and use sources in dbt
• Ref functions and model dependencies in dbt
• Best practices for organizing dbt models with ref() and source()

Overview:

A comprehensive look at the three primary layers in dbt models: Staging, Intermediate, and Mart layers, including both conceptual and implementation details.

Topics to Cover:

• Conceptual overview of the three layers
• The purpose of each layer in a dbt project
• Common practices for defining each layer
• Detailed exploration of each layer’s structure
• Data transformations at each layer
• Optimizing the flow of data between the layers

Overview:

Learn about dbt’s materialization strategies and how they affect performance and scalability.

Topics to Cover:

• Overview of dbt materializations (Table, Incremental, Ephemeral, etc.)
• When and why to use different materializations
• Configuring materializations in dbt

Overview:

A deep dive into the Incremental materialization, a key feature for scaling large datasets.

Topics to Cover:

• Benefits of incremental models
• How to configure and implement incremental materialization
• Troubleshooting incremental models in dbt

Overview:

Learn how to organize and structure your dbt project efficiently.

Topics to Cover:

• Default dbt folder structure
• Best practices for managing models, tests, and macros
• Version control and managing changes in dbt projects

Overview:

Introduction to dbt snapshots for handling slowly changing dimensions (SCD).

Topics to Cover:

• What are dbt snapshots and why are they used?
• How to implement snapshots to track historical data changes
• Best practices for using snapshots in dbt

Overview:

Learn how to implement testing and leverage dbt packages to extend dbt functionality.

Topics to Cover:

• Introduction to dbt testing
• Types of tests (singular, relationships, etc.)
• Leveraging dbt packages and the dbt hub for reusable models and macros

Overview:

This module introduces the concept of versioning individual dbt models to handle data pipeline changes more efficiently. We’ll explore strategies for managing breaking changes, testing new versions, and ensuring smooth transitions.

Topics to Cover:

• What is model-level version control and its importance?
• Implementing version control in dbt models
• Managing breaking changes in dbt models
• Testing new versions of dbt models