Databricks Platform - Architecture, Security, Automation and much more!!
PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
Databricks DLT Apparel Pipeline Project: Learn medallion architecture, streaming, and data engineering with Delta Live Tables. Includes synthetic data, step-by-step guide, and certification prep.
End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)
An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
Repository for Microsoft Databricks Training Events - Hosted by BlueGranite
[archived] A Python SDK for the Azure Databricks REST API 2.0
Reusable Python classes that extend open source PySpark capabilities. Examples of implementation is available under notebooks of repo https://github.com/bennyaustin/synapse-dataplatform
Automated pipeline for energy consumption forecasting across Europe using Azure cloud and Databricks.
Free High-Quality Financial Data in Azure
A wrapper for the Azure Databricks REST API
A demand forecasting pipeline deployed on Azure and AWS
Applying data engineering techniques to create data pipeline with Azure Cloud Computing
ETL motor racing data project using Azure Databricks, Pyspark and Azure Date Lakes
A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.
End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.
End-to-end Azure Databricks data engineering project using Autoloader, Delta Live Tables (DLT), and Medallion Architecture.
config, logs, error handling and projects
Production-grade real-time ELT pipeline using PySpark Structured Streaming and Delta Lake. Replicates a high-impact architectural migration from Mercedes-Benz to achieve exactly-once upsert semantics and 60% reduction in cloud compute overhead.
Notebooks Azure Databricks sem versionamento, testes ou deploy estruturado criam caos operacional em ambientes regulados | Pipeline com Arquitetura Medalhão, libs Python testáveis e CI/CD com GitHub Actions que bloqueia código sem qualidade | Deploy automatizado de notebooks, clusters e jobs no workspace Databricks a cada push aprovado
Add a description, image, and links to the azure-databricks topic page so that developers can more easily learn about it.
To associate your repository with the azure-databricks topic, visit your repo's landing page and select "manage topics."