☁️ Big Data Analytics & Processing with Microsoft Azure Course

Module 1: Fundamentals of Azure Big Data Architecture

ü  Defining Big Data characteristics (Volume, Velocity, Variety)

ü  Overview of the Modern Data Warehouse Architecture on Azure

ü  Key Azure services: Data Lake, Synapse, Data Factory

ü  Choosing the right compute for specific workloads

ü  Architectural considerations for batch vs. streaming data

ü  Practical Session: Setting up an Azure Resource Group and Initial Data Services

Module 2: Azure Data Lake Storage (ADLS Gen2) and Data Hierarchy

ü  Understanding Data Lake fundamentals (unstructured storage)

ü  Implementing Hierarchical Namespace and Access Control Lists (ACLs)

ü  Organizing data using the Medallion Architecture (Bronze, Silver, Gold)

ü  Data partitioning and file format optimization (Parquet, Delta)

ü  Data ingestion security and storage account configuration

ü  Practical Session: Creating ADLS Gen2 Containers and Implementing Folder Structure

Module 3: Data Ingestion and ETL Orchestration with Azure Data Factory (ADF)

ü  Introduction to Azure Data Factory interface and components (Pipelines, Activities)

ü  Connecting data sources using Linked Services and Datasets

ü  Using the Copy Activity for bulk data movement

ü  Implementing control flow activities (If Condition, For Each)

ü  Monitoring and troubleshooting ADF pipeline runs

ü  Practical Session: Designing and Deploying an End-to-End ELT Pipeline in Azure Data Factory

Module 4: Advanced Data Flow Transformation in ADF

ü  Understanding ADF Mapping Data Flows for code-free transformation

ü  Data flow operations: Joins, aggregates, conditional splits

ü  Implementing data quality checks within a Data Flow

ü  Using parameters in Data Flows for reusability

ü  Data Flow performance optimization techniques

ü  Practical Session: Building a Data Flow to Clean and Aggregate Data

Module 5: Introduction to Azure Synapse Analytics Workspace

ü  Overview of the Synapse unified workspace features

ü  Provisioning and managing Synapse SQL and Spark pools

ü  Integrating ADLS Gen2 with Synapse

ü  Using Synapse Studio for development and monitoring

ü  Understanding the shared metadata model

ü  Practical Session: Creating a Synapse Workspace and Connecting to ADLS

Module 6: Data Processing with Synapse Serverless and Dedicated SQL Pools

ü  Differentiating between Serverless and Dedicated SQL Pools

ü  Querying data directly in ADLS using Serverless SQL (OPENROWSET)

ü  Designing and loading data into Dedicated SQL Pool tables (PolyBase/COPY)

ü  Best practices for distribution and indexing in Dedicated SQL Pool

ü  Optimizing queries for cost and performance

ü  Practical Session: Querying Parquet Data in ADLS using Serverless SQL

Module 7: Big Data Processing with Synapse Spark Pools (PySpark)

ü  Introduction to Apache Spark architecture in Synapse

ü  Working with Synapse Notebooks (PySpark, Scala)

ü  Loading data from ADLS into Spark DataFrames

ü  Common Spark data transformations and actions

ü  Using Spark for complex joins and aggregations

ü  Practical Session: Writing and Executing a PySpark Notebook for Data Transformation

Module 8: Mastering Delta Lake and Data Quality

ü  Understanding the Delta Lake storage layer benefits (ACID transactions)

ü  Implementing Delta tables for reliability and schema enforcement

ü  Using Delta Lake for data versioning and time travel

ü  Performing UPSERT (Merge) operations using Delta Lake

ü  Building a reliable data ingestion pipeline using Delta Lake principles

ü  Practical Session: Converting Parquet Files to Delta Lake Format and Performing a Merge

Module 9: Real-Time Data Ingestion with Azure Event Hubs/IoT Hub

ü  Introduction to Event Hubs for high-throughput stream ingestion

ü  Utilizing IoT Hub for device-to-cloud telemetry

ü  Designing partition keys for maximizing throughput

ü  Data formatting and serialization (JSON, Avro)

ü  Integrating streaming sources with the data lake

ü  Practical Session: Simulating Data Ingestion into an Azure Event Hub

Module 10: Stream Processing with Azure Stream Analytics

ü  Defining inputs, outputs, and transformations in Stream Analytics

ü  Using Stream Analytics Query Language (SQL-like)

ü  Implementing windowing functions (Tumbling, Hopping) for time-series analysis

ü  Outputting stream results to Synapse or Power BI

ü  Monitoring stream job performance and latency

ü  Practical Session: Creating a Stream Analytics Job to Aggregate Windowed Data

Module 11: NoSQL Data Processing with Azure Cosmos DB

ü  Overview of Cosmos DB API types and global distribution

ü  Data modelling and partition key strategy for Cosmos DB

ü  Integrating Cosmos DB with Synapse Analytics (Synapse Link)

ü  Using ADF to ingest/extract data from Cosmos DB

ü  Performance tuning and cost management for NoSQL workloads

ü  Practical Session: Querying Cosmos DB Data using Synapse Serverless SQL

Module 12: Data Governance and Cataloging (Azure Purview)

ü  Introduction to Azure Purview for unified data governance

ü  Scanning data sources and automatic data classification

ü  Utilizing the Purview Data Catalog for discovery and lineage

ü  Defining business glossary and data policies

ü  Implementing access management through Purview

ü  Practical Session: Searching the Purview Catalog for Data Assets

Module 13: Data Security in the Azure Data Estate

ü  Implementing encryption at rest (ADLS, Synapse) and in transit

ü  Role-Based Access Control (RBAC) implementation across services

ü  Securing endpoints and implementing Virtual Network integration

ü  Column-level security and dynamic data masking in Synapse SQL

ü  Managing secrets using Azure Key Vault

ü  Practical Session: Applying RBAC to Limit Access to Specific ADLS Folders

Module 14: Monitoring and Logging with Azure Monitor

ü  Setting up diagnostics and logging for ADF and Synapse

ü  Utilizing Azure Monitor and Log Analytics for centralized logging

ü  Creating custom alerts for pipeline failures and performance issues

ü  Tracking Big Data job metrics (Spark, SQL)

ü  Dashboarding operational health in Azure Monitor

ü  Practical Session: Creating an Alert for ADF Pipeline Failure

Module 15: Performance Tuning and Cost Optimization

ü  Optimizing Spark cluster size, worker nodes, and settings

ü  Best practices for file size, compression, and partitioning in ADLS

ü  Dedicated SQL Pool resource class management

ü  Scaling and pausing Synapse resources to manage costs

ü  Practical Session: Analyzing a PySpark Job Profile for Performance Bottlenecks

Module 16: Implementing CI/CD for Data Pipelines

ü  Introduction to DevOps principles for Data Engineering

ü  Integrating Azure DevOps/GitHub for source control

ü  Implementing CI/CD pipelines for Azure Data Factory deployment

ü  Automating Synapse asset deployment (notebooks, SQL scripts)

ü  Managing development, staging, and production environments

ü  Practical Session: Setting up a Basic CI/CD Pipeline in Azure DevOps

Module 17: Introduction to Machine Learning Pipelines

ü  Overview of Azure Machine Learning service

ü  Using Synapse Spark to prep features for ML training

ü  Integrating Synapse with Azure ML for model training

ü  Scoring data in batch using Synapse pipelines

ü  Monitoring model performance and drift

ü  Practical Session: Loading Pre-Processed Data into an Azure ML Workspace

Module 18: Certification Review and Capstone Project

ü  Comprehensive review of all modules and key certification topics

ü  Participants architect and build a complete Big Data solution based on a case study

ü  Final presentation and defense of the integrated solution (Storage, ETL, Processing)

ü  Q&A and final exam preparation strategies

ü  Practical Session: Final Capstone Project Solution Deployment and Presentation

About Our Trainers

Our trainers are Microsoft Certified Trainers (MCTs) and seasoned Cloud Data Architects with a minimum of 10 years of experience designing and managing enterprise-scale Big Data solutions on Azure. They hold advanced certifications (such as the relevant Microsoft big data certification) and specialize in Microsoft Azure big data analytics corporate training. Their practical expertise ensures participants gain deep; actionable knowledge aligned with industry best practices and the requirements for a Microsoft Certified: big data processing role.

Quality Statement

Phoenix Training Center is committed to delivering a superior Microsoft Azure big data analytics corporate training course. We guarantee a rigorous, hands-on curriculum, expert-led instruction, and a focus on practical deployment, ensuring participants are fully prepared to achieve their Microsoft big data certification and lead their organization's Big Data initiatives.

Admission Criteria

ü  Participants should be reasonably proficient in English. 

ü  Applicants must live up to Phoenix Center for Policy, Research and Training admission criteria.

Terms and Conditions

1. Discounts: Organizations sponsoring Four Participants will have the 5th attend Free
2. What is catered for by the Course Fees: Fees cater for all requirements for the training – Learning materials, Lunches, Teas, Snacks and Certification. All participants will additionally cater for their travel and accommodation expenses, visa application, insurance, and other personal expenses.
3. Certificate Awarded: Participants are awarded Certificates of Participation at the end of the training.
4. The program content shown here is for guidance purposes only. Our continuous course improvement process may lead to changes in topics and course structure.
5. Approval of Course: Our Programs are NITA Approved. Participating organizations can therefore claim reimbursement on fees paid in accordance with NITA Rules.

Booking for Training

Simply send an email to the Training Officer on training@phoenixtrainingcenter.com and we will send you a registration form. We advise you to book early to avoid missing a seat to this training.

Or call us on +254720272325 / +254737296202

Payment Options

We provide 3 payment options, choose one for your convenience, and kindly make payments at least 5 days before the Training start date to reserve your seat:

1. Groups of 5 People and Above – Cheque Payments to: Phoenix Center for Policy, Research and Training Limited should be paid in advance, 5 days to the training.
2. Invoice: We can send a bill directly to you or your company.
3. Deposit directly into Bank Account (Account details provided upon request)

Cancellation Policy

1. Payment for all courses includes a registration fee, which is non-refundable, and equals 15% of the total sum of the course fee.
2. Participants may cancel attendance 14 days or more prior to the training commencement date.
3. No refunds will be made 14 days or less before the training commencement date. However, participants who are unable to attend may opt to attend a similar training course at a later date or send a substitute participant provided the participation criteria have been met.

Tailor-Made Courses

We understand that every organization has unique challenges and opportunities as well as unique training needs. Phoenix Training Center offers tailor-made courses designed to address specific requirements and challenges faced by your team or organization. Whether you need a customized curriculum, a specific duration, or on-site delivery, we can adapt our expertise to provide a training solution that perfectly aligns with your objectives.

We can customize this Course to focus on your industry, specific risk profile, or internal stakeholder dynamics. Contact us to discuss how we can create a bespoke training program that maximizes value and impact for your team. For further inquiries, please contact us on Tel: +254720272325 / +254737296202 or Email training@phoenixtrainingcenter.com

Accommodation and Airport Pick-up

For physical training attendees, we can assist with recommendations for accommodation near the training venue. Airport pick-up services can also be arranged upon request to ensure a smooth arrival. Please inform us of your travel details in advance if you require these services. For reservations contact the Training Officer on Email: training@phoenixtrainingcenter.com or on Tel: +254720272325 / +254737296202