Data Engineering with Azure Synapse Analytics
The objective of this course is to equip individuals who have no prior experience in data engineering with the skills from a beginner to an advanced level, utilizing tools such as SQL, Microsoft Azure Synapse Analytics, and Spark Pool. We will provide Azure Subscriptions for all the Lab Sessions in this course.
-
SQL Fundamentals
- Introduction to SQL Statements
- SELECT
- FROM
- WHERE
- GROUP BY
- HAVING
- ORDER BY
- LIMIT
- Aliases
- NULL
-
SQL Lab Files
-
SQL JOINS
- INNER JOIN
- FULL OUTER JOIN
- LEFT JOIN
- RIGHT JOIN
- CROSS JOIN
- Subqueries
- Aggregate Functions
-
Introduction to Data Engineering
- Roles of Data Engineers
- Data Engineering Tools
- Programming Languages for Data Engineering
- Data Operations and Pipelines
-
Azure Data Services and Overview
- Azure Synapse Analytics Overview
- Azure Data Factory Overview
- Azure Databricks
- Azure Data Lake Storage
- Azure CosmoDB
- Azure SQL Database
- Hands-On Lab
-
Azure Synapse Analytics
- Introduction to Serverless SQL Pool
- Introduction to Dedicated SQL Pool
- Introduction to Lakehouse Database
- Introduction to Synapse Spark
- Introduction to Synapse Pipeline
- Managing and Monitoring a Synapse Workspace
-
Serverless SQL Pool
- Data Exploration with Serverless SQL Pool
- Logical Data warehouse / Lakehouse Database
- Data Transformation with Serverless SQL Pool
- Hands-On Lab
-
Data Ingestion with Serverless SQL Pool
- Introduction to Data Ingestion
- Create External Tables AS (CETAS)
- Transform to Parquet
- Transform JSON to Parquet
- Transform Partitioned Data to Parquet
- Introduction to Stored Procedure
- Hands-On Lab
-
Dedicated SQL Pool
- Introduction to Dedicated SQL Pool
- Understanding the architecture and components of Dedicated SQL Pool in Azure Synapse.
- Creating and managing Dedicated SQL Pool using Azure Synapse Studio.
- Configuring and optimizing the performance of Dedicated SQL Pool.
- Loading data into Dedicated SQL Pool from various sources.
- Implementing data security and access control in Dedicated SQL Pool.
- Developing and executing complex SQL queries in Dedicated SQL Pool.
- Monitoring and troubleshooting Dedicated SQL Pool using Azure Synapse Analytics.
- Integrating Dedicated SQL Pool with Azure Power BI.
- Designing and implementing data warehousing solutions using Dedicated SQL Pool.
- Scaling up and down Dedicated SQL Pool based on business requirements and workload patterns.
- Hands-On Lab
-
Azure Data Engineering Solution
- Stream Data processing with Stream Analytics and Event Hub and Synapse
- Hybrid Data integration with Azure Synapse Analytics (on-premises Data sources)
- Hybrid Transactional and Analytical Processing (HTAP) with Cosmos DB and Synapse Link
- Data engineering with Databricks (overview)
-
Data Pipelines & Triggers with Azure Synapse
- Introduction to Data Pipelines in Azure Synapse
- Synapse Pipelines Overview
- Synapse Pipelines Components
- Transformation Pipeline Design
- Create Linked Service & Dataset
- Creating Pipeline
- Parameters and Variables Overview
- Dynamic Pipeline
- Pipeline Design - Partitioned File
- Pipeline Dependencies
- Manual Triggers
- Automated Triggers
- Hands-On Lab
-
Stream Data Processing with Event Hub
- Event Hub concepts: Understanding the core concepts of Event Hub, including event publishers, event consumers, partitions, and consumer groups.
- Event ingestion: Knowledge of different methods to ingest events into Event Hub, such as using Azure Functions.
- Scaling and performance tuning: Understanding how to scale Event Hub to handle increased event throughput and optimize performance by configuring throughput units and managing partitions.
-
Monitoring and diagnostics: Familiarity with monitoring and diagnostic capabilities of Event Hub, including monitoring ingress/egress rates, latency, and availability metrics.
-
Stream Data Processing with Stream Analytics
- Stream processing concepts: Understanding the fundamentals of stream processing, such as data ingestion, data transformation, windowing, aggregation, and output sinks.
- Query language: Familiarity with the query language used in Stream Analytics, such as SQL-like syntax or the Stream Analytics Query Language (SAQL).
- Joins and enrichment: Ability to join multiple streams or enrich streaming data with reference data using various techniques, such as temporal joins or lookup operations.
-
Output sink configuration: Knowledge of configuring output sinks for storing or forwarding processed streaming data to various destinations, such as Azure Storage, Azure Data Lake, or other external systems.
-
Stream Data Processing with Synapse
- Data ingestion and integration: Knowledge of various methods to ingest streaming data into Synapse.
- Query optimization: Ability to optimize queries for performance, such as partitioning tables, indexing strategies, and utilizing distribution keys effectively.
- Data warehousing concepts: Familiarity with data warehousing concepts, such as star schema, data modeling, fact and dimension tables, and managing data consistency.
- Data security and access control: Knowledge of implementing data security measures, including managing access control, encrypting data at rest and in transit, and applying data masking or row-level security.
-
Monitoring and performance tuning: Understanding how to monitor and tune the performance of Synapse workloads, including query performance, resource utilization, and data loading efficiency.
-
Integrating Azure Synapse with PowerBI
- Introduction to PowerBI
- PowerBI Integration Overview
- Connecting from PowerBI Desktop
- Publish to PowerBI Workspace
- Azure Synapse Studio PowerBI Integration
- Creating Reports from Azure Synapse Studio
- Hands-On Lab
In this course, you will learn how to perform data engineering with Microsoft Azure Synapse Analytics and Advanced SQL which will enable you boost the performance of big data analytic applications.
What you'll learn
-
You will learn how to build a real world project using Azure Synapse Analytics.
-
You will acquire professional level data engineering skills in Azure Synapse Analytics
-
You will learn how to create SQL scripts and Spark notebooks in Azure Synapse Analytics
-
You will learn how to create dedicated SQL pools and spark pools in Azure Synapse Analytics
-
You will learn how to enable synapse link and enable analytic store in Cosmos DB
-
You will learn how to ingest and transform data Serverless SQL Pool and Spark Pool
-
You will learn how to load data into dedicated SQL Pool
-
You will learn how to serve data to Power BI from Serverless SQL Pool and Dedicated SQL Pool
-
You will learn how to execute scripts and notebooks using Synapse Pipelines and Triggers
-
You will learn how to do operational reporting from the data stored in Cosmos DB using Azure Synapse Analytics
-
You will learn how to build reports in Power BI for the data stored in Azure Synapse Analytics
How students rated this courses
4.7
(Based on 10 reviews)
Reviews
Ajayi Kehinde 06 Aug, 2023 - 8:33 AM
I really enjoyed the course. The instructor was very detailed. I enjoyed the lab sessions on Microsoft Azure as this made the course more hands-on.
Was this review helpful?
Dare Adebayo 06 Aug, 2023 - 2:33 PM
This course is fantastic! It is worth learning.
Was this review helpful?
abiola ahmed 06 Aug, 2023 - 2:33 PM
Great
Was this review helpful?
Oluwadare Olaitan 06 Aug, 2023 - 2:36 PM
Top notch lecturers! I learnt a lot
Was this review helpful?
Agboola Tijani Rofiat 06 Aug, 2023 - 2:36 PM
The Azure Data Engineering course was well-detailed, and the tutor did his best to pass the knowledge down to us in the simplest way possible. I will recommend this course with Ailead to anyone who wants to tap into this career path.
Was this review helpful?
Nsofor Ngozi 06 Aug, 2023 - 2:38 PM
A lot of concepts were covered with hands-on experience on Azure. Case studies were used in labs, and step-by-step solutions were provided, giving students sufficient practice. Very good experience!
Was this review helpful?
Tabansi James 06 Aug, 2023 - 2:42 PM
A wonderful blend of theory and hands-on experience.
Was this review helpful?
Onyekelu Gabriel 06 Aug, 2023 - 4:27 PM
It was an awesome experience, I will recommend the tech school to anyone, anytime, anyday. I\'m very happy
Was this review helpful?
toriola ayodeji 06 Aug, 2023 - 5:05 PM
It was a great experience with the tutor, his lecture delivery is superb. Despite not having background as a data engineer, the class as given me the require skillsets to have engagement on Data Engineering with Azure synapse. I recommend this to anyone ready to venture into data engineering. It gives a great value for your money and time.
Was this review helpful?
Atughara John 07 Aug, 2023 - 3:27 AM
The facilitator for the course was patient and very supportive even when you are behind in class. It was a worth it experience.
Was this review helpful?
Transcript from the "Introduction" Lesson
Course Overview [00:00:00]
My name is John Deo and I work as human duct tape at Gatsby, that means that I do a lot of different things. Everything from dev roll to writing content to writing code. And I used to work as an architect at IBM. I live in Portland, Oregon.
Introduction [00:00:16]
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
Why Take This Course? [00:00:37]
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
A Look at the Demo Application [00:00:54]
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
Summary [00:01:31]
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
Course - Frequently Asked Questions
How this course help me to design layout?
My name is Jason Woo and I work as human duct tape at Gatsby, that means that I do a lot of different things. Everything from dev roll to writing content to writing code. And I used to work as an architect at IBM. I live in Portland, Oregon.
What is important of this course?
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
Why Take This Course?
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
Is able to create application after this course?
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
We'll dive into GraphQL, the fundamentals of GraphQL. We're only gonna use the pieces of it that we need to build in Gatsby. We're not gonna be doing a deep dive into what GraphQL is or the language specifics. We're also gonna get into MDX. MDX is a way to write React components in your markdown.
