1. Introduction to Azure Cloud and Data Engineer profile

• Create Azure Subscription
• Azure Portal
• How did Data Engineer Profile Evolve
• Data Engineer Role and Responsibility
• Data Engineer Technologies

3. Implement relational data stores

• Azure SQL: Why?
• Azure Iaas vs Pass Database Offerings
• Azure SQL: PaaS Deployment Options
• Azure SQL Server Demo: Provision Single Database
• Azure SQL Server: Purchasing models and service tiers
• Azure Elastic Database
• Azure Elastic Database Demo: Provision
• Azure SQL Database: Security layers
• Azure SQL Database High Availability and Disaster Recovery options
• Traditional vs Modern Warehouse architecture
• What is Synapse Analytics Service [Updated: Dec 2020]
• Azure Synapse Analytics Demo: Formerly Azure SQL Data Warehouse
• Azure Synapse: MPP Architecture
• Azure Synapse: Storage and Sharding Patterns
• Azure Synapse: Data Distribution and Distributing Keys
• Azure Synapse: Data Types and Table Types
• Azure Synapse: Partitioning
• Azure Synapse: Best Practices for Fact and Dimension tables
• Demo: Analyze Data distribution before migration to Azure
• Azure Synapse: Different loading methods
• Azure Synapse: Loading with SSIS vs PolyBase
• Azure Synapse Demo: Loading with Polybase
• Azure Database vs Azure Datawarehouse (Synapse Data Pool)

5. Develop batch processing solutions ‘

• What is Data Factory
• Data Factory within Azure Eco system
• Provision Data Factory
• Data Factory – Components
• Data Factory – Pipeline and Activities
• Data Factory – Linked service and Datasets
• Data Factory – Integration Runtime
• Data Factory – Triggers
• Demo: Copy Data Activity through wizard
• Demo: Copy Data Activity using Author page
• What is Azure Databricks
• How to save Databricks demo Cost
• Demo: Provision Databricks, Clusters and workbook
• Demo: Mount Data Lake to Databricks DBFS
• Demo: Explore, Analyze, Clean, Transform and Load Data in Databricks
• Azure Databricks Clusters
• Azure Databricks Other Important Components

7. Monitor data storage

• intro to Azure Monitor service
• Demo: Azure Monitor Service
• Implementing Blob and Data Lake Storage monitoring
• Implement Azure Synapse Analytics monitoring
• Implement Cosmos DB monitoring

2. Implement non-relational data stores

• NoSQL Offerings by Microsoft Azure
• Azure Storage Introduction
• Demo Part 1 Provision Azure Storage Account
• Demo Part 2 Replication
• Demo Part 3 Continue Provisioning Storage account
• Azure Blob Storage
• Cosmos DB: Problem Statement – How Cosmos DB Evolved?
• Cosmos DB: Features
• Cosmos DB: Multi Model 5 APIs
• Cosmos DB: Provision Account
• Cosmos DB: Database Containers and items
• Cosmos DB: Throughput and request units
• Cosmos DB: Horizontal Scaling
• Cosmos DB: What is partitioning and partition key
• Cosmos DB: Dedicated vs Shared throughput
• Cosmos DB: Avoiding hot partition
• Cosmos DB: Single partition vs Cross partition
• Cosmos DB: Composite Key
• Cosmos DB: Partition key best practice
• Cosmos DB: Demo – Insert and query data
• Cosmos DB: Time to Live
• Cosmos DB: Globally Distribution
• Cosmos DB: Multi Master
• Cosmos DB: Manual vs Automatics Fail-over
• Cosmos DB: 5 consistent level
• Cosmos DB: CLI
• Cosmos DB: Pricing
• Cosmos DB: Security
• What is Data Lake?
• How Data Lake Gen 2 evolved
• Azure Blob Storage vs Azure Data Lake
• Azure Blob & Data Lake Security options
• High Availability vs Disaster Recovery
• Azure Storage – HA and DR Options
• Cosmos DB – HA and DR Options

4. Manage Data Security

• Implement Data masking
• Encrypt data at rest and in motion

6. Develop streaming solutions

• Introducing Azure Stream Analytics (configure input and output)
• Select the appropriate Windowing Functions
• Tumbling Window
• Hopping Window
• Sliding Window
• Session Window
• Demo: Implement Blob event processing by using Stream Analytics

8. Monitor data processing

• Monitor Data Factory Pipelines
• Monitor Data Factory – Metrics, Alerts, Diagnostic Settings
• Monitor Azure Databricks
• Monitor Stream Analytics

9. optimize of Azure data Solutions

• Troubleshoot Data Partitioning Bottlenecks
• Optimize Data Lake Storage
• Optimize Stream Analytics
• Optimize Azure Synapse Analytics
• Manage the Data Lifecycle