Skip to content Skip to sidebar Skip to footer

Widget HTML #1

The Ultimate Hands-On Course To Master Apache Airflow

The Ultimate Hands-On Course To Master Apache Airflow
Limk : The Ultimate Hands-On Course To Master Apache Airflow
udemy course code 
Preview this course ... Use Apache Airflow in a Big Data ecosystem with Hive, PostgreSQL, ... Apache NiFi Complete Master Course - HDP - Automation ETL.
by Marc Lamberti

What you'll learn

  • Building end-to-end and production grade data pipelines by mastering Airflow through Hands-On examples
  • Interacting with Hive, Spark, HDFS, Slack and more with Apache Airflow
  • Scaling Airflow with the different executors: Local, Celery and Kubernetes executors
  • Monitoring Airflow with Elasticsearch and Grafana
  • Securing Airflow with authentication, crypto and RBAC UI
  • Applying advanced concepts such as SubDAGs, XComs, Templates, with pro and limitations hands-on examples
  • Mastering DAGs with timezones, unit testing, backfill and catchup
  • Organising the DAG folder and keep things clean
  • Best practices with Airflow

Airflow is a platform created by community to programmatically author, schedule and monitor workflows.

It is scalable, dynamic, extensible and modulable.

Without any doubts, mastering Airflow is becoming a must-have and an attractive skill for anyone working with data.

What you will learn in the course:

  • Fundamentals of Airflow are explained such as what is Airflow, how the scheduler and the web server works
  • The Forex Data Pipeline project is incredible way to discover many operators in Airflow and deal with Slack, Spark, Hadoop and more
  • Mastering your DAGs is a top priority and you will be able to play with timezones, unit testing your DAGs, how to structure your DAG folder and much more
  • Scaling Airflow through different executors such as the Local Executor, the Celery Executor and the Kubernetes Executor will be explained in details. You will discover how to specialise your workers, how to add new workers, what happens when a node crashes.
  • A Kubernetes cluster of 3 nodes will be set up with Rancher, Airflow and the Kubernetes Executor in local to run your data pipelines.
  • Advanced concepts will be shown through practical examples such as templatating your DAGs, how to make your DAG dependent of another, what are Subdags and deadlocks, and more.
  • You will set up a Kubernetes cluster in the cloud with AWS EKS and Rancher  in order to use Airflow along with the Kubernetes Executor
  • Monitoring Airflow is extremely important! That's why you will know how to do it with Elasticsearch and Grafana.
  • Security will be also addressed in order to make your Airflow instance compliant with your company. Specifying roles and permissions for your users with RBAC, Prevent from accessing the Airflow UI with authentication and password,  data encryption and more.
Online Course CoupoNED
Online Course CoupoNED I am very happy that there are bloggers who can help my business

Post a Comment for "The Ultimate Hands-On Course To Master Apache Airflow"