# Pipeline Scheduling Basics Lab

> Understanding how to schedule jobs is crucial for effective data engineering, and this lab focuses on using Cron and Python to automate tasks. Participants will learn to set up job scheduling in a stock trading application, covering both Python's scheduling capabilities and Cron tab syntax for Linu…

- Web page: https://learn.dataexpert.io/lesson/pipeline-scheduling-basics-lab
- Program: [The Absolutely Free Beginner Data Engineering Boot Camp starting August 14th](https://learn.dataexpert.io/program/the-absolute-beginner-data-engineering-boot-camp-starting-august-7th-6453.md)
- Access: Free — no enrollment needed
- Length: 9 min video
- Skills: Python, ETL/ELT, Airflow
- Academy: DataExpert.io Community Academy

## About this lesson

Understanding how to schedule jobs is crucial for effective data engineering, and this lab focuses on using Cron and Python to automate tasks. Participants will learn to set up job scheduling in a stock trading application, covering both Python's scheduling capabilities and Cron tab syntax for Linux systems. This lesson is ideal for data engineers and developers looking to enhance their automation skills and ensure reliable job execution in data pipelines.

## Transcript excerpt

Cron is the fundamental component of 90% of data pipelines and data engineering, and it allows you to schedule things on a regular interval, whether it be daily, hourly, every minute, every week, every year, whatever it might be. Uh, in this lab, we are going to be diving deep and we're going to be connecting in our stock trading app and allow you to see how this stuff will run on a schedule. I'm really excited for you to check it out. Hey, welcome back everyone to Data Pipeline Basics. So today we're going to be going over how to regularly schedule a job, and we're going to show how to do it in two different ways. Uh, first off, we're going to show how to do it in Python, and then we're going to show how to use it in Cron tab. So, uh, if you're on Windows, you need to use the task scheduler, and that will not be covered in this video because I'm only covering. and Linux systems here, not Windows systems, but it's very straightforward to do on Windows if you just chat GPT how to schedule a task, you can check out how to do it on Windows as well. So anyways, let's go over um what's going on here, right? So, uh, if you can clone the stock trading Python app, that's gonna be here. This is what we used from uh yesterday's uh lecture on rest, rest extraction. So we're gonna be using this job here. So what we're gonna do is We are just going to run this code, but before we do that, we need to install something, right?

## Navigation

- Previous lesson: [How do data pipelines work](https://learn.dataexpert.io/lesson/how-do-data-pipelines-work.md)
- Next lesson: [Loading API Data Into Snowflake](https://learn.dataexpert.io/lesson/loading-api-data-into-snowflake.md)