On this page

Scheduling Tasks

Enrico Hochkoeppler Profile

By Enrico Hochkoeppler

Web Developer & TutoriaLibre Founder

Why scheduling matters in automation​

Automation is not only about performing actions; it is about performing them at the right time. Many workflows require tasks to run periodically, at fixed intervals, or in response to specific conditions. Examples include refreshing datasets, rotating logs, polling APIs, generating reports, or monitoring directories. Python provides all the primitives needed to build reliable scheduling systems without external dependencies. Understanding these primitives allows you to design scripts that run predictably, efficiently, and safely.

Using time.sleep for fixed‑interval scheduling​

The simplest form of scheduling is running a task repeatedly with a delay between executions. The time.sleep() function pauses execution for a specified number of seconds.


import time



while True:

    perform_task()

    time.sleep(60)  # run every 60 seconds

This pattern is appropriate for lightweight, continuous background tasks such as polling a status endpoint or checking for new files.

Scheduling tasks at specific times​

Some workflows require tasks to run at a particular time of day. Python’s datetime module allows you to compute the delay until the next scheduled run.


import time

from datetime import datetime, timedelta



def seconds_until(hour, minute):

    now = datetime.now()

    target = now.replace(hour=hour, minute=minute, second=0, microsecond=0)

    if target <= now:

        target += timedelta(days=1)

    return (target - now).total_seconds()



while True:

    delay = seconds_until(3, 0)  # run daily at 03:00

    time.sleep(delay)

    perform_daily_task()

This approach supports daily, weekly, or periodic scheduling without external tools.

Event‑driven scheduling based on conditions​

Some tasks should run only when certain conditions are met. This is common in monitoring, file‑processing pipelines, and system‑health checks.


import time

from pathlib import Path



watch_dir = Path("incoming")



while True:

    if any(watch_dir.iterdir()):

        process_new_files()

    time.sleep(5)

This pattern allows your script to react to external events without constant manual intervention.

Scheduling with timestamps and logs​

Reliable automation requires visibility into when tasks run. Logging timestamps ensures traceability and helps diagnose issues.


from datetime import datetime



def log(message):

    print(f"[{datetime.now()}] {message}")



log("Starting scheduled task")

perform_task()

log("Task completed")

Timestamped logs are essential for long‑running automation systems.

Avoiding drift in repeated schedules​

Using sleep() after each task introduces drift because the delay begins after the task finishes. To maintain precise intervals, compute the next scheduled time explicitly.


import time

from datetime import datetime, timedelta



interval = timedelta(minutes=10)

next_run = datetime.now()



while True:

    now = datetime.now()

    if now >= next_run:

        perform_task()

        next_run += interval

    time.sleep(1)

This ensures consistent timing even when tasks take variable amounts of time.

Running tasks in parallel​

Some automation workflows require multiple scheduled tasks to run independently. Python’s threading module allows you to run separate schedulers concurrently.


import threading



def run_every(interval, func):

    while True:

        func()

        time.sleep(interval)



threading.Thread(target=run_every, args=(60, task_a), daemon=True).start()

threading.Thread(target=run_every, args=(300, task_b), daemon=True).start()



while True:

    time.sleep(1)

This pattern supports multi‑task automation without blocking the main thread.

Using subprocess for external scheduled commands​

Automation often involves running external tools or scripts. Python’s subprocess module integrates these commands into your scheduling logic.


import subprocess

import time



while True:

    subprocess.run(["python", "generate_report.py"])

    time.sleep(3600)  # run every hour

This allows Python to orchestrate entire pipelines.

Principles of reliable scheduling​

Effective scheduling requires attention to timing accuracy, error handling, and system stability. Robust scripts:

These principles ensure that scheduled automation remains predictable and maintainable over long periods.

Found this helpful? Share the knowledge...

... or support this project to keep it alive!

Buy Me a Coffee

Previous

Automating Web Requests

Next

CLI Tools