- Courses
- 💻Programming Languages
- 🐍Python
- Automation & Scripting
- Scheduling Tasks
On this page
Scheduling Tasks
By Enrico Hochkoeppler
Web Developer & TutoriaLibre Founder
Why scheduling matters in automation
Automation is not only about performing actions; it is about performing them at the right time. Many workflows require tasks to run periodically, at fixed intervals, or in response to specific conditions. Examples include refreshing datasets, rotating logs, polling APIs, generating reports, or monitoring directories. Python provides all the primitives needed to build reliable scheduling systems without external dependencies. Understanding these primitives allows you to design scripts that run predictably, efficiently, and safely.
Using time.sleep for fixed‑interval scheduling
The simplest form of scheduling is running a task repeatedly with a delay between executions. The time.sleep() function pauses execution for a specified number of seconds.
import time
while True:
perform_task()
time.sleep(60) # run every 60 seconds
This pattern is appropriate for lightweight, continuous background tasks such as polling a status endpoint or checking for new files.
Scheduling tasks at specific times
Some workflows require tasks to run at a particular time of day. Python’s datetime module allows you to compute the delay until the next scheduled run.
import time
from datetime import datetime, timedelta
def seconds_until(hour, minute):
now = datetime.now()
target = now.replace(hour=hour, minute=minute, second=0, microsecond=0)
if target <= now:
target += timedelta(days=1)
return (target - now).total_seconds()
while True:
delay = seconds_until(3, 0) # run daily at 03:00
time.sleep(delay)
perform_daily_task()
This approach supports daily, weekly, or periodic scheduling without external tools.
Event‑driven scheduling based on conditions
Some tasks should run only when certain conditions are met. This is common in monitoring, file‑processing pipelines, and system‑health checks.
import time
from pathlib import Path
watch_dir = Path("incoming")
while True:
if any(watch_dir.iterdir()):
process_new_files()
time.sleep(5)
This pattern allows your script to react to external events without constant manual intervention.
Scheduling with timestamps and logs
Reliable automation requires visibility into when tasks run. Logging timestamps ensures traceability and helps diagnose issues.
from datetime import datetime
def log(message):
print(f"[{datetime.now()}] {message}")
log("Starting scheduled task")
perform_task()
log("Task completed")
Timestamped logs are essential for long‑running automation systems.
Avoiding drift in repeated schedules
Using sleep() after each task introduces drift because the delay begins after the task finishes. To maintain precise intervals, compute the next scheduled time explicitly.
import time
from datetime import datetime, timedelta
interval = timedelta(minutes=10)
next_run = datetime.now()
while True:
now = datetime.now()
if now >= next_run:
perform_task()
next_run += interval
time.sleep(1)
This ensures consistent timing even when tasks take variable amounts of time.
Running tasks in parallel
Some automation workflows require multiple scheduled tasks to run independently. Python’s threading module allows you to run separate schedulers concurrently.
import threading
def run_every(interval, func):
while True:
func()
time.sleep(interval)
threading.Thread(target=run_every, args=(60, task_a), daemon=True).start()
threading.Thread(target=run_every, args=(300, task_b), daemon=True).start()
while True:
time.sleep(1)
This pattern supports multi‑task automation without blocking the main thread.
Using subprocess for external scheduled commands
Automation often involves running external tools or scripts. Python’s subprocess module integrates these commands into your scheduling logic.
import subprocess
import time
while True:
subprocess.run(["python", "generate_report.py"])
time.sleep(3600) # run every hour
This allows Python to orchestrate entire pipelines.
Principles of reliable scheduling
Effective scheduling requires attention to timing accuracy, error handling, and system stability. Robust scripts:
- Use explicit timestamps to avoid drift
- Log every execution
- Handle exceptions without stopping the scheduler
- Avoid tight loops that consume unnecessary CPU
- Use daemon threads for background tasks
- Validate assumptions about timing and system load
These principles ensure that scheduled automation remains predictable and maintainable over long periods.
Found this helpful? Share the knowledge...
... or support this project to keep it alive!
Previous
Automating Web Requests
Next
CLI Tools
- Why scheduling matters in automation
- Using time.sleep for fixed‑interval scheduling
- Scheduling tasks at specific times
- Event‑driven scheduling based on conditions
- Scheduling with timestamps and logs
- Avoiding drift in repeated schedules
- Running tasks in parallel
- Using subprocess for external scheduled commands
- Principles of reliable scheduling
