- Courses
- đ»Programming Languages
- đPython
- Automation & Scripting
- Automating Files & Folders
On this page
Automating Files & Folders
By Enrico Hochkoeppler
Web Developer & TutoriaLibre Founder
Why filesystem automation mattersâ
Whenever a project involves repeated interactions with files and directories, manual work quickly becomes unreliable and inefficient. Tasks such as reorganizing datasets, generating backups, preparing directory structures, or enforcing naming conventions require deterministic, repeatable processes. Pythonâs standard library provides a complete, crossâplatform toolkit for these operations, allowing you to design scripts that behave predictably and remain maintainable over time.
A professional automation script must handle unexpected states gracefully: missing files, alreadyâexisting directories, or partially processed data. This demands a clear understanding of how Python represents paths, how it traverses directories, and how it performs operations such as copying, moving, renaming, and deleting.
Representing paths with pathlibâ
The pathlib module treats filesystem paths as objects rather than raw strings. A Path object encapsulates both the structure of the path and the operations that can be performed on it. Creating a Path does not touch the filesystem; it simply represents a location.
from pathlib import Path
p = Path("data/report.txt")
You can inspect a Path before acting on it. This is essential for safe automation.
p.exists()
p.is_file()
p.is_dir()
By reasoning about paths as objects, your code becomes clearer and less errorâprone than when manipulating plain strings.
Exploring directory contentsâ
Automation often starts by understanding what is inside a directory. The iterdir() method yields each entry as a Path object.
from pathlib import Path
logs_dir = Path("logs")
for entry in logs_dir.iterdir():
print(entry)
When you need to work only with files that match a pattern, glob() provides a concise, declarative way to express that requirement.
data_dir = Path("data")
for csv_file in data_dir.glob("*.csv"):
print(csv_file)
This patternâbased traversal is fundamental for batch operations such as processing all images, logs, or documents of a particular type.
Reading and writing filesâ
Automated workflows frequently involve reading data from files and generating new files as output. Path provides highâlevel methods that simplify these operations.
from pathlib import Path
notes_path = Path("notes.txt")
notes_path.write_text("Important notes", encoding="utf-8")
content = notes_path.read_text(encoding="utf-8")
print(content)
These methods handle opening and closing file handles internally, reducing boilerplate and the risk of resourceâhandling errors.
Creating directories safelyâ
Automation scripts often need to ensure that a directory exists before writing files into it. The mkdir() method gives you precise control over directory creation.
from pathlib import Path
reports_dir = Path("reports/2026")
reports_dir.mkdir(parents=True, exist_ok=True)
The parents=True argument creates intermediate directories if they do not exist, and exist_ok=True prevents an error if the directory is already present. This pattern supports idempotent scripts that can be run repeatedly without failure.
Copying, moving, and deleting filesâ
For copying and moving, Python uses the shutil module. These operations must be used carefully, because they modify the filesystem directly.
Copying a file:
from pathlib import Path
import shutil
src = Path("data/report.csv")
dst = Path("backup/report.csv")
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy(src, dst)
Moving a file:
import shutil
shutil.move("incoming/file.txt", "processed/file.txt")
Deleting a file is done with unlink(). It is good practice to check for existence first.
from pathlib import Path
temp_log = Path("temp/output.log")
if temp_log.exists():
temp_log.unlink()
Entire directory trees can be removed with shutil.rmtree(). This should only be used when you are certain the target directory is safe to delete.
import shutil
from pathlib import Path
temp_dir = Path("temp")
if temp_dir.exists():
shutil.rmtree(temp_dir)
Bulk renaming operationsâ
Renaming many files according to a consistent pattern is a common automation task. A deterministic approach ensures predictable results. Sorting the files before renaming guarantees stable ordering.
from pathlib import Path
photos_dir = Path("photos")
for index, file in enumerate(sorted(photos_dir.glob("*.jpg")), start=1):
new_name = f"photo_{index:03d}.jpg"
file.rename(file.with_name(new_name))
This technique is useful when you need uniform naming conventions for further processing or archiving.
Organizing files by extensionâ
A practical automation pattern is to organize files based on their extension. This transforms a cluttered directory into a structured one.
from pathlib import Path
import shutil
downloads = Path("downloads")
for item in downloads.iterdir():
if item.is_file():
ext = item.suffix.lower().lstrip(".") or "no_extension"
target_dir = downloads / ext
target_dir.mkdir(exist_ok=True)
shutil.move(str(item), str(target_dir / item.name))
After running this script, files are grouped into subdirectories named after their extensions, such as pdf, jpg, or no_extension.
Creating timestamped backupsâ
Backups are central to robust automation. A common pattern is to create a directory whose name includes a timestamp, ensuring uniqueness and chronological ordering.
from pathlib import Path
from datetime import datetime
import shutil
source = Path("project")
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
backup_root = Path("backups")
backup_root.mkdir(exist_ok=True)
destination = backup_root / f"project_{timestamp}"
shutil.copytree(source, destination)
This approach allows you to maintain a history of project states that can be restored or inspected later.
Filtering files using metadataâ
The stat() method exposes metadata such as file size and modification time. This enables scripts that act only on files meeting specific criteria.
from pathlib import Path
videos = Path("videos")
limit_bytes = 500 * 1024 * 1024 # 500 MB
for file in videos.glob("*"):
if file.is_file() and file.stat().st_size > limit_bytes:
print("Large file:", file)
Metadataâdriven filtering is useful for cleanup tasks, storage audits, and identifying candidates for archiving or compression.
Principles of safe automationâ
Safe automation is not just about knowing the available functions; it is about how you use them. A robust script anticipates missing files, existing directories, and partial runs. It avoids destructive actions unless they are explicitly intended, prefers moving to an archive over immediate deletion, and is tested on sample data before being applied to important directories. Logging key actionsâsuch as which files were moved, renamed, or deletedâmakes it easier to audit behavior and diagnose issues.
These principles ensure that your filesystem automation increases reliability and efficiency rather than introducing new risks.
Found this helpful? Share the knowledge...
... or support this project to keep it alive!
Previous
Automation & Scripting
Next
Automating Text Processing
- Why filesystem automation matters
- Representing paths with pathlib
- Exploring directory contents
- Reading and writing files
- Creating directories safely
- Copying, moving, and deleting files
- Bulk renaming operations
- Organizing files by extension
- Creating timestamped backups
- Filtering files using metadata
- Principles of safe automation
