Working with files and folders

Understanding Files and Folders in Automation

When you automate tasks, you will very often need to:

• Find files
• Create new files and folders
• Move, copy, or delete files
• Rename files
• Work with paths (file locations)

This chapter focuses on doing these things with Python so your scripts can work on many files automatically.

You will mainly use two standard libraries:

• os and os.path (older but still very common)
• pathlib (modern, object‑oriented way to handle paths)

You don’t need to install these; they are part of Python.

File Paths

A file path tells Python where a file or folder lives on your computer.

Examples (they may look different on your system):

• Windows: C:\Users\Alex\Documents\report.txt
• macOS / Linux: /home/alex/documents/report.txt

To write paths in Python strings:

• Use raw strings on Windows so backslashes don’t become escape characters:
• r"C:\Users\Alex\Documents\report.txt"
• Or just use forward slashes (Python on Windows understands them):
• "C:/Users/Alex/Documents/report.txt"

Using pathlib:

from pathlib import Path
path = Path("C:/Users/Alex/Documents/report.txt")
print(path)

Current Working Directory

The current working directory (CWD) is the folder your script is “standing in” when it runs. Relative paths are based on this.

Using os:

import os
print(os.getcwd())          # Show current working directory
os.chdir("C:/Users/Alex")   # Change current working directory
print(os.getcwd())

Using pathlib:

from pathlib import Path
current = Path.cwd()
print(current)
# Change working directory (less common; prefer absolute paths when you can)
import os
os.chdir(current / "Documents")
print(Path.cwd())

Whenever possible, prefer full (absolute) paths in automation scripts so they are more predictable.

Listing Files and Folders

You will often need to know what files exist in a directory.

Using `os.listdir`

import os
folder = "C:/Users/Alex/Documents"
entries = os.listdir(folder)    # Names only, not full paths
for name in entries:
    print(name)

os.listdir returns both files and folders.

Using `os.scandir` (gets more info)

import os
folder = "C:/Users/Alex/Documents"
with os.scandir(folder) as it:
    for entry in it:
        print(entry.name, "->", "dir" if entry.is_dir() else "file")

Using `pathlib.Path.iterdir`

from pathlib import Path
folder = Path("C:/Users/Alex/Documents")
for item in folder.iterdir():
    print(item, "->", "dir" if item.is_dir() else "file")

Checking If Files or Folders Exist

Before reading, writing, or deleting, you often need to check if something exists.

With `os.path`

import os
path = "C:/Users/Alex/Documents/report.txt"
print(os.path.exists(path))   # True or False
print(os.path.isfile(path))   # True if it's a file
print(os.path.isdir(path))    # True if it's a directory

With `pathlib`

from pathlib import Path
path = Path("C:/Users/Alex/Documents/report.txt")
print(path.exists())
print(path.is_file())
print(path.is_dir())

In automated scripts, use these checks to avoid crashes and to decide what to do next.

Creating Folders

If your script needs to save output somewhere, it may need to create folders first.

Using `os.mkdir` and `os.makedirs`

import os
# Create a single folder
os.mkdir("output")   # Fails if 'output' already exists
# Create nested folders
os.makedirs("logs/2025/12", exist_ok=True)
# exist_ok=True means "don't crash if it already exists"

Using `pathlib.Path.mkdir`

from pathlib import Path
folder = Path("output")
folder.mkdir(exist_ok=True)  # Create 'output' if needed
nested = Path("logs/2025/12")
nested.mkdir(parents=True, exist_ok=True)
# parents=True allows creating all missing parent folders

Building and Joining Paths

Avoid hard-coding paths by string concatenation like "folder/" + filename. Different systems use different separators, so use the tools Python gives you.

With `os.path.join`

import os
folder = "C:/Users/Alex/Documents"
filename = "report.txt"
full_path = os.path.join(folder, filename)
print(full_path)

With `pathlib` (recommended)

The / operator builds paths in a cross‑platform way:

from pathlib import Path
folder = Path("C:/Users/Alex/Documents")
filename = "report.txt"
full_path = folder / filename
print(full_path)

Renaming and Moving Files

Renaming and moving are closely related: moving is often just renaming with a different path.

Using `os.rename`

import os
# Rename in the same folder
os.rename("old_name.txt", "new_name.txt")
# Move file to another folder (and possibly rename)
os.rename("report.txt", "archive/2025_report.txt")

If source or destination doesn’t exist, you’ll get an error, so use os.path.exists if needed.

Using `pathlib.Path.rename`

from pathlib import Path
file = Path("report.txt")
new_file = Path("archive/2025_report.txt")
file.rename(new_file)

Copying Files

To copy files (not just move them), use the shutil module.

import shutil
# Copy file to another folder (keeps same name)
shutil.copy("report.txt", "backup/")
# Copy and rename
shutil.copy("report.txt", "backup/report_backup.txt")

For full directory copies, there is shutil.copytree, but be careful: it usually expects the destination directory not to exist.

import shutil
shutil.copytree("data", "data_backup")

Deleting Files and Folders

Automation often needs cleanup: removing old temporary files or generated folders. Always be careful when deleting.

Deleting Files

Using os.remove:

import os
if os.path.exists("old_log.txt"):
    os.remove("old_log.txt")

Using pathlib.Path.unlink:

from pathlib import Path
path = Path("old_log.txt")
if path.exists():
    path.unlink()

Deleting Empty Folders

Using os.rmdir:

import os
os.rmdir("empty_folder")   # Only works if the folder is empty

Using pathlib:

from pathlib import Path
folder = Path("empty_folder")
folder.rmdir()             # Also requires folder to be empty

Deleting Non‑Empty Folders

To delete a folder and everything inside it, use shutil.rmtree. This is powerful and dangerous—double‑check paths.

import shutil
shutil.rmtree("old_project")   # Deletes folder and all contents

In many real scripts, instead of deleting, you might move items to an “archive” or “trash” folder to avoid accidental loss.

Recursively Walking Through Folders

Sometimes you need to process all files in a folder and its subfolders, such as “find all .txt files under documents”.

Using `os.walk`

os.walk goes through a directory tree for you.

import os
root_folder = "C:/Users/Alex/documents"
for current_folder, subfolders, files in os.walk(root_folder):
    print("Current folder:", current_folder)
    print("Subfolders:", subfolders)
    print("Files:", files)
    print()

You can then do something with each file:

import os
root_folder = "C:/Users/Alex/documents"
for current_folder, _, files in os.walk(root_folder):
    for name in files:
        if name.endswith(".txt"):
            full_path = os.path.join(current_folder, name)
            print("Found text file:", full_path)

Using `pathlib.rglob` / `glob`

pathlib has convenient ways to search for patterns like “all .txt files”.

from pathlib import Path
root = Path("C:/Users/Alex/documents")
for txt_file in root.rglob("*.txt"):
    print("Found:", txt_file)

This is very useful for automation, such as:

• Convert all .txt files to another format
• Compress all .log files
• Rename all .jpg files in a folder tree

---

Practical Automation Examples with Files and Folders

Here are a few small patterns that often appear in scripts.

Example 1: Organizing Files by Extension

Sort files in a folder into subfolders like images, documents, etc.

from pathlib import Path
import shutil
downloads = Path("C:/Users/Alex/Downloads")
# Create target folders
images = downloads / "images"
docs = downloads / "documents"
others = downloads / "others"
for folder in (images, docs, others):
    folder.mkdir(exist_ok=True)
for item in downloads.iterdir():
    if item.is_file():
        if item.suffix.lower() in {".png", ".jpg", ".jpeg", ".gif"}:
            shutil.move(str(item), images / item.name)
        elif item.suffix.lower() in {".pdf", ".docx", ".txt"}:
            shutil.move(str(item), docs / item.name)
        else:
            shutil.move(str(item), others / item.name)

Example 2: Adding a Prefix to Many Filenames

Rename all .txt files by adding processed_ in front of the name.

from pathlib import Path
folder = Path("C:/Users/Alex/reports")
for file in folder.iterdir():
    if file.is_file() and file.suffix == ".txt":
        new_name = "processed_" + file.name
        file.rename(folder / new_name)

Example 3: Cleaning Up Old Log Files

Delete all .log files in a folder that are larger than a certain size (e.g., over 1 MB).

from pathlib import Path
logs = Path("C:/Users/Alex/logs")
max_size = 1_000_000   # bytes (~1 MB)
for log_file in logs.glob("*.log"):
    if log_file.stat().st_size > max_size:
        print("Deleting:", log_file)
        log_file.unlink()

Safety Tips for File and Folder Automation

When your script can modify many files quickly, mistakes can be costly. A few habits help prevent problems:

1. Test with print statements first

Before deleting or moving, print what you would do.

   # "Dry run" – don't actually delete yet
   for path in folder.rglob("*.tmp"):
       print("Would delete:", path)

2. Work on a copy

At the beginning, practice on a test directory, not your real documents.

3. Use clear, absolute paths

Avoid vague paths like "./" unless you are sure of the current working directory.

4. Back up important data

For scripts that run regularly (e.g., scheduled tasks), create backups or archives instead of immediate deletion.

Summary

In automation scripts, working with files and folders is central. In this chapter you learned how to:

• Inspect where you are (os.getcwd, Path.cwd) and work with file paths
• List files and folders in a directory
• Check whether files or folders exist
• Create new folders (including nested ones)
• Join paths safely and portably (os.path.join, Path / "file")
• Rename, move, copy, and delete files and folders
• Walk through directory trees and find files matching patterns
• Apply these tools in small automation tasks while staying safe

These techniques combine naturally with reading and writing files and with scheduling your scripts, so you can build powerful, fully automated workflows.