File Handling in Python: A Complete Guide

Python provides you with incredibly versatile and powerful ways in which to handle files. Being a general-purpose programming language, completing I/O operations in Python is quite easy. Being able to work with files with the simple and intuitive syntax of Python makes it easy to work with files in many different ways.

In this complete guide, you’ll learn how to use Python for file handling, such as creating and reading files, as well as moving and deleting them. By the end of this guide, you’ll have learned the following:

• How to perform common file operations in Python, such as creating, renaming, copying, and deleting files
• How to read and write files in Python, including working with text files
• How to get file attributes in Python, such as listing files in a directory or checking if files exist

Why Use Python to Work with Files

Python is a general-purpose programming language, meaning that it can be used for a myriad of different tasks. One such area of tasks is being able to work with files. This can include tasks such as creating and moving files, reading data from files, and compressing files using .zip files.

This begs the question, why would you want to use Python to work with files? Being able to work with files programmatically, especially using the simple syntax that Python provides, opens you up to automate repetitive tasks.

Say that you have ten, one hundred, or even a thousand files that are all named by the date that they were created. Now imagine that the dates in all these files are written as "October 31, 2023". While this is easy to read, it makes sorting the files by date really difficult. It may be much better to change the dates to the format "2023-10-31". You can use Python to automatically rename the files for you!

Of course the benefits of working with files goes much beyond being able to rename files. You can use Python to read from and write to files. Similarly, you can use Python to compress files or move them around to organize a folder.

Convinced? Great! Let’s dive into how to use Python for file handling!

Working with Files in Python

Python provides a lot of functionality for working with files. If it seems daunting at first, don’t worry – this guide will walk you through everything you need to know to get started! We’ll start off by looking at how you can work with files, such as .txt files.

By the end of this section, you’ll have learned:

1. How to open files,
2. How to close files,
3. How to write to text files,
4. How to append to text files, and
5. How to read text files

Let’s get started!

How to Open Files with Python

To open a file with Python, you can use the open() function. As the name implies, this function facilitates opening a file. Let’s take a look at this Python open function:

open(
    file,           # The pathname
    mode=’r’,       # The mode to open the file in
    buffering=-1,   # The buffering policy
    encoding=None,  # The encoding used for the file
    errors=None,    # How encoding/decoding errors are handled
    newline=None,   # How to identify new lines
    closefd=True,   # Whether to keep file descriptor open
    opener=None     # Using a custom opener
    )

In this tutorial, we’ll focus on just three of the most important parameters: file=, mode=, and encoding=. When opening a file, we have a number of different options in terms of how to open the file. This is controlled by the mode parameter. Let’s take a look at the various arguments this parameter takes:

Mode	What the mode does
'r'	Open for reading (default)
'w'	Open for writing (first truncates the file)
'x'	Open for exclusive creation (fails if file already exists)
'a'	Open for writing (appends to file if it exists)
'b'	Open in binary mode
't'	Text mode
'+'	Open for updating (reading and writing)

Ok, let’s see how we can open a file in Python. In general, you’ll want to use a context manager to open a file. A context manager allows you to allocate and release resources when you need them. In essence, they allow you to manage memory more safely when working with files.

What does this actually mean? A practical use case of this is that a context manager actually closes the file for your when you’re done working with the file. Normally, you’d need to specify that the file should be closed, but using a context manager, this is handled for you.

# How to Open a File with Python
# Using a context manager to open a file
file_path = '/Users/datagy/Desktop/sample_text.txt'

with open(file_path, 'a') as file:
    ...

Now that the file is open, you can complete many different tasks such as reading data from the file or adding new data to the file. Note that we used the mode of 'a', which let’s you easily append text to a .txt file, as you’ll soon learn!

To learn more about opening files, check out the complete guide to opening files with Python.

How to Close Files with Python

When we open a file, we need to be mindful of closing it as well. As I mentioned earlier, one of the benefits of using a context manager is that Python will automatically close the file for you when the code is done using it.

If you’re not using a context manager, Python must be explicitly told to manage the external resources we pass in. By default, Python will try and retain the resource for as long as possible, even when we’re done using it.

Because of this, we can close the file by using the .close() method:

# Closing a file with .close()
file_path = '/Users/datagy/Desktop/sample_text.txt'

file = open(file_path)
file.close()

Knowing how to do this can be a great tool when you’re working with other people’s code. While you’ve learned that context managers provide great ways in which you can better manage resources, some people may simply open the file. If it’s left open indefinitely, then you may encounter memory issues.

In the following section, you’ll learn how to write to files using Python.

How to Write to Files with Python

Python provides a number of ways to write text to a file, depending on how many lines you’re writing:

• .write() will write a single line to a file
• .writelines() will write multiple lines to a file

These methods allow you to write either a single line at a time or write multiple lines to an opened file. While Python allows you to open a file using the open(), it’s best to use a context manager to more efficiently and safely handle closing the file.

Let’s see what this looks like:

# Writing a Single Line to a Text File
text = 'Welcome to datagy.io!'

with open('/Users/datagy/Desktop/textfile.txt', 'w') as f:
    f.write(text)

Let’s break down what the code above is doing:

1. We load a string that holds our text in a variable text
2. We then use a context manager to open a file in 'w' mode, which allows us to overwrite an existing text
3. The file doesn’t need to exist – if it doesn’t, it’ll automatically be created
4. When then use the .write() method to write our string to the file

Because we’re using a context manager, we don’t explicitly need to close the file.

To learn more about writing to files, check out the in-depth guide to writing to text files in Python.

How to Append to Files with Python

In the previous section, you learned how to write a new file with text in Python. In this section, you’ll learn how to append to a given text file using Python. We previously used the write mode, 'w' when opening the file – in order to append, we use the append mode, 'a'.

Let’s see how we can append to a text file in Python:

# Appending to a Text File in Python
text = 'Welcome to datagy.io!\n'

with open('/Users/nikpi/Desktop/textfile.txt', 'a') as f:
    f.write(text)

Running this will append to the end of the text file. Note that we applied the newline character into the string.

How to Read Files with Python

Python provides a number of helpful ways to read files. For example, you can read a single line at a time. Alternatively, you can read the entire file at once.

Let’s start by reading the entire text file. This can be helpful when you don’t have a lot of content in your file and want to see the entirety of the file’s content. To do this, we use the aptly-named .read() method.

Let’s see how we can use a context manager and the .read() method to read an entire text file in Python:

# Reading an entire text file in Python
file_path = '/Users/datagy/Desktop/sample_text.txt'

with open(file_path) as file:
    print(file.read())

# Returns:
# Hi there!
# Welcome to datagy!
# Today we’re learning how to read text files.
# See you later!

In some cases, your files will be too large to conveniently read all at once. This is where being able to read your file line by line becomes important.

The Python .readline() method returns only a single line at a time. This can be very helpful when you’re parsing the file for a specific line, or simply want to print the lines slightly modified.

Let’s see how we can use this method to print out the file line by line:

# Reading a single line in Python
file_path = '/Users/datagy/Desktop/sample_text.txt'

with open(file_path) as file:
    print(file.readline())

# Returns:
# Hi there!

In the example above, only the first line was returned. We can call the method multiple times in order to print more than one line.

To learn dive deeper into read text files with Python, check out our in-depth guide on using Python to read text files.

Other Ways to Use Python for I/O

Check out these other helpful posts on I/O in Python:

• How to Read a Text File in Python (Python open)
• Python: Count Words in a String or File

Common File Operations with Python

In this section, we’ll explore how to take on common file operations with Python. Because of its widespread utility, Python can help automate tasks related to files themselves. For example, you can use Python to move or copy files. Similarly, you can use Python to rename files or even delete them. You can even use Python to compress and uncompress files from the population .zip format.

Let’s start diving into file operations with Python, first by looking at copying files with Python. For many of these methods, we’ll be relying on a built-in library, shutil. This is an abbreviation for “shell utilities” and helps you automate many file system processes!

How to Copy a File with Python

The shutil.copyfile() function copies a file to another destination file path, meaning that we need to specify not just the destination directory (folder), but also the filename and extension we want to use. This can be very helpful if you want to move and rename the file you’re copying.

Let’s take a look at how we can use the shutil.copyfile() method to copy a file using Python:

# Copy a file using shutil.copyfile()
import shutil

shutil.copyfile('/Users/datagy/Desktop/file.py', '/Users/datagy/Desktop/file2.py')

When we use this function, it’s important to note that if a file already exists in the destination, it will overwrite the existing file. The function also doesn’t copy over the metadata of the file, meaning that you’ll need to recreate some of that if it’s important.

How to Rename a File with Python

To rename a file with Python, we’ll dive into the os library. The shutil library we used above is largely a wrapper for the os library, so it’s good to understand some of the mechanics of this library, too.

Let’s take a look at an example of how to rename a file with Python.

# Rename a File with Python os.rename()
import os

os.rename('/Users/datagy/Desktop/old_name.txt', '/Users/datagy/Desktop/new_name.txt')

In the example above, we were able to pass in the old filename and the one that we want to rename the file to.

How to Move a File with Python

To move a file with Python, we can use the aptly named shutil.move() function. You can simply pass in the file you want to move and the directory that you want to move your file to.

Let’s take a look at what this looks like:

# How to Move a File with shutil.move()
import shutil

old_file = '/Users/datagy/file.txt'
destination = '/Users/datagy/new_folder/'

shutil.move(old_file, destination)

In the example above, we passed in the file that we want to move and the directory to which we want to move it.

How to Delete a File with Python

Deleting a single file using Python is incredibly easy, using the os.remove() function. The os library makes it easy to work with, well, your operating system. Because deleting a file is a common function, the library comes with the .remove() function built in.

What you need to do is simply pass the path of the file into the function, and Python will delete the filter. Be careful though: there is no confirmation prompt to this, so be sure that this is actually what you want to be doing!

# Delete a single file using os
import os
file_path = "/Users/datagy/Desktop/datagy.py"

# Check if the file exists
if os.path.exists(file_path):
    os.remove(file_path)

Let’s explore what we’ve done here:

1. We declared our file path, including an extension. If you’re using Windows and don’t want to escape your backslashes, be sure to make the string a raw string by prepending a letter r to it.
2. We run a conditional expression that uses the os.path.exists() function to check whether a file exists or not.
3. If the file exists, we use the remove() function to pass in the file we want to delete

Now that we’ve covered many different operations with files, let’s dive into working with files and directories as they related to the operating system.

Other Ways to Manipulate Files

Check out these other helpful posts on manipulating files in Python:

• How to zip and unzip a file in Python
• How to Use requirements.txt Files in Python
• Python New Line and How to Print Without Newline

Working with Files and Directories in Python

In the previous sections, you learned how to work with files using Python, including opening and reading them. You also learned how to interact with files by learning how to create, delete, move and rename them.

Now, let’s dive into working with files in how they relate to the operating system. For example, you’ll learn how to list all of the files in a folder or how to get the extension of a file.

How to List All Files in a Directory Using Python

The os.listdir() function generates a list of all files (and directories) in a folder. To use this, simply pass the directory into the function as an argument.

# Create a List of All Files in a Directory
files = os.listdir(file_path)

print(files)
# Returns
# ['November.xlsx', 'October.xlsx', 'Other Files']

We can see that function returns a list of all the filenames in the directory. If you want to include the path to the file, you can use a simple for loop:

# Create a List of All Files (including directories) in a Directory
import os
files_list = []

for root, directories, files in os.walk(file_path):
	for name in files:
		files_list.append(os.path.join(root, name))

print(files_list)

Now, let’s take a look at how you can get the extension of a file using Python.

How to Get the Extension of a File using Python

The os.path module allows us to easily work with our operating system! The path module let’s us use file paths in different ways, including allowing us to get a file’s extension.

The os.path module has a helpful function, splitext(), which allows us to split file-paths into their individual components. Thankfully, splitext() is a smart function that knows how to separate out file extensions, rather than simply splitting a string.

Let’s take a look at how we can use the splitext() function to get a file’s extension:

# Get a file's extension using os.path
import os.path
file_path = "/Users/datagy/Desktop/Important Spreadsheet.xlsx"

extension = os.path.splitext(file_path)[-1]
print(extension)

# Returns: .xlsx

Let’s take a look at what we’ve done here:

1. We import os.path. Rather than writing from os import path, we use this form of import so that we can leave the variable path open and clear.
2. We load our file_path variable. Remember: if you’re Windows, make your file path a raw string, by pre-fixing an r before the opening quotation mark.
3. Apply the splitext() function to the file path. We then access the item’s last item.

Create a Directory (or Folder) with Python

The Python os library allows you to check if a directory exists and, if not, create it. To create a directory in Python, we can use the makedir() function. Let’s take a look at how we can create a directory:

# Creating a Directory in Python
import os
os.mkdir('sample')

The directory will be created if it doesn’t already exist in the path specified. If no path is explicitly stated, the directory will be made in the directory where your script is running.

Other Ways to Work with Directories and Files

Check out these other helpful posts on working with directories and files in Python:

• Python: Check if a File or Directory Exists
• Python: Get and Change the Working Directory
• Python: Get Filename From Path (Windows, Mac & Linux)

Conclusion

In this tutorial, you learned how to use Python to handle files. Since Python is a general purpose programming language, it provides a ton of functionality of working with files. In this guide, you worked through many important aspects of file handling in Python.

First, you took on I/O tasks in Python, including reading and writing to files. As part of this, you learned how to open and close files, as well as the importance and benefits of using context managers. From there, you dove into process of handling the files themselves, by learning how to move, copy, and delete files. Finally, you learned how to work with directories and file types, such as using Python to get a list of files or a file’s extension.

To learn about the modules used, check out their official documentation:

• shutil Official Documentation
• os Official Documentation