In this tutorial, you’ll learn how to use Pandas to drop an index column. You’ll learn how to do this using the .reset_index() DataFrame method, the .set_index() method, and how to read and write CSV files without an index.
As you create a DataFrame, Pandas will attempt to infer an index column. While many times these indices are relevant, there may times when you simply want to remove the index. Pandas provides a number of helpful ways of doing this, either after a DataFrame is loaded or prior to loading the DataFrame.
The Quick Answer: Use Pandas .reset_index(drop=True) to Drop an Index Column
# Dropping an Index Column in Pandas
import pandas as pd
df = pd.DataFrame.from_dict({
'Name': ['Jane', 'Nik', 'Kate', 'Melissa', 'Evan', 'Doug', 'Joe'],
'Age': [10, 35, 34, 23, 70, 55, 89]
}).set_index('Name')
df = df.reset_index(drop=True)What is a Pandas Index Column?
The Pandas index is analogous to an Excel row number. But simply saying only that would do the index a great disservice. This is because it’s much, much more than a row number. We can think of the row index as being the way in which access a DataFrame’s records – similar to an address or a dictionary’s key.
By default, unless a specific index is passed in, Pandas will simply generate an index for you. This index will start at the number 0 and go up to the length of the DataFrame minus 1. This type of index is called a RangeIndex (as it represents the values from the range function). However, if you’re working with specific data, such as time series data, you may want to index your data by another column.
Technically speaking, the data behind a Pandas Dataframe are backed by a hash table. This is similar to how Python dictionaries perform. Because of this, using an index to locate your data makes it significantly faster than searching across the entire column’s values.
Note: While indices technically exist across the DataFrame columns as well (i.e., along axis 1), when this article refers to an index, I’m only referring to the row index.
Loading a Sample Pandas Dataframe
To follow along with this tutorial, I have provided a sample Pandas DataFrame below. Feel free to copy the code below into your favorite text editor to follow along.
# Loading a Sample Pandas Dataframe
import pandas as pd
df = pd.DataFrame.from_dict({
'Name': ['Jane', 'Nik', 'Kate', 'Melissa', 'Evan', 'Doug', 'Joe'],
'Age': [10, 35, 34, 23, 70, 55, 89],
'Height': [130, 178, 155, 133, 195, 150, 205],
'Weight': [80, 200, 220, 150, 140, 95, 180]
}).set_index('Name')We can take a look at what this DataFrame looks like by printing out the first five records using the df.head() method. This returns the following:
Age Height Weight
Name
Jane 10 130 80
Nik 35 178 200
Kate 34 155 220
Melissa 23 133 150
Evan 70 195 140We can see here that we now have a DataFrame that has an index of Name and three other columns. We used the .set_index() method to set the DataFrame index.
Dropping a Pandas Index Column Using reset_index
The most straightforward way to drop a Pandas DataFrame index is to use the Pandas .reset_index() method. By default, the method will only reset the index, creating a RangeIndex (from 0 to the length of the DataFrame minus 1). The method will also insert the DataFrame index into a column in the DataFrame.
Let’s see what this looks like:
# Resetting a dataframe index with .reset_index()
df = df.reset_index()
print(df.head())
# Returns:
# Name Age Height Weight
# 0 Jane 10 130 80
# 1 Nik 35 178 200
# 2 Kate 34 155 220
# 3 Melissa 23 133 150
# 4 Evan 70 195 140But what if we wanted to drop the DataFrame index and not keep it? We could then pass in the drop=True argument, instructing Pandas to reset the index and to drop the original values. Let’s see what this looks like:
# Drop a Pandas Dataframe index with .reset_index() Method
df = df.reset_index(drop=True)
print(df.head())
# Returns:
# Age Height Weight
# 0 10 130 80
# 1 35 178 200
# 2 34 155 220
# 3 23 133 150
# 4 70 195 140We can see here that the DataFrame’s index is reset to the default behavior and that the original index is completely removed from the DataFrame.
Dropping a Pandas Index a Multi-Index DataFrame
Pandas also allows you to work with multi-index DataFrames, where the DataFrame has more than one column representing the index. This means that each record is represented by two or more unique identifiers. Let’s create a sample MultiIndex DataFrame:
# Creating a MultiIndex DataFrame
import pandas as pd
df = pd.DataFrame.from_dict({
'Gender': ['Female', 'Female', 'Female', 'Male', 'Male', 'Male', 'Male'],
'Name': ['Jane', 'Kate', 'Melissa', 'Nik', 'Evan', 'Doug', 'Joe'],
'Age': [10, 35, 34, 23, 70, 55, 89],
'Height': [130, 178, 155, 133, 195, 150, 205],
'Weight': [80, 200, 220, 150, 140, 95, 180]
}).set_index(['Gender', 'Name'])
print(df.head())
# Returns:
# Age Height Weight
# Name Gender
# Jane Female 10 130 80
# Nik Male 35 178 200
# Kate Female 34 155 220
# Melissa Female 23 133 150
# Evan Male 70 195 140In order to drop both indices, you can simply call .reset_index(drop=True) method. However, if you only want to drop a single index, you need to use the level= parameter. Let’s see how we can drop the 'Gender' index column, while retaining the values:
# Dropping a Single MultiIndex and Keeping Values
import pandas as pd
df = pd.DataFrame.from_dict({
'Gender': ['Female', 'Female', 'Female', 'Male', 'Male', 'Male', 'Male'],
'Name': ['Jane', 'Kate', 'Melissa', 'Nik', 'Evan', 'Doug', 'Joe'],
'Age': [10, 35, 34, 23, 70, 55, 89],
'Height': [130, 178, 155, 133, 195, 150, 205],
'Weight': [80, 200, 220, 150, 140, 95, 180]
}).set_index(['Gender', 'Name'])
df = df.reset_index(level='Gender')
print(df.head())
# Returns:
# Gender Age Height Weight
# Name
# Jane Female 10 130 80
# Kate Female 35 178 200
# Melissa Female 34 155 220
# Nik Male 23 133 150
# Evan Male 70 195 140Similarly, we can drop just one index column and remove all of the values, we can also pass in drop=True, as shown below:
# Dropping a Single MultiIndex and Dropping Values
import pandas as pd
df = pd.DataFrame.from_dict({
'Gender': ['Female', 'Female', 'Female', 'Male', 'Male', 'Male', 'Male'],
'Name': ['Jane', 'Kate', 'Melissa', 'Nik', 'Evan', 'Doug', 'Joe'],
'Age': [10, 35, 34, 23, 70, 55, 89],
'Height': [130, 178, 155, 133, 195, 150, 205],
'Weight': [80, 200, 220, 150, 140, 95, 180]
}).set_index(['Gender', 'Name'])
df = df.reset_index(level='Gender', drop=True)
print(df.head())
# Returns:
# Age Height Weight
# Name
# Jane 10 130 80
# Kate 35 178 200
# Melissa 34 155 220
# Nik 23 133 150
# Evan 70 195 140In the next section, you’ll learn how to use the Pandas .set_index() method to drop a DataFrame’s index in Pandas.
Dropping a Pandas Index Column Using set_index
We can also use a workaround of setting an index with a column that simply mirrors the normal index pattern. We can do this by first creating a column that contains the values from 0 through to the length of the list minus 1. We can do this directly using the .assign() method, which can be used to add a column to a Pandas DataFrame. We then use the .set_index() method to set that new column to the DataFrame’s index.
Let’s see what this looks like:
# Delete a Pandas Dataframe Index with .set_index()
df = df.assign(Index=range(len(df))).set_index('Index')
print(df.head())
# Returns:
# Age Height Weight
# 0 10 130 80
# 1 35 178 200
# 2 34 155 220
# 3 23 133 150
# 4 70 195 140What we’ve done here is first create a column called “Index” by using the .assign() method. We then chain in the .set_index() method to assign this new column to the index. This overwrites and deletes the former index.
In the next section, you’ll learn how to read a CSV file into a Pandas DataFrame without the implied index.
Read a CSV File into a Pandas DataFrame without an Index
You may encounter CSV files that are malformed, such as those that have a delimiter at the end of a given row. These may look like this:
Age,Height,Weight
10,130,80,
35,178,200,
34,155,220,
23,133,150,
70,195,140,
55,150,95,
89,205,180,Because there is a trailing comma, Pandas will incorrectly interpret the first values to be the index values. When we read the file into a DataFrame, it will look like this:
# Reading a malformed .csv file with Pandas
df = pd.read_csv('file.csv')
print(df.head())
# Returns:
# Age Height Weight
# 10 130 80 NaN
# 35 178 200 NaN
# 34 155 220 NaN
# 23 133 150 NaN
# 70 195 140 NaNOf course, this is not what we want. We would want the data to be properly aligned with the columns so that an empty column is returned at the end. Because these files can often be found, Pandas introduced a parameter that allows us to overwrite the default behavior, when using the read_csv function.
Let’s see what happens when we pass in index_col = False into our function:
# Reading a malformed CSV file correctly with Pandas
df = pd.read_csv('file.csv', index_col=False)
print(df.head())
# Returns:
# Age Height Weight
# 0 10 130 80
# 1 35 178 200
# 2 34 155 220
# 3 23 133 150
# 4 70 195 140We can see that by using the index_col=False argument, that Pandas overwrites the default behavior and assigns a proper index.
Conclusion
In this tutorial, you learned how to use Pandas to drop an index column. You learned how to use the Pandas .reset_index() and .set_index() methods to drop an index. You also learned how both read and write a CSV file to a Pandas DataFrame. Being able to work with Pandas indices is a useful skill as you learn how to manipulate data using Pandas.
To learn more about the Pandas .reset_index() method, check out the official documentation here.
Additional Resources
To learn more about similar topics, check out some of these related articles:
I want to seek python language.
You’re in the right place!