Create an Empty Pandas Dataframe and Append Data

In this post, you’ll learn how to create an empty pandas dataframe and how to add data to them. Specifically, you’ll learn how to create the dataframe, create one with columns, add rows one-by-one and add rows via a loop.

Create an Empty Pandas Dataframe

To start things off, let’s begin by import the Pandas library as pd:

import pandas as pd

Creating a completely empty Pandas Dataframe is very easy. We simply create a dataframe object without actually passing in any data:

df = pd.DataFrame()

print(df)

This returns the following:

Empty DataFrame
Columns: []
Index: []

We can see from the output that the dataframe is empty.

However, we can also check if it’s empty by using the Pandas .empty attribute, which returns a boolean value indicating if the dataframe is empty:

>> print(df.empty)
True

Create an Empty Pandas Dataframe with Columns

There may be time when you know the columns you’ll want in a dataframe, but just don’t have the data for it yet (more on that in appending data to an empty dataframe below).

In order to do this, we can use the columns= parameter when creating the dataframe object to pass in a list of columns. Let’s create a dataframe with the following columns: Name, Age, Birth City, and Gender.

df = pd.DataFrame(columns=['Name', 'Age', 'Birth City', 'Gender'])

print(df)

This prints out the following, indicating that we now have an empty dataframe but with columns attached to it:

Empty DataFrame
Columns: [Name, Age, Birth City, Gender]
Index: []

Create an Empty Pandas Dataframe with Columns and Indices

Similar to the situation above, there may be times when you know both column names and the different indices of a dataframe, but not the data.

We can accomplish creating such a dataframe by including both the columns= and index= parameters. Let’s create the same dataframe as above, but use the Name column as the index and fill in some sample indices:

df = pd.DataFrame(
    columns=['Age', 'Birth City', 'Gender'],
    index=['Jane', 'Melissa', 'John', 'Matt'])

print(df)

This returns the following:

         Age Birth City Gender
Jane     NaN        NaN    NaN
Melissa  NaN        NaN    NaN
John     NaN        NaN    NaN
Matt     NaN        NaN    NaN

Now, technically, this isn’t an empty dataframe anymore. It’s simply a dataframe without data. We can verify this by using the .empty attribute:

print(df.empty)

This returns False.

Add Data to an Empty Dataframe

Now that we have our dataframe with both columns and indices, we can use .loc to add data to it. If you want to learn more about .loc, check out my tutorial here.

Let’s add some data to the record with index Jane:

df.loc['Jane',:] = [23, 'London', 'F']

print(df)

This now returns the following dataframe:

         Age Birth City Gender
Jane      23     London      F
Melissa  NaN        NaN    NaN
John     NaN        NaN    NaN
Matt     NaN        NaN    NaN

Append Data to an Empty Pandas Dataframe

Similar to adding rows one-by-one using the Pandas .loc, we can also use the .append() method to add rows.

The .append() method works by, well, appending a dataframe to another dataframe.

Let’s add the same row above using the append method:

df2 = pd.DataFrame(
    [['Jane', 23, 'London', 'F']], 
    columns=['Name', 'Age', 'Birth City', 'Gender']
    )

df = df.append(df2)

print(df)

This returns the following dataframe:

   Name Age Birth City Gender
0  Jane  23     London      F

To speed things up, we can also use a for loop to add data, as explore below.

Append to Empty Pandas Dataframe with a Loop

There may be times when you need to add multiple pieces of data to a dataframe. This can be simplified using a for loop, to, say, read multiple files and append them. To learn more about Python’s for loops, check out my post here.

In the example below, we’ll just work with different lists, but the method works the same if you read data from multiple iterative files.

We use the ignore_index = True argument to ensure that we create new indices. Otherwise, each index could be duplicated when reading in multiple dataframes.

df = pd.DataFrame(
    columns=['Name', 'Age', 'Birth City', 'Gender'])

people = [
    ['Jane', 23, 'London', 'F'],
    ['Melissa', 45, 'Paris', 'F'],
    ['John', 35, 'Toronto', 'M']
]

for person in people:
    temporary_df = pd.DataFrame([person], columns=['Name', 'Age', 'Birth City', 'Gender'])
    df = df.append(temporary_df, ignore_index=True)

print(df)

This returns the following dataframe:

      Name Age Birth City Gender
0     Jane  23     London      F
1  Melissa  45      Paris      F
2     John  35    Toronto      M

Conclusion

In this post, you learned how to create an empty dataframe, both with and without columns. Following that, you learned how to append data to an empty dataframe, both a single time as well as how to do it with a for loop. To learn more about the Pandas .DataFrame() class, check out the official documentation here. To learn more about the .append() method, check out the official documentation here.