Pandas: Split a Column of Lists into Multiple Columns

In this tutorial, you’ll learn how to split a Pandas DataFrame column that contains lists into multiple columns. Pandas makes working with DataFrames easy, including splitting a single column into multiple columns.

By the end of this tutorial, you’ll have learned how to do the following:

• How to split a column of lists into multiple columns using direct assignments
• How to split a column of lists into multiple columns using the Pandas concat function

The Quick Answer: Use Pandas tolist()

If you’re in a hurry, you can use the code block below. This method is expanded upon in more detail below, but if you just need a quick answer, take a look at the code block below:

# How to Create Pandas Columns from a Column of Lists
df[['Val1', 'Val2', 'Val3']] = pd.DataFrame(df['Values'].tolist())

The code block above assumes that you have a DataFrame column containing lists, where each list contains three different values. We directly assign three new Pandas DataFrame columns.

To this, we assign a DataFrame created by passing the values generated from applying the .tolist() method to a Pandas DataFrame column.

How to Create Pandas Columns from a Column of Lists

In some cases, you’ll end up working with a DataFrame where a single column contains a list of values. Being able to split these columns into multiple columns can be important to support your data analysis.

Loading a Sample Pandas DataFrame

Let’s take a look at a sample DataFrame that contains a list of values in a single column:

import pandas as pd
df = pd.DataFrame({
    'Name': ['Nik', 'Katie', 'Evan'],
    'Values': [[1,2,3], [4,5,6], [7,8,9]]})
print(df)

# Returns:
#     Name     Values
# 0    Nik  [1, 2, 3]
# 1  Katie  [4, 5, 6]
# 2   Evan  [7, 8, 9]

In the code block above, we created and printed a DataFrame. The DataFrame has two columns:

1. Name: contains the names of people
2. Values: contains a list of values. This is the column that we want to split into multiple columns.

We’ll now learn how to split the values in this column into multiple columns.

How to Create Pandas Columns from a Column of Lists

In order to split a column that contains lists of values, we can apply the .tolist() method to our Pandas DataFrame column. This converts our Pandas DataFrame column to a list of lists, which can be shaped into a Pandas DataFrame in itself.

This means that we can assign this DataFrame to a list of columns, as shown below:

# Split a Column of List Values to Columns
import pandas as pd
df = pd.DataFrame({'Name': ['Nik', 'Katie', 'Evan'],'Values': [[1,2,3], [4,5,6], [7,8,9]]})
df[['Val1', 'Val2', 'Val3']] = pd.DataFrame(df['Values'].tolist())

print(df)

# Returns:
#     Name     Values  Val1  Val2  Val3
# 0    Nik  [1, 2, 3]     1     2     3
# 1  Katie  [4, 5, 6]     4     5     6
# 2   Evan  [7, 8, 9]     7     8     9

In the example above, we created a DataFrame after applying the .tolist() method. We then direct assigned this DataFrame to a list of column labels. It’s important that the number of columns match the number of values in the list, otherwise, Pandas will throw a ValueError.

If we want to drop the original column, we can use the Pandas .drop() function. This can be done using the code below:

# Dropping the Original Column
df = df.drop(columns=['Values'])

Let’s now take a look at another method.

Split Column of Lists into Multiple Columns in Pandas by Merging

Similar to the example above, we can create a DataFrame first and then merge it into the original DataFrame. In order to do this, we first create our DataFrame, by assigning it to a variable. Let’s see what this looks like:

# Create a DataFrame by Splitting a Column of Lists to Columns
import pandas as pd
df = pd.DataFrame({'Name': ['Nik', 'Katie', 'Evan'],'Values': [[1,2,3], [4,5,6], [7,8,9]]})
df2 = pd.DataFrame(df['Values'].tolist(), columns=['Val1', 'Val2', 'Val3'], index=df.index)

print(df2)

# Returns:
#    Val1  Val2  Val3
# 0     1     2     3
# 1     4     5     6
# 2     7     8     9

We can see that by creating our DataFrame, it returns a DataFrame containing our values. Note, that we pass in the index of the original DataFrame. While, in this case, it makes no difference, this ensures that we can successfully merge the DataFrame.

Let’s see how we can merge the two DataFrames in order to combine the data using the pd.concat() function. We concatenate the DataFrames along the first axis, which merges the DataFrames along the row indices.

# Merge the Two DataFrames Together
import pandas as pd
df = pd.DataFrame({'Name': ['Nik', 'Katie', 'Evan'],'Values': [[1,2,3], [4,5,6], [7,8,9]]})
df2 = pd.DataFrame(df['Values'].tolist(), columns=['Val1', 'Val2', 'Val3'])
df = pd.concat([df, df2], axis=1)

print(df)

# Returns:
#     Name     Values  Val1  Val2  Val3
# 0    Nik  [1, 2, 3]     1     2     3
# 1  Katie  [4, 5, 6]     4     5     6
# 2   Evan  [7, 8, 9]     7     8     9

We can see that this returns the same result as the previous example, ensuring that our column of values is split into multiple columns.

If we want to drop the original column, we can use the Pandas .drop() function.

Conclusion

So, which approach is the best on to accomplish this goal? Personally, I prefer the first approach. Not only is it more concise, but it’s also more explicit. It removes some ambiguity for future readers of your code.

In this tutorial, you learned how to convert a Pandas DataFrame column which contains a list of values into multiple columns. This can be helpful when working from a dataset from different sources, such as a database or the web.

To learn more about the Pandas tolist method, check out the official documentation. The method provides helpful functionality when working with Pandas DataFrames and DataFrame columns.