Python requests: GET Request Explained

In this tutorial, you’ll learn how to use the Python requests library’s get method to fetch data via HTTP. The Python requests library abstracts the complexities in making HTTP requests. The requests.get() method allows you to fetch an HTTP response and analyze it in different ways.

By the end of this tutorial, you’ll have learned:

• How the Python requests get method works
• How to customize the Python requests get method with headers
• How to use the Python response objects

Understanding the Python requests get Function

An HTTP GET request is used to retrieve data from the specified resource, such as a website. When using the Python requests library, you can use the .get() function to create a GET request for a specified resource.

The function accepts a number of different parameters. Let’s take a look at the function and the different parameters that it accepts:

# Understand the Python requests.get() Function
import requests
req = requests.get(
   url,
   params=None,
   **kwargs
)

We can see in the code block above, that the function accepts two parameters:

1. A url, which points to a resource, and
2. params, which accepts a dictionary or tuples to send in the query string

The .get() function is actually a convenience function based off of the .request() function. The optional keyword arguments that you can pass in derive from that function. Let’s take a look at the options that you can pass in:

Parameter	Description	Default Value
allow_redirects	Whether or not to allow redirection	True
auth	A tuple to enable secure HTTP authentication	None
cert	A string or tuple specifying the certification file or key	None
cookies	A dictionary of cookies to send to the specified URL	None
headers	A dictionary of HTTP headers to send	None
proxies	A dictionary of the protocol of the proxy URL	None
stream	A boolean indicator to determine if the response should be downloaded (when set to False) or streamed (when set to True)	False
timeout	A number or tuple indicating how many seconds to wait for a client to make a connection or send a response. The default value of None means that the request will wait indefinitely.	None
verify	A boolean or string indication to test the server’s TLS certificate	True

Now that you have a strong understanding of the Python requests.get() function, let’s take a look at how you can make a GET request.

Making a Python requests GET Request

In this section, you’ll learn how to use the requests.get() function to make a GET request. For this tutorial, we’ll use a mock API that allows you to emulate a real-world scenario. For this, we’ll query the /users endpoint of the URL, by accessing the URL https://reqres.in/api/users.

Let’s see how we can use the get() function to make a GET request:

# Making a GET Request with requests.get()
import requests

resp = requests.get('https://reqres.in/api/users')
print(resp)

# Returns:
# <Response [200]>

Let’s break down what we did in the code above:

1. We imported the requests library
2. We created a new variable, resp, which is the result of passing our endpoint into the get() function
3. We then printed out the variable, which returned a Response object

In the following section, you’ll learn how to make use of this Response object to retrieve response information.

Understanding the Python requests response Object

In the previous section, you learned how to create a Response object. The Response object has a number of different properties and methods that we can access to learn more about the data that has been returned.

Some of the key properties and methods available to a Response object include:

• .status_code, which is the integer code of the responded HTTP status, such as 200 or 404
• .ok, which return True is the status code of the response is less than 400, otherwise False
• .json(), which returns the JSON-encoded content of a response
• .content, which contains the content of the response
• .next, which returns a request for the new request in a redirect chain, if there is one

Let’s see how we can use some of these properties and methods to understand the response we get back from our API call:

# Understanding the Response Object
import requests

resp = requests.get('https://reqres.in/api/users')
print(resp.status_code)

# Returns: 200

In the code block above, we checked what the .status_code of the Response object was. Since it was in the 200s, we can be confident that an acceptable request was made.

Let’s take a look at how we can understand the data that has been returned:

# Converting the Response Object to a Dictionary
import requests

resp = requests.get('https://reqres.in/api/users')
print(resp.json())

# Returns: 
# {'page': 1, 'per_page': 6, 'total': 12, 'total_pages': 2, 'data': [{'id': 1, 'email': '[email protected]', 'first_name': 'George', 'last_name': 'Bluth', 'avatar': 'https://reqres.in/img/faces/1-image.jpg'}, {'id': 2, 'email': '[email protected]', 'first_name': 'Janet', 'last_name': 'Weaver', 'avatar': 'https://reqres.in/img/faces/2-image.jpg'}, {'id': 3, 'email': '[email protected]', 'first_name': 'Emma', 'last_name': 'Wong', 'avatar': 'https://reqres.in/img/faces/3-image.jpg'}, {'id': 4, 'email': '[email protected]', 'first_name': 'Eve', 'last_name': 'Holt', 'avatar': 'https://reqres.in/img/faces/4-image.jpg'}, {'id': 5, 'email': '[email protected]', 'first_name': 'Charles', 'last_name': 'Morris', 'avatar': 'https://reqres.in/img/faces/5-image.jpg'}, {'id': 6, 'email': '[email protected]', 'first_name': 'Tracey', 'last_name': 'Ramos', 'avatar': 'https://reqres.in/img/faces/6-image.jpg'}], 'support': {'url': 'https://reqres.in/#support-heading', 'text': 'To keep ReqRes free, contributions towards server costs are appreciated!'}}

In the code above, we converted our Response object to a Python dictionary, by applying the .json() method to our object. Because of this, we can now access items in the response by using their key:

# Getting a Response Item from the Dictionary
import requests

resp = requests.get('https://reqres.in/api/users')
response_dict = resp.json()
print(response_dict.get('page'))

# Returns: 
# 1

In the next section, you’ll learn how to pass headers into a Python GET request.

Passing Headers into a Python requests get Method

In many cases, when accessing APIs or other web data, you’ll need to provide some form of authentication. This can be done by passing in information via headers. Headers can also be used to pass in other forms of information, such as the content type you’d like returned.

Let’s see how we can pass in a mock API key into our query:

# Passing in Sample Headers Into Our Request
import requests
headers = {"Authorization": "abc123"}
resp = requests.get('https://reqres.in/api/users', headers=headers)

Let’s break down what the code above does:

1. We create a dictionary of headers, headers, that contains key-value pairs of information to pass into our request.
2. We use the headers= parameter to pass in our dictionary into our request.

The types of headers that will be accepted will vary depending on the web resource that you’re accessing.

In the following section, you’ll learn how to use query string parameters in a Python requests get() function.

Using Query String Parameters in a Python requests get Method

Query string parameters allow you to customize a GET request by passing values directly into the URL or into the params= parameter.

Before diving further, you may be wondering why you’d want to do this. Take a look at the data that has been returned by our previous API call. The data indicates that six records were returned. It also indicates that the first of two pages was returned. So, how do we access the second page?

The answer to this, as you may have guessed, is to use query parameters. In order to access the second page of our data, we can pass in that we want to access page #2. There are three ways to accomplish this:

1. Pass in a dictionary of parameters into the params= parameter, such as {'page': '2'}
2. Pass in a list of tuples into the params= parameter, such as [('page', '2')]
3. Directly modify the URL string by appending ?page=2 to the URL

Let’s see how we can use the first method to accomplish this:

# Passing in Query Parameters Into Our GET Request
import requests
params = {'page': '2'}
resp = requests.get('https://reqres.in/api/users', params=params)

print(resp.content)

# Returns:
# b'{"page":2,"per_page":6,"total":12,"total_pages":2,"data":[{"id":7,"email":"[email protected]","first_name":"Michael","last_name":"Lawson","avatar":"https://reqres.in/img/faces/7-image.jpg"},{"id":8,"email":"[email protected]","first_name":"Lindsay","last_name":"Ferguson","avatar":"https://reqres.in/img/faces/8-image.jpg"},{"id":9,"email":"[email protected]","first_name":"Tobias","last_name":"Funke","avatar":"https://reqres.in/img/faces/9-image.jpg"},{"id":10,"email":"[email protected]","first_name":"Byron","last_name":"Fields","avatar":"https://reqres.in/img/faces/10-image.jpg"},{"id":11,"email":"[email protected]","first_name":"George","last_name":"Edwards","avatar":"https://reqres.in/img/faces/11-image.jpg"},{"id":12,"email":"[email protected]","first_name":"Rachel","last_name":"Howell","avatar":"https://reqres.in/img/faces/12-image.jpg"}],"support":{"url":"https://reqres.in/#support-heading","text":"To keep ReqRes free, contributions towards server costs are appreciated!"}}'

We can see that by passing in our query parameters our query returned the second page.

Conclusion

In this tutorial, you learned how to make GET requests using the Python library, requests. You first learned how the get() function works, including how to access additional parameters via keyword arguments.

Then, you learned how to use the get() function to make a request and how to access properties and use methods of the returned Response object. Finally, you learned how to apply headers and query parameters to your GET request.

Additional Resources

To learn more about related topics, check out the tutorials below:

• Python requests: POST Request Explained
• Python requests Response Object Explained
• How to Install requests Package in Python – Windows, macOS, and Linux