User agent

Another important request header is the User-Agent header. Any client that communicates using HTTP can be referred to as a user agent. RFC 7231 suggests that user agents should use the User-Agent header to identify themselves in every request. For example, the user agent if you are using the Chrome browser might be as follows:

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36

Also, we can view the user agent used by the urllib Python version:

>>> from urllib.request import Request
>>> from urllib.request import urlopen
>>> req = Request('http://www.python.org')
 >>> urlopen(req)
<http.client.HTTPResponse object at 0x034AEBF0>
>>> req.get_header('User-agent')
 'Python-urllib/3.7'

Here, we have created a request and submitted it using urlopen, and urlopen added the user agent header to the request. We can examine this header by using the get_header() method. This header and its value are included in every request made by urllib, so every server we make a request to can see that we are using Python 3.7 and the urllib library.