Converting a relative url to an absolute one is very important when you are crawling a site, which can allow you to continue to crawl. In this tutorial, we will introduce you how to convert.
Preliminaries
from urllib.parse import urljoin, urldefrag
Create a base url and a relative url
base_url = 'https://www.tutorialexample.com/understand-python-zip-function-python-tutorial/' relative_url = '/calculate-spearmans-correlation-coefficient-for-beginners-numpy-tutorial/#hello'
Convert relative url to absolute
url = urljoin(base_url, relative_url) print (url)
The output is:
https://www.tutorialexample.com/calculate-spearmans-correlation-coefficient-for-beginners-numpy-tutorial/#hello
If you want to remove fragment in url, you can do like this.
Remove fragment in url
ux = urldefrag(url) print(ux) print (ux[0])
The output is:
https://www.tutorialexample.com/calculate-spearmans-correlation-coefficient-for-beginners-numpy-tutorial/