Python- Scraping Images Using Requests
I'm unable to save/download the images at the location. I can't figure out the problem although the code seems right. I'm using requests library for scraping the images. import os
Solution 1:
Try wrapping your r = requests.get("https://www.scoopwhoop.com/subreddit-nature/#.lce3tjfci")
in a try:
or while:
statement to make sure that the website you are scraping is returning a 200 response, it could be that the website is timing out or not serving your request.
Solution 2:
import os
from bs4 import BeautifulSoup
import urllib
import requests
import urlparse
from lxml.html import fromstring
r = requests.get("https://www.scoopwhoop.com/subreddit-nature/#.lce3tjfci")
data = r.text
soup = BeautifulSoup(data, "lxml")
for link in soup.find_all('img'):
image = link.get('src')
if bool(urlparse.urlparse(image).netloc):
print(image)
imageName = image[image.rfind("/")+1:]
print(imageName)
urllib.urlretrieve(image,imageName)
Post a Comment for "Python- Scraping Images Using Requests"