How to download image using requests

I'm trying to download and save an image from the web using python's requests module.

requests

Here is the (working) code I used:

img = urllib2.urlopen(settings.STATICMAP_URL.format(**data))
with open(path, 'w') as f:
f.write(img.read())


Here is the new (non-working) code using requests:

requests

r = requests.get(settings.STATICMAP_URL.format(**data))
if r.status_code == 200:
img = r.raw.read()
with open(path, 'w') as f:
f.write(img)


Can you help me on what attribute from the response to use from requests?

requests

10 Answers
10

You can either use the response.raw file object, or iterate over the response.

response.raw

To use the response.raw file-like object will not, by default, decode compressed responses (with GZIP or deflate). You can force it to decompress for you anyway by setting the decode_content attribute to True (requests sets it to False to control decoding itself). You can then use shutil.copyfileobj() to have Python stream the data to a file object:

response.raw

decode_content

True

requests

False

shutil.copyfileobj()

import requests
import shutil

r = requests.get(settings.STATICMAP_URL.format(**data), stream=True)
if r.status_code == 200:
with open(path, 'wb') as f:
r.raw.decode_content = True
shutil.copyfileobj(r.raw, f)


To iterate over the response use a loop; iterating like this ensures that data is decompressed by this stage:

r = requests.get(settings.STATICMAP_URL.format(**data), stream=True)
if r.status_code == 200:
with open(path, 'wb') as f:
for chunk in r:
f.write(chunk)


This'll read the data in 128 byte chunks; if you feel another chunk size works better, use the Response.iter_content() method with a custom chunk size:

Response.iter_content()

r = requests.get(settings.STATICMAP_URL.format(**data), stream=True)
if r.status_code == 200:
with open(path, 'wb') as f:
for chunk in r.iter_content(1024):
f.write(chunk)


Note that you need to open the destination file in binary mode to ensure python doesn't try and translate newlines for you. We also set stream=True so that requests doesn't download the whole image into memory first.

stream=True

requests

r2 = requests.post(r.url, data); print r2.content

filename

r2.headers['content-disposition']

'attachment; filename=DELS36532G290115.csi'

content-disposition

cgi.parse_header()

params = cgi.parse_header(r2.headers['content-disposition'])[1]

params['filename']

requests.Response

for chunk in r: ...

iter_content()

chunk_size

response.ok

Get a file-like object from the request and copy it to a file. This will also avoid reading the whole thing into memory at once.

import shutil

import requests

url = 'http://example.com/img.png'
response = requests.get(url, stream=True)
with open('img.png', 'wb') as out_file:
shutil.copyfileobj(response.raw, out_file)
del response


r.raw.decode_content = True

shutil.copyfileobj(response.raw, out_file)

by default, decode compressed responses (with GZIP or deflate)

How about this, a quick solution.

import requests

url = "http://craphound.com/images/1006884_2adf8fc7.jpg"
response = requests.get(url)
if response.status_code == 200:
with open("/Users/apple/Desktop/sample.jpg", 'wb') as f:
f.write(response.content)


f = open("/Users/apple/Desktop/sample.jpg", 'wb')

I have the same need for downloading images using requests. I first tried the answer of Martijn Pieters, and it works well. But when I did a profile on this simple function, I found that it uses so many function calls compared to urllib and urllib2.

I then tried the way recommended by the author of requests module:

import requests
from PIL import Image
from StringIO import StringIO

r = requests.get('https://example.com/image.jpg')
i = Image.open(StringIO(r.content))


This much more reduced the number of function calls, thus speeded up my application.
Here is the code of my profiler and the result.

#!/usr/bin/python
import requests
from StringIO import StringIO
from PIL import Image
import profile

def testRequest():
image_name = 'test1.jpg'
url = 'http://example.com/image.jpg'

r = requests.get(url, stream=True)
with open(image_name, 'wb') as f:
for chunk in r.iter_content():
f.write(chunk)

def testRequest2():
image_name = 'test2.jpg'
url = 'http://example.com/image.jpg'

r = requests.get(url)

i = Image.open(StringIO(r.content))
i.save(image_name)

if __name__ == '__main__':
profile.run('testUrllib()')
profile.run('testUrllib2()')
profile.run('testRequest()')


The result for testRequest:

343080 function calls (343068 primitive calls) in 2.580 seconds


And the result for testRequest2:

3129 function calls (3105 primitive calls) in 0.024 seconds


chunk_size

iter_content

PIL

with open(image_name, 'wb') as outfile: outfile.write(r.content)

PIL

iter_content

chunk_size

This might be easier than using requests. This is the only time I'll ever suggest not using requests to do HTTP stuff.

requests

requests

Two liner using urllib:

urllib

>>> import urllib
>>> urllib.urlretrieve("http://www.example.com/songs/mp3.mp3", "mp3.mp3")


There is also a nice Python module named wget that is pretty easy to use. Found here.

wget

This demonstrates the simplicity of the design:

>>> import wget
>>> url = 'http://www.futurecrew.com/skaven/song_files/mp3/razorback.mp3'
>>> filename = wget.download(url)
100% [................................................] 3841532 / 3841532>
>> filename
'razorback.mp3'


Enjoy.

Edit: You can also add an out parameter to specify a path.

out

>>> out_filepath = <output_filepath>
>>> filename = wget.download(url, out=out_filepath)


Following code snippet downloads a file.

The file is saved with its filename as in specified url.

import requests

url = "http://beispiel.dort/ichbineinbild.jpg"
filename = url.split("/")[-1]
r = requests.get(url, timeout=0.5)

if r.status_code == 200:
with open(filename, 'wb') as f:
f.write(r.content)


There are 2 main ways:

Using .content (simplest/official) (see Zhenyi Zhang's answer):

.content

import io # Note: io.BytesIO is StringIO.StringIO on Python2.
import requests

r = requests.get('http://lorempixel.com/400/200')
r.raise_for_status()
with io.BytesIO(r.content) as f:
with Image.open(f) as img:
img.show()


Using .raw (see Martijn Pieters's answer):

.raw

import requests

r = requests.get('http://lorempixel.com/400/200', stream=True)
r.raise_for_status()
r.raw.decode_content = True # Required to decompress gzip/deflate compressed responses.
with PIL.Image.open(r.raw) as img:
img.show()
r.close() # Safety when stream=True ensure the connection is released.


Timing both shows no noticeable difference.

1.

io.BytesIO

Image

from PIL import Image

pip install Pillow

Here is a more user-friendly answer that still uses streaming.

Just define these functions and call getImage(). It will use the same file name as the url and write to the current directory by default, but both can be changed.

getImage()

import requests
from StringIO import StringIO
from PIL import Image

def createFilename(url, name, folder):
dotSplit = url.split('.')
if name == None:
# use the same as the url
slashSplit = dotSplit[-2].split('/')
name = slashSplit[-1]
ext = dotSplit[-1]
file = '{}{}.{}'.format(folder, name, ext)
return file

def getImage(url, name=None, folder='./'):
file = createFilename(url, name, folder)
with open(file, 'wb') as f:
r = requests.get(url, stream=True)
for block in r.iter_content(1024):
if not block:
break
f.write(block)

def getImageFast(url, name=None, folder='./'):
file = createFilename(url, name, folder)
r = requests.get(url)
i = Image.open(StringIO(r.content))
i.save(file)

if __name__ == '__main__':
# Uses Less Memory
getImage('http://www.example.com/image.jpg')
# Faster
getImageFast('http://www.example.com/image.jpg')


The request guts of getImage() are based on the answer here and the guts of getImageFast() are based on the answer above.

request

getImage()

getImageFast()

I'm going to post an answer as I don't have enough rep to make a comment, but with wget as posted by Blairg23, you can also provide an out parameter for the path.

wget.download(url, out=path)


when I try to run the below code,the image is getting downaloded but the size is always confined to 34 KB.

import requests
import shutil

r = requests.get(settings.STATICMAP_URL.format(**data), stream=True)
if r.status_code == 200:
with open(path, 'wb') as f:
r.raw.decode_content = True
shutil.copyfileobj(r.raw, f)


And aslo please let me know what is settings.STATICMAP_URL.format(**data),I'm using my usl in place of settings.STATICMAP_URL.format(**data)

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.