Experimental support for the requests package
I've just pushed a branch of the latest version of libcloud using the popular
requests package by Kenneth Reitz instead of our home-rolled HTTP client library.
This article is for both users and developers of libcloud. If you want to give feedback, please join the developer mailing list.
- requests is the defacto standard - it would be in the standard library but agreed against to allow it to develop faster https://github.com/kennethreitz/requests/issues/2424
- it works with python 2.6->3.5
- Our SSL experience has a lot to be desired for Windows users, having to download the CA cert package and setting environment variables just to get SSL working
- Developers can use requests_mock for deeper integration testing
- less code to maintain
- the role of libcloud is for cloud abstraction, we provide no value in writing and maintaining our own HTTP client library
Benefits of requests
There are a number of benefits to having a requests package
- The client library code is smaller, leaner and simpler.
- Requests has built in decompression support, we no longer need to support this
- Requests has built in RAW download, upload support, helping with our storage drivers
Implications of the change
- There are no longer 2 classes (
LibcloudHTTPConnection) to be provided to each driver, they are now 1 class -
LibcloudConnection. You probably won't notice this because it is a property of the
Connectionclass, but if you are developing or extending functionality then it is implicated.
- Unit tests will look slightly different (see below)
- This change broke 4200 unit tests (out of 6340)! I've since fixed them all since they were coupled to the original implementation, but now I don't know if all of tests are valid.
Testing with requests
Unit tests that were written like this:
class DigitalOceanTests(LibcloudTestCase): def setUp(self): DigitalOceanBaseDriver.connectionCls.conn_classes = \ (None, DigitalOceanMockHttp) DigitalOceanMockHttp.type = None self.driver = DigitalOceanBaseDriver(*DIGITALOCEAN_v1_PARAMS)
Because of the change have been modified to (I updated all of them - so this is just for future reference)
class DigitalOceanTests(LibcloudTestCase): def setUp(self): DigitalOceanBaseDriver.connectionCls.conn_class = DigitalOceanMockHttp DigitalOceanMockHttp.type = None self.driver = DigitalOceanBaseDriver(*DIGITALOCEAN_v1_PARAMS)
Check it out!
The package is on my personal apache site, you can download it and install it in a virtualenv for testing.
pip install -e http://people.apache.org/~anthonyshaw/libcloud/1.0.0-rc2-requests/apache-libcloud-1.0.0-rc2-requests.zip@feature#egg=apache-libcloud
The hashes are my apache space
Have a look at the PR and the change set for a list of changes
What might break?
What I'm really looking for is for users of Libcloud to take 15 minutes, an existing (working) libcloud script, install this package in a virtualenv and just validate that there are no regression bugs with this change.
I'm particularly sceptical about the storage drivers.
Once we have enough community feedback, we will propose a vote to merge this into trunk for future release.
Credit to dz0ny on IRC for contributing some of the requests patch.