We are pleased to announce the release of Libcloud 1.2.1.

This release brings many new features, improvements, bug-fixes, and drivers.

Release highlights

Changes in Apache Libcloud 1.2.1

• Fix issue enabling backups on Dimension Data driver

Changes in Apache Libcloud 1.2.0

General

• Fix caching of auth tokens in the Google Compute Engine drivers. Now we make sure that the file is truncated before writing a new token. Not truncating the file would cause issues if the new token is shorted then the existing one which is cached in the file.

Compute

• Fix image undeprecation in GCE
• Added Managed Instance Groups in GCE
• Allow undeprecation of an image in GCE
• BUGFIX Values with wildcards failed signature validation in cloudstack
• Added StorageState-Migrating to the cloudstack driver.
• Update copy image logic to match create image in GCE driver.
• Removed HD attribute from the Abiquo compute driver to support the 3.4 API
• Add image and size details to list_nodes response in Dimension Data driver
• Add support for changing VM admin password in VMware driver
• Add Barcelona (Spain) region to the Aurora Compute driver.
• Various improvements in the libvirt driver.

Load balancer

• Add support for temporary IAM role credentials (token) to the AWS ELB driver.

DNS

• Updated the ‘extra’ parameter in update_record() to be optional in aurora driver
• Support for iterating over records and zones in the Aurora DNS driver
• Add support for DS, PTR, SSFHFP and TLSA record type to the Aurora DNS driver.

Container

• Add network mode and labels when creating containers within docker driver

Storage

• Fix authentication issue in S3/China region, disabled multipart uploads as not supported by region.

Full change log can be found at here.

Special thank you

I would like to wish a special thank you to all of our community contributors for their ongoing support to the project.

Download

The release can can be downloaded from https://libcloud.apache.org/downloads.html or installed using pip:

pip install apache-libcloud==1.2.1

Upgrading

If you have installed Libcloud using pip you can also use it to upgrade it:

pip install --upgrade apache-libcloud==1.2.1

Upgrade notes

A page which describes backward incompatible or semi-incompatible changes and how to preserve the old behavior when this is possible can be found at https://libcloud.readthedocs.org/en/latest/upgrade_notes.html

Documentation

Regular and API documentation is available at https://libcloud.readthedocs.org/en/latest/

Bugs / Issues

If you find any bug or issue, please report it on our issue tracker https://issues.apache.org/jira/browse/LIBCLOUD. Don’t forget to attach an example and / or test which reproduces your problem.

Thanks

Thanks to everyone who contributed and made this release possible! Full list of people who contributed to this release can be found in the CHANGES file.

We are pleased to announce the release of Libcloud 1.1.0.

This is a first release after v1.0.0. It includes various improvements and bug-fixes.

Release highlights

• Support for automatic SNI (SSL extension) using the hostname supplied to connect to.
• Various improvements in the libvirt driver
• Various improvements in the DimensionData driver
• Various improvements in the Aliyun driver

Full change log can be found at here.

Download

The release can can be downloaded from https://libcloud.apache.org/downloads.html or installed using pip:

pip install apache-libcloud==1.1.0

Upgrading

If you have installed Libcloud using pip you can also use it to upgrade it:

pip install --upgrade apache-libcloud==1.1.0

Upgrade notes

A page which describes backward incompatible or semi-incompatible changes and how to preserve the old behavior when this is possible can be found at https://libcloud.readthedocs.org/en/latest/upgrade_notes.html

Documentation

Regular and API documentation is available at https://libcloud.readthedocs.org/en/v1.1.0/

Bugs / Issues

If you find any bug or issue, please report it on our issue tracker https://issues.apache.org/jira/browse/LIBCLOUD. Don’t forget to attach an example and / or test which reproduces your problem.

Thanks

Thanks to everyone who contributed and made this release possible! Full list of people who contributed to this release can be found in the CHANGES file.

We are pleased to announce the release of Libcloud 1.0.0.

This first release in the 1.0 series which means it brings many new features, improvements, bug-fixes, and drivers. The 1.0 series includes 2 new driver types, container-as-a-service and backup-as-a-service.

Release highlights

This includes:

• New backup-as-a-service and container-as-a-service driver types
• Deprecated drivers that were no longer available such as Ninefold, IBM SCE more details
• Add support for authenticating against Keystone and OpenStack based clouds using OpenID Connect tokens.
• GCE nodes can be launched in a subnetwork and support for subnetworks
• Add Image Family support for GCE
• Add new Perth, Australia and Manila, Philippines region to the CloudSigma v2 driver.
• Add new S3 RGW storage driver.
• The Amazon EC2 driver has been changed to use region codes instead of separate drivers for each region.
• Introduce new list_regions class method on the base driver class
• Support for Dimension Data backup
• Added NSOne, LuaDNS, NearlyFreeSpeech.NET, BuddyNS, DNSPod DNS drivers
• Added Aliyun compute, load balancer and storage drivers
• Added Outscale storage driver

Full change log can be found at here.

Special thank you

I would like to wish a special thank you to all of our community contributors for their ongoing support to the project.

Tomaz Muraus, Paul Querna, Jerry Chen, Jeff Dunham, Eric Johnson, Jed Smith, Rick Wright, Roman Bogorodskiy, Mathew Baldwin, David Crossland, Alex Gaynor, Allard Hoeve, Sebastien Goasguen, Franck Cuny, Ken Dreyer, Javier Castillo II, Wido den Hollander, Michael Bennett, Anthony Monthe, Lior Goikhburg, Geunwoo Shin, Juan Font, “schaubl”, Misha Brukman, Oltjano Terpollari, Peter Schmidt, Ming Sheng, Gertjan Oude Lohuis, Markos Gogoulos, Bernard Paques, Evgeny Egorochkin, Oleg, Atsushi Sasaki, Miguel Caballer, Kumar, Gavin McDonald, Roeland Kuipers, Brian Curtin, Ivan Kusalic, Vanč Levstik, None, Anže Pečar, “MrBasset”, None, Javier M. Mellid, Greg Hill, None, Stefan Friesel, Mark Maglana, Alejandro Gabriel Pereira, Kyle Long, Teemu Vesala, Filipe Silva, Jay, Bernard Kerckenaere, John Obelenus, Jon Chen, Larry Lui,

Important breaking change- Amazon EC2 driver changes

The Amazon EC2 API was updated to consolidate the regional-based drivers into a single driver with a region argument in the constructor.

Amazon Instances should now be instantiated using the following syntax:

from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver

cls = get_driver(Provider.EC2)
driver = cls('access key', 'secret key', region='us-east-1')

This brings the Amazon API inline with the other drivers, makes it easier to maintain and switch between regions.

Bug fixes

General

• Fix a bug with consuming stdout and stderr in the paramiko SSH client which would manifest itself under very rare condition when a consumed chunk only contained a single byte or part of a multi byte UTF-8 character. [Lakshmi Kannan, Tomaz Muraus]

• Increase default chunk size from 1024 to 4096 bytes in the paramiko SSH client. This results in smaller number of receive calls on the average. [Tomaz Muraus]

• Fix to Dimension Data API address for Middle-East and Africa (GITHUB-700) [Anthony Shaw]

• Throw a more user-friendly exception on “No address associated with hostname”. (GITHUB-711, GITHUB-714, LIBCLOUD-803) [Tomaz Muraus, Scott Crunkleton]

• Remove deprecated provider constants with the region in the name and related driver classes (e.g. EC2_US_EAST, etc.).

  Those drivers have moved to single provider constant + region constructor argument model. [Tomaz Muraus]

New or deprecated drivers

Compute

• Deprecated IBM SCE, HP Helion, OpSource, Ninefold and CloudFrames drivers, removed driver code and tests. (GITHUB-701, LIBCLOUD-801) [Anthony Shaw]

• Introduced error messages (libcloud.compute.deprecated) for deprecated drivers (GITHUB-701, LIBCLOUD-801) [Anthony Shaw]

• New Compute drivers- BSNL, Indosat, Med-1, NTT-America, Internet Solutions (GITHUB-700) [Anthony Shaw]

• New driver for Aliyun Elastic Compute Service. (LIBCLOUD-802, GITHUB-712) [Sam Song]

Storage

• Added Outscale storage driver (GITHUB-730) [Javier M. Mellid]

• New driver for Aliyun OSS Storage Service. (LIBCLOUD-802, GITHUB-712) [Sam Song]

Loadbalancer

• New driver for Aliyun SLB Loadbalancer Service. (LIBCLOUD-802, GITHUB-712) [Sam Song]

DNS

• Added NearlyFreeSpeech.net (NSFN) driver [Ken Drayer]

• Added Lua DNS driver [Oltjano Terpollari]

• Added NSOne driver [Oltjano Terpollari]

Bug fixes

• Fix a bug in the GoDaddy driver - make sure host attribute on the connection class is correctly set to the hostname. [Tomaz Muraus]

• Fix handling of MX records in the Gandi driver. (GITHUB-718) [Ryan Lee]

Improvements

Compute

• Introduce new list_regions class method on the base driver class. This method is to be used with provider drivers which support multiple regions and region constructor argument. It allows users to enumerate available / supported regions. [Tomaz Muraus]

• [dimension data] added support for VMWare tools VM information inside list_nodes responses (GITHUB-734) [Jeff Dunham]

• [ec2] added ex_encrypted and ex_kms_key_id optional parameters to the create volume method (GITHUB-729) [Viktor Ognev]

• [dimension data] added support for managing host anti-affinity rules, added paging support to all supported calls and added support for requesting priority ordering when creating ACL rules (GITHUB-726) [Jeff Dunham]

• Addition of Dimension Data Australia federal government region to dimension data drivers. (GITHUB-700) [Anthony Shaw]

• [openstack] when creating floating IPs, added pool_id as an optional argument (GITHUB-725) [marko-p]

• [google compute] Added setMachineType method to allow for changing sizes of instances (GITHUB-721) [Eric Johnson]

• [google compute] allow bypassing image search in standard project list (GITHUB-713) [Max Illfelder]

• Add support for requesting a MKS token for accessing the remote console in VMware vCloud driver (GITHUB-706) [Juan Font Alonso]

• Add support in VMware vCloud driver for v5.5 API, with snapshot support (GITHUB-658) [Juan Font Alonso]

• Added support for adding a family to an image on Google Compute Driver (GITHUB-704) [Max Illfelder]

• Fix to set default signature version for AWS Seoul region to v4, removed non-supported size (hs1.xlarge) (GITHUB-684) [Geunwoo Shin]

• Support filtering by location in list_nodes for dimension data compute driver fix lack of paging support (GITHUB-691) [Jeff Dunham]

• Support for filtering by IPv4, IPv6, network, network domain, VLAN in Dimension data driver. (GITHUB-694) [Jeff Dunham]

• Added Node.created_at which, on supported drivers, contains the datetime the node was first started. (GITHUB-698) [Allard Hoeve] [Rick van de Loo]

Storage

• Improvements to Google Auth for Storage and Compute and MIME bug fix (LIBCLOUD-800, GITHUB-689) [Scott Crunkleton]

• Implement get_container, get_object and upload_object_via_stream methods in the Backblaze B2 storage driver.

  Note: Backblaze API doesn’t upload streaming uploads so when using upload_object_via_stream whole file is read and buffered in memory. (GITHUB-696) [Jay jshridha]

Backup

• Dimension Data - added additional testing, fixed bug on client response naming, added support for adding backup clients to a backup enabled node. (GITHUB-692, GITHUB-693, GITHUB-695) [Jeff Dunham]

Download

The release can can be downloaded from https://libcloud.apache.org/downloads.html or installed using pip:

pip install apache-libcloud==1.0.0

Upgrading

If you have installed Libcloud using pip you can also use it to upgrade it:

pip install --upgrade apache-libcloud==1.0.0

Upgrade notes

A page which describes backward incompatible or semi-incompatible changes and how to preserve the old behavior when this is possible can be found at https://libcloud.readthedocs.org/en/latest/upgrade_notes.html

Documentation

Regular and API documentation is available at https://libcloud.readthedocs.org/en/latest/

Bugs / Issues

If you find any bug or issue, please report it on our issue tracker https://issues.apache.org/jira/browse/LIBCLOUD. Don’t forget to attach an example and / or test which reproduces your problem.

Thanks

Thanks to everyone who contributed and made this release possible! Full list of people who contributed to this release can be found in the CHANGES file.

We are pleased to announce the release of Libcloud 1.0.0-rc2.

This the second pre-release in the 1.0.0 series which means it brings many new features, improvements, bug-fixes, and DNS drivers.

Release highlights

This includes:

• Deprecated drivers that were no longer available such as Ninefold, IBM SCE more details
• The Amazon EC2 driver has been changed to use region codes instead of separate drivers for each region.
• Introduce new list_regions class method on the base driver class
• Support for Dimension Data backup
• Added NSOne, LuaDNS, NearlyFreeSpeech.NET DNS drivers
• Added Aliyun compute, load balancer and storage drivers
• Added Outscale storage driver

Full change log can be found at here.

Important breaking change- Amazon EC2 driver changes

The Amazon EC2 API was updated to consolidate the regional-based drivers into a single driver with a region argument in the constructor.

Amazon Instances should now be instantiated using the following syntax:

from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver

cls = get_driver(Provider.EC2, region='us-east-i1')
driver = cls('access key', 'secret key')

This brings the Amazon API inline with the other drivers, makes it easier to maintain and switch between regions.

Bug fixes

General

• Fix a bug with consuming stdout and stderr in the paramiko SSH client which would manifest itself under very rare condition when a consumed chunk only contained a single byte or part of a multi byte UTF-8 character. [Lakshmi Kannan, Tomaz Muraus]

• Increase default chunk size from 1024 to 4096 bytes in the paramiko SSH client. This results in smaller number of receive calls on the average. [Tomaz Muraus]

• Fix to Dimension Data API address for Middle-East and Africa (GITHUB-700) [Anthony Shaw]

• Throw a more user-friendly exception on “No address associated with hostname”. (GITHUB-711, GITHUB-714, LIBCLOUD-803) [Tomaz Muraus, Scott Crunkleton]

• Remove deprecated provider constants with the region in the name and related driver classes (e.g. EC2_US_EAST, etc.).

  Those drivers have moved to single provider constant + region constructor argument model. [Tomaz Muraus]

New or deprecated drivers

Compute

• Deprecated IBM SCE, HP Helion, OpSource, Ninefold and CloudFrames drivers, removed driver code and tests. (GITHUB-701, LIBCLOUD-801) [Anthony Shaw]

• Introduced error messages (libcloud.compute.deprecated) for deprecated drivers (GITHUB-701, LIBCLOUD-801) [Anthony Shaw]

• New Compute drivers- BSNL, Indosat, Med-1, NTT-America, Internet Solutions (GITHUB-700) [Anthony Shaw]

• New driver for Aliyun Elastic Compute Service. (LIBCLOUD-802, GITHUB-712) [Sam Song]

Storage

• Added Outscale storage driver (GITHUB-730) [Javier M. Mellid]

• New driver for Aliyun OSS Storage Service. (LIBCLOUD-802, GITHUB-712) [Sam Song]

Loadbalancer

• New driver for Aliyun SLB Loadbalancer Service. (LIBCLOUD-802, GITHUB-712) [Sam Song]

DNS

• Added NearlyFreeSpeech.net (NSFN) driver [Ken Drayer]

• Added Lua DNS driver [Oltjano Terpollari]

• Added NSOne driver [Oltjano Terpollari]

Bug fixes

• Fix a bug in the GoDaddy driver - make sure host attribute on the connection class is correctly set to the hostname. [Tomaz Muraus]

• Fix handling of MX records in the Gandi driver. (GITHUB-718) [Ryan Lee]

Improvements

Compute

• Introduce new list_regions class method on the base driver class. This method is to be used with provider drivers which support multiple regions and region constructor argument. It allows users to enumerate available / supported regions. [Tomaz Muraus]

• [dimension data] added support for VMWare tools VM information inside list_nodes responses (GITHUB-734) [Jeff Dunham]

• [ec2] added ex_encrypted and ex_kms_key_id optional parameters to the create volume method (GITHUB-729) [Viktor Ognev]

• [dimension data] added support for managing host anti-affinity rules, added paging support to all supported calls and added support for requesting priority ordering when creating ACL rules (GITHUB-726) [Jeff Dunham]

• Addition of Dimension Data Australia federal government region to dimension data drivers. (GITHUB-700) [Anthony Shaw]

• [openstack] when creating floating IPs, added pool_id as an optional argument (GITHUB-725) [marko-p]

• [google compute] Added setMachineType method to allow for changing sizes of instances (GITHUB-721) [Eric Johnson]

• [google compute] allow bypassing image search in standard project list (GITHUB-713) [Max Illfelder]

• Add support for requesting a MKS token for accessing the remote console in VMware vCloud driver (GITHUB-706) [Juan Font Alonso]

• Add support in VMware vCloud driver for v5.5 API, with snapshot support (GITHUB-658) [Juan Font Alonso]

• Added support for adding a family to an image on Google Compute Driver (GITHUB-704) [Max Illfelder]

• Fix to set default signature version for AWS Seoul region to v4, removed non-supported size (hs1.xlarge) (GITHUB-684) [Geunwoo Shin]

• Support filtering by location in list_nodes for dimension data compute driver fix lack of paging support (GITHUB-691) [Jeff Dunham]

• Support for filtering by IPv4, IPv6, network, network domain, VLAN in Dimension data driver. (GITHUB-694) [Jeff Dunham]

• Added Node.created_at which, on supported drivers, contains the datetime the node was first started. (GITHUB-698) [Allard Hoeve] [Rick van de Loo]

Storage

• Improvements to Google Auth for Storage and Compute and MIME bug fix (LIBCLOUD-800, GITHUB-689) [Scott Crunkleton]

• Implement get_container, get_object and upload_object_via_stream methods in the Backblaze B2 storage driver.

  Note: Backblaze API doesn’t upload streaming uploads so when using upload_object_via_stream whole file is read and buffered in memory. (GITHUB-696) [Jay jshridha]

Backup

• Dimension Data - added additional testing, fixed bug on client response naming, added support for adding backup clients to a backup enabled node. (GITHUB-692, GITHUB-693, GITHUB-695) [Jeff Dunham]

Download

The release can can be downloaded from https://libcloud.apache.org/downloads.html or installed using pip:

pip install apache-libcloud==1.0.0-rc2

Upgrading

If you have installed Libcloud using pip you can also use it to upgrade it:

pip install --upgrade apache-libcloud==1.0.0-rc2

Upgrade notes

A page which describes backward incompatible or semi-incompatible changes and how to preserve the old behavior when this is possible can be found at https://libcloud.readthedocs.org/en/latest/upgrade_notes.html

Documentation

Regular and API documentation is available at https://libcloud.readthedocs.org/en/latest/

Bugs / Issues

If you find any bug or issue, please report it on our issue tracker https://issues.apache.org/jira/browse/LIBCLOUD. Don’t forget to attach an example and / or test which reproduces your problem.

Thanks

Thanks to everyone who contributed and made this release possible! Full list of people who contributed to this release can be found in the CHANGES file.

We would like to thank the following community members for their contribution to this release:

• Jeff Dunham
• Max Illfelder
• Ken Dreyer
• Sam Song
• Oltjano Terpollari
• Javier M. Mellid

Background

I’ve just pushed a branch of the latest version of libcloud using the popular requests package by Kenneth Reitz instead of our home-rolled HTTP client library.

This article is for both users and developers of libcloud. If you want to give feedback, please join the developer mailing list.

Why?

• requests is the defacto standard - it would be in the standard library but agreed against to allow it to develop faster https://github.com/kennethreitz/requests/issues/2424
• it works with python 2.6->3.5
• Our SSL experience has a lot to be desired for Windows users, having to download the CA cert package and setting environment variables just to get SSL working
• Developers can use requests_mock for deeper integration testing
• less code to maintain
• the role of libcloud is for cloud abstraction, we provide no value in writing and maintaining our own HTTP client library

Benefits of requests

There are a number of benefits to having a requests package

• The client library code is smaller, leaner and simpler.
• Requests has built in decompression support, we no longer need to support this
• Requests has built in RAW download, upload support, helping with our storage drivers

Implications of the change

• There are no longer 2 classes (LibcloudHTTPSConnection and LibcloudHTTPConnection) to be provided to each driver, they are now 1 class - LibcloudConnection. You probably won’t notice this because it is a property of the Connection class, but if you are developing or extending functionality then it is implicated.
• Unit tests will look slightly different (see below)
• This change broke 4200 unit tests (out of 6340)! I’ve since fixed them all since they were coupled to the original implementation, but now I don’t know if all of tests are valid.

Testing with requests

Unit tests that were written like this:

class DigitalOceanTests(LibcloudTestCase):
  		  
      def setUp(self):
          DigitalOceanBaseDriver.connectionCls.conn_classes = \	
           (None, DigitalOceanMockHttp)
          DigitalOceanMockHttp.type = None
          self.driver = DigitalOceanBaseDriver(*DIGITALOCEAN_v1_PARAMS)

Because of the change have been modified to (I updated all of them - so this is just for future reference)

class DigitalOceanTests(LibcloudTestCase):
  		  
      def setUp(self):
          DigitalOceanBaseDriver.connectionCls.conn_class = DigitalOceanMockHttp
          DigitalOceanMockHttp.type = None
          self.driver = DigitalOceanBaseDriver(*DIGITALOCEAN_v1_PARAMS)

Check it out!

The package is on my personal apache site, you can download it and install it in a virtualenv for testing.

pip install -e http://people.apache.org/~anthonyshaw/libcloud/1.0.0-rc2-requests/apache-libcloud-1.0.0-rc2-requests.zip@feature#egg=apache-libcloud

The hashes are my apache space

Have a look at the PR and the change set for a list of changes

What might break?

What I’m really looking for is for users of Libcloud to take 15 minutes, an existing (working) libcloud script, install this package in a virtualenv and just validate that there are no regression bugs with this change.

I’m particularly sceptical about the storage drivers.

Once we have enough community feedback, we will propose a vote to merge this into trunk for future release.

Credit

Credit to dz0ny on IRC for contributing some of the requests patch.