Posts

On Thursday, April 19 the CloudStack community joined up with the Ceph community for a combined event in London, and what an event it was! The meetup took place at the Early Excellence Centre at Canada Water, on a beautiful, sunny day (in fact the hottest day of the year so far), and registration started early with people enjoying coffee and pastries by the canal.

CloudStack & Ceph combined morning sessions

Once everyone had enjoyed breakfast, we all settled down as Wido den Hollander (of 42on) took to the podium to welcome everyone, explain a bit about how the event had come about, and talk through both technologies (CloudStack and Ceph), including how well they work together. Having set the scene Wido was ready to deliver the first presentation of the day, which was prepared to appeal to both communities – ‘Building a highly available cloud with Ceph and CloudStack’. In this talk, Wido talked about how he came across and started using Ceph, and how his company utilises both Ceph and CloudStack to offer services. To quote Wido… “We manage tens of thousands of Virtual Machines running with Ceph and CloudStack and we think it is a winning and golden combination”! This first talk really got the event off to a flying start, with plenty of questions and interaction from the floor, and Wido’s slides can be found here:

Next up was John Spray (RedHat) with a talk entitled ‘Ceph in Kubernetes’ John talked through Kubernetes, Rook, and why and how it all works together. Lot’s more detail in John’s slides:

After a short break, it was the turn of Phil Straw (SoftIron) to present. Phil’s talk was ‘Ceph on ARM64’, and covered all the fundamentals including some real world results from SoftIton’s appliance. Take a look through Phil’s slides:

Sebastien Bretschneider (intelligence) was last up before lunch, with ‘Our way to Ceph’ – another great talk for both communities as he talked about his experience using Ceph with CloudStack. Starting with some background, Sebastien talked through lessons learnt and how the platform looks now. More information in Sebastien’s slides:

CloudStack afternoon sessions

After lunch, the day split into two separate tracks, the first dedicated to CloudStack (the CloudStack European User Group – CSEUG), the second to Ceph. As with the morning, both rooms were very busy, with full agendas. As I focussed on CloudStack, I cover the CloudStack talks here (at the end of the article I include links to the Ceph presentations).

Giles Sirett (CSEUG chairman) brought the room to order and started with introductions and CloudStack news, which (as always) there is lots of, including new releases – 4.9.3, 4.10, and the latest LTS release – 4.11. Sticking with 4.11, Giles touched on some of the new features that have been introduced, such as new host HA framework, new CA framework and Prometheus integration (much more detail in Paul’s later presentation). Giles also mentioned that we are moving towards zero-downtime upgrades – a highly anticipated and much needed enhancement! What is exciting about this community is that nearly all of the 4.11 features were contributed or instigated by USERS… and speaking of new features – this was someone’s response to Citrix changing the XenServer licencing model: https://xcp-ng.org/

Giles then advised us about some dates or our diaries – lots of upcoming CloudStack events, and shared how CloudStack is growing in awareness and use – we are seeing 800 downloads per month, just from our (ShapeBlue) repo, and have several new, large scale private CloudStack clouds being deployed this year. Please see Giles slides for much more information:

Once Giles had finished taking questions, he introduced the first presentation, and this was both Wido den Hollander and Mike Tutkowski (SolidFire) for a brief retrospective of Wido’s year as VP
of the CloudStack project, and welcoming Mike as the new VP! This was a quick introduction, and next up was Antoine Coetsier (Exoscale) with his talk entitled ‘Billing the cloud’.

Starting with Exoscale’s background, Antoine talked through pros and cons of different approaches, and explained Exoscale’s solution. Antoine’s slides are here:

As mentioned earlier, next up was Paul Angus (ShapeBlue), with ‘What’s new in ACS 4.11’, which includes 100s of updates, over 30 new features and the best automated test coverage yet. Paul went through user features, operator features and integrations. This demonstrated just how much work and development is going into CloudStack, and if you are interested to see just how much work and development, I urge you to read through Paul’s comprehensive slides:

After a short break, Giles introduced the next talk of the day – Boyan Krosnov (Storpool), ‘Building a software-defined cloud’. After a little background, Boyan started by explaining how a hyper-converged cloud infrastructure looks, and then talked about how Storpool works with CloudStack.

Ivan’s talk was filmed: https://youtu.be/_d_xnqpmY0M, and his slides can be found here:

Next up was Mike Tutkowski (SolidFire), who talked us through managed storage in CloudStack, and then presented a live demo so we could see first-hand how it works. As Mike works for SolidFire this was the storage he used for his demo!

The end of this last talk of the CloudStack track coincided with the end of the last Ceph talk next door, so we all congregated together to hear some last thoughts and acknowledgements from Wido. This was the first collaborative day between Ceph and CloudStack, and it was a resounding success, thanks in no small part to the vibrant communities that support these technologies. Bringing the event to a close, we all moved the conversations to a nearby pub, where the event continued late into a beautiful London evening.

Thanks to our sponsors, without which these great events wouldn’t be possible – 42on, RedHat and ShapeBlue (us!) for providing the venue and refreshments, and to SoftIron for providing the evening drinks.

Ceph afternoon sessions

Wido den Hollander – 10 ways to break your Ceph cluster

Nick Fisk – low latency Ceph

Version 4.11 of Apache CloudStack has been released with some exciting new features and a long list of improvements and fixes. It includes more than 400 commits, 220 pull requests, and fixes more than 250 issues.  This version has been worked on for 8 months and is the first release of the 4.11 LTS releases, which will be supported until  1 July 2019.

We’ve been heavily involved in this release at ShapeBlue; our engineering team has contributed a number of the major new features and our own Rohit Yadav has been the 4.11 Release Manager.

As well as some really interesting new features, CloudStack 4.11 has significant performance and reliability improvements to the Virtual Router.

This is far from an exhaustive list, but here are the headline items that we think are most significant.

New Features and Improvements

  • Support for XenServer 7.1 and 7.2, and improved support for VMware 6.5.
  • Host-HA framework and HA-provider for KVM hosts with and NFS as primary storage, and a new background polling task manager.
  • Secure agents communication: new certificate authority framework and a default built-in root CA provider.
  • New network type – L2.
  • CloudStack metrics exporter for Prometheus.
  • Cloudian Hyperstore connector for CloudStack.
  • Annotation feature for CloudStack entities such as hosts.
  • Separation of volume snapshot creation on primary storage and backing operation on secondary storage.
  • Limit admin access from specified CIDRs.
  • Expansion of Management IP Range.
  • Dedication of public IPs to SSVM and CPVM.
  • Support for separate subnet for SSVM and CPVM.
  • Bypass secondary storage template copy/transfer for KVM.
  • Support for multi-disk OVA template for VMware.
  • Storage overprovisioning for local storage.
  • LDAP mapping with domain scope, and mapping of LDAP group to an account.
  • Move user across accounts.
  • Support for “VSD managed” networks with Nuage Networks.
  • Extend config drive support for user data, metadata, and password (Nuage networks).
  • Nuage domain template selection per VPC and support for network migration.
  • Managed storage enhancements.
  • Support for watchdog timer to KVM Instances.
  • Support for Secondary IPv6 Addresses and Subnets.
  • IPv6 Prefix Delegation support in basic networking.
  • Ability to specific MAC address while deploying VM or adding a NIC to a VM.
  • VMware dvSwitch security policies configuration in network offering
  • Allow more than 7 NICs to be added to a VMware VM.
  • Network rate usage for guest offering for VRs.
  • Usage metrics for VM snapshot on primary storage.
  • Enable Netscaler inline mode.
  • NCC integration in CloudStack.
  • The retirement of the Midonet network plugin.

UI Improvements

  • High precision of metrics percentage in the dashboard:
  • Event timeline – filter related events:

  • Navigation improvements between related entities:
  • Bulk operation support for stopping and destroying VMs (note: minor known issue where manual refresh required afterwards):
  • List view improvements and additional columns with state icon:






Structural Improvements

  • Embedded Jetty and improved CloudStack management server configuration.
  • Improved support for Java 8 for building artifacts/modules, packaging, and in the systemvm template.
  • New Debian 9 based systemvm template:
    • Patches system VM without reboot, reduces VR/system VM startup time to few tens of seconds.
    • Faster console proxy startup and service availability.
    • Improved support for redundant virtual routers, conntrackd and keepalived.
    • Improved strongswan provided VPN (s2s and remote access).
    • Packer based systemvm template generation and reduced disk size.
    • Several optimization and improvements.

Documentation and Downloads

The official installation, administration and API documentation can be found below:
http://docs.cloudstack.apache.org 

The 4.11.0.0 release notes can be found at:
http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.11.0.0 

The instruction and links to use ShapeBlue provided (noredist) packages repository can be found at:
https://www.shapeblue.com/packages 

Many people find it challenging to get started with CloudStack’s networking. There are some basic concepts, which although not overly complicated, are not especially obvious either. This blog will try to explain these underlying concepts, in order to make getting started with CloudStack networking models much easier.