Posts

On Thursday, October 24 2019 the CloudStack community joined up with the Ceph community for a combined event in London, and it was another super day! The meetup took place at Student Central (formerly University of London Union) on a crisp London day, and registration started early with people enjoying coffee and pastries whilst greeting friends and colleagues. The morning was a mixture of Ceph and CloudStack talks, and the afternoon was split into two tracks – one devoted to Ceph and the other to CloudStack.

All the day’s talks were recorded, and all video can be found here. Links to all presentation slides can be found at the bottom of this article.

CloudStack & Ceph combined morning sessions

Once everyone had enjoyed breakfast, Wido den Hollander (of 42on) and I (Steve Roles, ShapeBlue) took to the podium to welcome everyone, explain a bit about how the event had come about (only the second such combined event, the first was April 2018), and talk a little about both technologies (CloudStack and Ceph) and how well they work together.

Having set the scene Wido was ready to deliver the first presentation of the day, which was prepared to appeal to both communities – ‘Deploying CloudStack and Ceph with flexible VXLAN and BGP networking’. This first talk really got the event off to a flying start, with plenty of questions and interaction from the floor.

Next up was Venko Moyankov (Storpool), with a talk entitled ‘Achieving the ultimate performance with KVM’. Venko talked through private cloud setup with KVM achieving 1M IOPS per hyper-converged (storage+compute) node, posing (and answering) the question: What is the optimum architecture and configuration for performance and efficiency?

After a short break, it was the turn of Andrija Panic (ShapeBlue) to present ‘Ceph with CloudStack’. In this presentation, Andrija gave a brief introduction to Ceph and covered some considerations around its architecture, both in general and related to CloudStack.

CloudStack afternoon sessions

After lunch, the day split into two separate tracks, the first dedicated to CloudStack (the CloudStack European User Group – CSEUG), the second to Ceph. As with the morning, both rooms were very busy with full agendas. As I focused on CloudStack, I cover the CloudStack talks here (at the end of the article I include links to all Ceph presentations).

In the absence of Giles Sirett (CSEUG chairman), I stepped up to talk about CloudStack news, which (as always) there is lots of, including new releases – 4.12, 4.11.3 (LTS), and 4.13 (LTS). I touched on some of the new features introduced in these releases, and also talked a little about what’s coming in the next LTS (Jan / Feb 2020 – 4.14). I won’t go into detail as that was the job of my colleague Andrija in a later talk. I mentioned the recent fantastic CloudStack Collaboration Conference in Las Vegas (all recorded talks can be found here) and that the next CSEUG meetup will likely be in February 2020, and will be in Berlin, hosted by out friends from iTelligence. Registration will be open soon, and please contact me if you would like to give a talk.

I then introduced the first talk of the CloudStack track, welcoming Andre Walter and Sebastien Bretschneider (iTelligence) with ‘Cloud Management Portal – Admin View’. Another great talk for both communities with a technical overview showing how iTelligence as a cloud service provider use CloudStack & Ceph and how they extended their infrastructure with additional service components to enhance their cloud service portfolio.

Next up was Slavka Peleva (also of Storpool) with a talk entitled ‘Storage-based Snapshots for KVM VMs in CloudStack’. Slavka talked us through this feature, starting with VM & disk / volume snapshots, before explaining in detail the solution of employing the underlying storage provider’s capability to create, revert and delete disk snapshots.

As mentioned earlier, next up was Andrija Panic (ShapeBlue), with ‘New stuff in CloudStack!’ Andrija went through new user features, operator features and integrations such as… VM Ingestion, Backup & Recovery framework, Vmware 6.7 support, Kubernetes integration, and of course – a new, modern UI for CloudStack. This demonstrated just how much work and development is going into CloudStack, and if you are interested to see just how much work and development, I urge you to read through Andrija’s comprehensive slides, and watch his video:

After a short break, we welcomed Graeme Wright (Smart Communications) with his talk ‘Our journey with CloudStack’. This sort is talk is always fascinating, as everyone has a slightly different journey and reasoning behind technology decisions. The questions Graeme answered: Why we chose CloudStack for Smart Communications… How we use CloudStack and the features we employ… What CloudStack enables us to achieve for our cloud solution.

The last talk of the day was Sven Vogel (eWerk) with ‘Running OpenShift Clusters in a Cloudstack Environment’. This talk provided an insight into the way in which eWerk run OpenShift (kubernetes) Clusters on Cloudstack.

The end of this last talk of the CloudStack track coincided with the end of the last Ceph talk next door, so we all congregated together to hear some last thoughts and acknowledgements from Wido. This was the second collaborative day between Ceph and CloudStack, and much like the first it was a resounding success, thanks in no small part to the vibrant communities that support these technologies. This will now be an annual event, with the next one provisionally scheduled for October 2020.

Bringing the event to a close, we all moved the conversations downstairs to the bar (a definite advantage of holding the event at a student union), where the event continued late into a busy London evening.

Thanks to our sponsors, without which these great events wouldn’t be possible – Softiron and the Ceph Foundation (for providing the venue and catering), the State51 Music Group (for buying drinks in the bar afterwards) and 42on for helping with organisation and logistics. And of course – thanks to the day’s speakers, many of whom traveled from afar (Amsterdam, Bulgaria, Serbia, Germany) to give their time and expertise freely. Hope to see you next time!

All presentation slides:

Deploying CloudStack and Ceph with flexible VXLAN and BGP networking

https://www.slideshare.net/ShapeBlue/deploying-cloudstack-and-ceph-with-flexible-vxlan-and-bgp-networking

Venko Moyankov: Achieving the ultimate performance with KVM

https://www.slideshare.net/ShapeBlue/achieving-the-ultimate-performance-with-kvm-190725090

Andrija Panic: Ceph with CloudStack

https://www.slideshare.net/ShapeBlue/ceph-with-cloudstack

Steve Roles: CloudStack news

https://www.slideshare.net/ShapeBlue/cloudstack-news-190743047

Andre Walter & Sebastian Bretschneider: Cloud Management Portal – Admin View

https://www.slideshare.net/ShapeBlue/cloud-management-portal-admin-view

Slavka Peleva: Storage based snapshots for KVM VMs in CloudStack

https://www.slideshare.net/ShapeBlue/storage-based-snapshots-for-kvm-vms-in-cloudstack

Chris Ellis: Routed Fabrics For Ceph

https://www.slideshare.net/ShapeBlue/routed-fabrics-for-ceph

Andrija Panic: New stuff in CloudStack!

https://www.slideshare.net/ShapeBlue/new-stuff-in-cloudstack

James Page: Managing Ceph operational complexity with Juju

https://www.slideshare.net/ShapeBlue/managing-ceph-operational-complexity-with-juju

Graeme Wright: Our journey with CloudStack

https://www.slideshare.net/ShapeBlue/our-journey-with-cloudstack

Zoltan Arnold: Disaggregating Ceph using NVMeoF

https://www.slideshare.net/ShapeBlue/disaggregating-ceph-using-nvmeof-190727874

Sven Vogel: Running OpenShift Clusters in a Cloudstack Environment

https://www.slideshare.net/ShapeBlue/running-openshift-clusters-in-a-cloudstack-environment

Thomas Bennet: Ceph for Storing MeerKAT Radio Telescope Data

https://www.slideshare.net/ShapeBlue/ceph-for-storing-meerkat-radio-telescope-data

It’s taken me a couple of weeks to write this blog, and not because it took me that long to recover from a week in Las Vegas! The CloudStack Collaboration Conference is part of ApacheCon, and this was a year for anniversaries – not only ApacheCon 20th anniversary, but an anniversary of sorts for CloudStack, as our first ever Collaboration Conference was in Las Vegas 7 years ago.

Just like 7 years ago (and every year since) this was a fantastic few days, with great talks and more evidence (if any was required) proving that Apache CloudStack continues getting better and stronger. What evidence? The representation at the conference from enterprises wanting to say ‘we use CloudStack’ (more on that later); the effort put into the hackathon where a room full of CloudStack users and developers talked passionately about new features; the sheer quantity of attendees (2nd largest registration ever) and the wide variety of talks on diverse topics (22 presentations over 2 tracks, plus a day long hackathon).

The conference started Monday, September 9, at 0900 when our very own Paul Angus (ShapeBlue’s CTO and current VP of the CloudStack project) took to the stage to welcome us to this year’s Cloudstack Collaboration Conference. Paul set the tone of the event by talking us through how the next few days look, and giving us a brief ‘State of the (CloudStack) Union’, talking about latest releases and features, the community, and how CloudStack has recently been recognised by Newsweek in their inaugural list of Best Business Tools in the ‘Cloud Services’ category. Paul also talked about the Apache Foundation (their mission statement always bears repeating: “The mission of the Apache Software Foundation (ASF) is to provide software for the public good”).

I won’t be giving a summary of all talks but Paul’s talk was notable as we had a few quotes and special guests who wanted to tell us why they use CloudStack. These included:

BT Compute: Maurice Nettesheim, Head of Shared & Cloud Services:

“CloudStack gives us the full range of functionality we require while still being easy to deploy, scalable, flexible and robust”

Leaseweb: Jeffrey Kriegsman, Senior System Engineer

“We like CloudStack because it is highly customisable”

Ticketmaster: Jean-Francois Nadeau, Principle Systems Engineer

“CloudStack is simple to understand, extensible and upgrading is easy”

Apple: Anthony Shortland, Senior Manager, Cloud Engineering, Internet Software and Services

“The vast majority of Apple Services Group revenue is from internet services, and right in the middle of that is a very large CloudStack cloud”

This opening presentation was indicative of the days to follow. Fascinating talks and presentations, a day long hackathon and evenings spent in the company of this great, growing, vibrant community.

If you were there – I hope you enjoyed the event as much as I did. If you weren’t there – all the talks were recorded and can be found on the Apache Foundation YouTube channel, and all ShapeBlue slide-decks can be found below. Either way – I hope to see you at next year’s conference (location TBC), or maybe at the next CloudStack event, here in London Thursday, October 24. We have a CloudStack / Ceph day. Registration, information on presentations and more information here.

Thanks to the Apache Foundation and to Las Vegas for a great week!

Al ShapeBlue slide-decks:

Paul Angus – Welcome to CloudStack Collaboration Conference

Abhishek Kumar – CloudStack Locking Service

Rohit Yadav – Modern UI for CloudStack

Anurag Awasthi – Machine Learning applications for CloudStack

Rohit Yadav – The future of the CloudStack Virtual Router

Boris Stoyanov – Troubleshooting the Virtual Router – Run and Get Diagnostics

Andrija Panic – Ceph with CloudStack

Nicolas Vazquez – Open vSwitch with DPDK on CloudStack

Paul Angus – CloudStack Backup and Recovery Framework

Blog by Ivet Petrova, StorPool.

On June 13th, StorPool had the honour and privilege to host and organize the European Cloud Infrastructure and CloudStack User Group together with its partner ShapeBlue. The event was a get together of the local IT infrastructure experts and CloudStack users. Main focus were talks presenting best practices and useful information on how to build an efficient public or private infrastructure. In addition, worlds leading experts and contributors to the open-source Apache CloudStack Project presented its latest functionalities and updates in the project.

What is CloudStack? Key features and use cases

CloudStack is a scalable cloud orchestration platform for delivering turnkey infrastructure as a service clouds. As it is relatively easy to deploy and manage, it attracts the attention of people considering which cloud management system to use. Firstly, its architecture is highly scalable and reliable. The most massive known production cloud with CloudStack installation was reaching approx. 35 000 physical hosts and was running smoothly. Secondly, CloudStack is hypervisor agnostic. It supports KVM, Xen, VMware, HyperV, OVM, etc. Moreover, it also presents a REST API and is used for cloud infrastructure as a service, containers as a service, and many more use cases in which enterprises need a reliable solution to manage complex infrastructure and virtualizations.

CloudStack supports different storage options, and StorPool has its CloudStack integration. More about the story of building StorPool’s CloudStack integration, you can read here. 

CloudStack Market Growth

The European Cloud Infrastructure and CloudStack User Day started with a keynote session of Giles Sirett, CEO of ShapeBlue and widely recognized contributor to the Apache project. Giles talked us through the history of CloudStack, its main advantages, and the value it can bring to companies. After that, he made an overview of interesting use case and shared information for its releases and user communities. According to him, the most significant value of CloudStack is that it is a user-driven project and community, which makes it vibrant and rapidly developed. In conclusion, Giles also shared that CloudStack adoption is quickly growing and now it is used by some of the biggest companies globally.

Achieving the ultimate performance with KVM

Next to the stage was Boyan Krosnov, CPO of StorPool. In his session, he discussed a private cloud setup with KVM achieving 1M IOPS per hyper-converged (storage+compute) node. Besides, Boyan answered the question: What is the optimum architecture and configuration for performance and efficiency? His session was a deep technical dive into the ways for building an efficient and high-performance cloud infrastructure. Furthermore, Boyan explained why performance matters and how many companies even do not understand they are struggling with performance issues … until the moment their customers notify them for this.

During the presentation, the CPO of StorPool covered essential aspects of building cloud infrastructure, part of which were:

  • why the same hardware can bring you 10 times better performance  than before
  • how hardware, compute and networking affect the performance
  • tips and trick for getting ultimate KVM performance
  • …and many more

Boyan advised all participants in the event to pay attention on their cloud performance, apply possible optimizations for accelerating it and closely monitor the cloud performance.

CloudStack: A Service Managers Perspective

After a short break, we welcomed Maria Barta from Itelligence Global Managed Services GmbH. Maria presented a different perspective on CloudStack – “A Service Managers Perspective”. Agile business processes are becoming increasingly important in successful IT services. Itelligence GmbH, provides many different ultra-flexible and highly adaptable cloud solutions. To ensure customer / user satisfaction (i.e. availability, data security and product transparency) and simultaneously facilitate effective agile product development within their team, the role of the service manager is steadily evolving. As a conclusion, the talk provided an insight to the benefits and limitations of CloudStack in relation to the service manager objectives and Maria’s attempt to overcome these in her specific internal IaaS solution.

What’s new in CloudStack 4.13

Paul Angus, VP Technology of ShapeBlue and current VP of CloudStack. He was one of the most awaited speakers at the event. Mainly because he is the most experienced person in the community, who has exceptional knowledge in CloudStack. His session was focused on the new release of CloudStack. The 4.13 version is due for release this summer. With 100s of updates and new features, Paul went through user features. He also talked about operator features and integrations, demonstrating just how much work and development is going into CloudStack.

Paul also shared that version 4.14 most probably will arrive at the end of 2019 / beginning of 2020. He enjoyed great attention from the European CloudStack community and managed to give valuable pieces of advice to the admins dealing with complex cloud issues.

Challenges with high-density networks 

Last, but not least, Marian Marinov from SiteGround web hosting company shared his experience in the problems when managing high-density networks. In cloud environments, people consider the network as a given and almost limitless resource. You get an interface and you are told its bandwidth capacity. From the perspective of the client, this is true. But from the perspective of the provider, this is far from the truth. In his talk, Marian took a look at some DataCenter network designs and what technologies / protocols were used to battle the problem with high-density clouds. All participants in the event had the chance to learn about VXLAN and “L3” switching.

After the final official talk, we managed to organize a great networking event between the speakers and the event attendees. One more opportunity to learn new things for cloud infrastructure and about building a cloud with StorPool and CloudStack.

For StorPool’s team, it was a pleasure to be host and co-organizer of the event and to put the beginning of a new CloudStack community in Bulgaria.

Our presenters’ slides can be found here:

Giles Sirett – CloudStack EU User Group 13 june 2019 – Sofia

Boyan Krosnov – Achieving the ultimate performance with KVM

Maria Barta – CS Day Sofia_ CS – A service manager perspective_20190613

Paul Angus – CSEUG19-What’s coming in CloudStack

Marian Marinov – Challenges with high-density networks

 

Our first meetup of 2019 saw us at a new venue – Ticketmaster’s London HQ, and if you’re a music lover it certainly takes the prize for coolest meeting venue yet! Walls covered with pictures of rock stars and a stage complete with guitars and Marshall amps (not to mention pinball machines and a bar) created a real buzz of excitement before the meeting had even started. Once everyone had met up with friends, taken photos and finished lunch, Giles Sirett (CSEUG chairman) called the meeting to order, and kicked the day off with CloudStack news.

Giles talked us through the current and upcoming releases of CloudStack, and the new release of Cloudmonkey (6.0), before ‘unofficially’ announcing the new VP of Apache CloudStack – our very own Paul Angus! Moving onto market news, Giles introduced a thought-provoking topic, starting by referencing an article titled ‘What happened to OpenStack?’, before moving onto the different marketing approaches taken by the technologies.

We then heard about upcoming events – the next CSEUG will be in Sofia in June (register here), and we are currently looking for speakers. The CSEUG returns to London in October (and we are working with Ceph on making this another collaboration event), and we have CloudStack Collaboration Conferences in April (Brazil) and September (Las Vegas). Again – the Call For Participation is open for Las Vegas. All the information provided by Giles can be found by watching his talk:

Giles then introduced our first guest speaker onto the stage – Mike Rowell (Director, Platform Infrastructure) of our hosts Ticketmaster, with a talk titled ‘Our journey to a next generation cloud’. Mike did indeed take us on a journey, first explaining what challenges they needed to overcome, and what solutions they initially implemented, before discussing their investigations into a scalable cloud solution. These investigations led them to Apache CloudStack, and Mike went on to share what issues he experienced, as well as what other tools they use, such as Ansible, Terraform and Prometheus in the stack. Mike finished his talk by expanding on some features he would like to see in CloudStack.

Next to the stage was Bobby Stoyanov (ShapeBlue), talking us through some of the new features in CloudStack. These new features include: more sophisticated options for specifying pod and cluster while deploying a VM; running and retrieving diagnostics on the VR; sending additional configuration to VMs; and adding options to cleanup additional data disks when destroying a VM. It’s always great to hear about new features, and to see evidence of the continuing innovation and commitment to the project from the community. Bobby ‘dived deep’ on each feature, so I recommend you watch his talk:

After a short break, we welcomed Wido den Hollander from PCextreme, who talked about flexible networking for scaling a cloud environment,. As Wido explained – regular layer 2 VLANs have their limitations when it comes to scalability, and VXLAN overcomes these limitations, making it easier to scale out your CloudStack deployment. As of CloudStack 4.12, VXLAN can use IPv6, and Wido talked about Advanced networking + IPv6 + VXLAN which he is putting into production right now with the 4.12 release. As usual, Wido covered his topic comprehensively, and if you want to hear more, watch his talk:

We then welcomed Boyan Ivanov (Storpool) with his talk ‘Latency: the #1 metric of your cloud’. As Boyan pointed out – no two clouds are the same. However, the leading clouds all have one thing in common: they deliver on metrics, which matter to the customer. In this session Boyan examined and presented his findings on leading clouds, demonstrating why low latency is the thing that makes a cloud stand out. Watch Boyan’s talk:

Towards the end of Boyan’s talk we weren’t sure whether there would be a fifth talk, or we would be enjoying the hospitality of the Ticketmaster bar a little sooner than anticipated. All day, Grégoire Lamodière (DIMSI) had been struggling to get to London from Paris, due to disruption to Eurostar services. We had already moved his talk to the final slot of the day, and with just a few minutes to spare, he arrived! Grégoire’s talk was ‘Using message broker to extend cloud features’. As he explained, many use cases involve communication between CloudStack admin (provider) and instances (end user) regarding configuration, build and management. Grégoire presented the DIMSI team’s communication framework that enables managing user infrastructure on Windows and Linux systems from a centralized panel. Grégoire’s full talk is on our channel:

After the final ‘official’ talk of the day, Mike (playing the part of bar tender) opened the bar and we enjoyed a couple of drinks and the unofficial discussions started. We were then truly spoiled as Computacenter led us to a nearby pub and carried on buying the drinks! As usual, a fantastic event made so by the CloudStack community. Great attendance from all over Europe (including a heroic effort from Grégoire), and varied, interesting talks (thanks to Mike, Bobby, Wido, Boyan, and Grégoire). Huge thanks to Ticketmaster for hosting and providing a very cool venue (have I mentioned the slide?), and thanks to Computacenter for their generosity. We are already planning the next CSEUG which will be in Sofia, Bulgaria, on June 13 (registration open) – we are looking for talks, so if you want to come along and give a talk, please let me know at steve.roles@shapeblue.com. See you soon!

All the day’s talks were recorded, and are available on the ShapeBlue YouTube channel.

 

Our presenter’s slides can be found on SlideShare:

Giles: https://www.slideshare.net/ShapeBlue/giles-sirett-cloudstack-news

Mike: https://www.slideshare.net/ShapeBlue/mike-rowell-our-journey-to-a-next-generation-cloud

Bobby: https://www.slideshare.net/ShapeBlue/boris-stoyanov-some-new-features-in-apache-cloudstack

Boyan: https://www.slideshare.net/ShapeBlue/boyan-ivanov-latency-the-1-metric-of-your-cloud

There was a definite feel of Christmas in the air in London as we made our way to last Thursday’s (December 13) winter meetup of the Cloudstack European User Group (CSEUG), and that only increased as we arrived at the BT Centre near St. Paul’s and saw the big Christmas tree in reception!

A great turnout for this, the last meetup of 2018, and a great representation of the CloudStack community in Europe with people travelling from Germany, Serbia, Glasgow, Switzerland and Latvia to name but a few. After a quick lunch we took our seats, and Giles Sirett (chairman of the user group) welcomed everyone and got the event started with introductions and CloudStack news.

Firstly, Giles spoke about software updates and new releases. CloudStack 4.11 is an LTS (long term support) release and included more than 250 new capabilities and a big step towards zero downtime upgrades, 4.11.2 has just been released (including 71 fixes), 4.11.3 is coming soon and 4.12 is in planning. Giles then mentioned CloudStack events starting with the recent CloudStack Collaboration Conference in September (Montreal), and events for 2019 – the next CSEUG in March (London), and the next Collaboration Conference in September (Las Vegas). During Giles’ presentation, Maurice Nettisheim (Head of Cloud Compute for BT) took to the stage to say a few words about BT’s ongoing use of CloudStack in their IaaS platform and their continued support and involvement in the CloudStack community.

Giles slides contain much more information:

After Giles, Paul Angus gave us an update on ShapeBlue’s CloudStack Container Service (CCS), giving us a walkthrough of the recently released update.This update brings CCS bang up-to-date by running the latest version of Kubernetes (v1.11.3) on the latest version of Container Linux. CCS also now makes use of CloudStack’s new CA framework to automatically secure the Kubernetes environments it creates. Paul’s talks and slides are always packed with detail:

Olivier Lambert of XCP-ng & Xen Orchestra took the floor next to tell us about the current state of the project. For those that are not familiar, XCP-ng is an opensource, community powered hypervisor based on Xen. It is easy to upgrade from XenServer (keeping all VMs, settings etc.), 100% API compatible, requires no license and has no feature restrictions.

Please take a look through Olivier’s slides for much more on this fascinating subject:

After a short break, we welcomed Ingo Jochim and Andre Walter (itelligence) with their talk entitled ‘How our cloud works’. They talked through full automation with Ansible for all infrastructure components of the cloud with CloudStack, check_mk, LDAP and more, with all functionality available through a customer portal, also covering how the setup is fully scalable for larger landscapes.
Ingo and Andre’s slides right here:

Next up was Adam Dagnall (Cloudian) with ‘Advanced S3 compatible storage integration in CloudStack’. To provide tighter integration between the S3 compatible object store and CloudStack, Cloudian has developed a connector to allow users and their applications to utilize the object store directly from within the CloudStack platform in a single sign-on manner with self-service provisioning. Additionally, CloudStack templates and snapshots are centrally stored within the object store and managed through the CloudStack service. The object store offers protection of these templates and snapshots across data centres using replication or erasure coding. Adam went into the feature-set in great detail, and his slides provide much more information:

Last talk of the day, and the honours fell to Andrija Panic (Hiag Data) with ‘CloudStack – 5 years in production’. Andrija shared real world experience of designing, deploying and managing a CloudStack public cloud, explaining how high availability for the CloudStack management components was implemented and discussing different storage technologies and networking models used, as well as the challenges faced. Andrija also presented alternate methods for deploying CloudStack as regards to regions / zones / pods, and also touched on physical networking, finally looking at the different CloudStack guest networking models available (from Basic Zone / Shared Networks to all the Advanced Zone’s networking models) and when to use each of them.
Andrija went into a lot of detail and I encourage you to look through his slides:

After Andrija had finished answering questions, Giles wrapped things up and we moved to a local pub, where I am pleased to say that conversation and collaboration continued into the night, with what rapidly became the unofficial ‘CloudStack Christmas Party’! Huge thanks to BT for providing a first-rate venue and lunch, and to all our speakers, who make these events so interesting and such a success.

The next CloudStack User Group meetup will be on Thursday, March 14, and will be hosted by our friends at Ticketmaster here in London. Please register here!

All the talks were recorded and will be made available shortly on the ShapeBlue YouTube channel.

Thursday, September 13 saw us back at the Early Excellence Centre, Canada Water, for the (late) Summer meetup of the CloudStack European User Group. As usual, a great turnout and representation of the community and Europe – with attendees traveling from Germany, Switzerland, Bulgaria, Latvia, Poland, and further afield from Ukraine. There were even a few of us there from the UK!

After we’d caught up with old friends and greeted new ones, we had a bite to eat and took our seats for the talks. Giles Sirett (ShapeBlue CEO and chairman of the CloudStack European User Group) was first up, starting with introductions, a run through the day’s agenda, and CloudStack news – and this past few months has seen lots of activity and development, including the release of the latest LTS branch of CloudStack (4.11), with 4.11.2 due soon. CloudStack 4.11 included more than 250 new capabilities, such as new host HA framework and Prometheus integration, whilst the 4.11.1 release brought us a step closer to ‘near zero downtime upgrades’ with a major refactor of the virtual router. Speaking of activity – approximately 800 downloads of CloudStack per month (in the last 6 months) shows continued strong adoption of the technology.

Giles then looked to the future, talking through upcoming events… and we were in Montreal for the CloudStack Collaboration Conference just last week! It was a fabulous event in a great city, and please see my blog for a roundup and some more information. Of course Giles also mentioned our next user group meetup – London, December 13, hosted by our friends at BT (London). Giles finished up with a call for users of CloudStack to talk more about it. For more information on that, and everything Giles talked about, here are his slides:

Giles then introduced our first featured speaker of the day – Paul Angus (VP Technology at ShapeBlue), with his talk: Backup & Recovery in CloudStack. As Paul explained – CloudStack users currently only have snapshots as a form of VM backup. With the Backup and Recovery Framework, end users will be presented with the features and functions that they have come to expect outside of ‘the cloud’, while cloud providers will be able to leverage the advantages of using enterprise backup and recovery products. In this talk, Paul explained some features of the forthcoming backup and recovery feature, the user experience and demonstrated the Veeam plugin working with the backup and recovery framework. This is a highly anticipated feature, and Paul’s slides are a treasure trove of information and detail:

Following Paul, Dag Sonstebo took control of the laser pointer. Dag is a Cloud Architect here at ShapeBlue and had chosen as his topic the CloudStack usage service. Dag started by explaining that the usage service is used to track consumption of resources in Apache CloudStack for reporting and billing purposes, before giving an overview of how the service is installed and configured. Dag then dived deeper into how data is processed from the database into the different usage types (VMs, network usage, storage, etc.), before being aggregated into billable units or time slices in the usage database.

The talk included several examples on how to query and report on this usage data, and looked at general maintenance and troubleshooting of the service. This really was a deep dive, as evidenced by Dag’s extensive slides:

After a brief interlude to grab coffee and some fresh air, next up was Olivier Lambert, the creator of Xen Orchestra and XCP-ng. Starting by talking about Citrix XenServer, Olivier explained why he developed an alternative that is truly open-source. He talked us through Xen Orchestra, before moving onto XCP-ng – a fork of XenServer removing all restrictions that were put in place with the free Citrix version. This is an exciting project, already proven and widely adopted. Olivier and his team continue to develop new functionality with a fast-growing community and have an exciting roadmap in place for future development. Olivier’s slides from his presentation are right here:

After Olivier we welcomed Vladimir Melnik to the podium (all the way from Ukraine, and I think the person who traveled the furthest). Vladimir is a co-founder of the first IaaS provider in Ukraine – Tucha, and his talk was ‘Building a redundant CloudStack management cluster’. Starting with a brief history of Tucha, Vlad covered building and maintaining an open-source-driven clustered environment for the Apache CloudStack management server with GNU Linux, HAProxy, HeartBeat, Bind, OpenLDAP and other tools. Vladimir’s slides are both entertaining and very interesting:

The honour of the last talk of the day fell to Boyan Ivanov of Storpool, providing advice on building software-defined clouds. Boyan posed the question ‘why software defined?’ and went on to answer the question quite comprehensively! Infrastructure is becoming more and more ‘software defined’, and Boyan illustrated how this should mean increased profitability, putting forward the business case for a software defined stack. Boyan was then good enough to provide several tactical tips and a free reference design!

Take a look through Boyan’s slides:

Once Boyan had finished taking questions, we all headed out to the nearest hostelry, and conversation continued into the night. A nice touch (indicative of the great CloudStack community) was that when it was time to say goodbye most people said ‘see you in Montreal’!

Thanks to Early Excellence for providing a first-class venue and refreshments, and huge thanks to the day’s speakers – Paul, Dag, Olivier, Vlad and Boyan, all of whom were good enough to donate their time, and in most cases travel great distances to share their expertise.

The next meetup of the CloudStack European User Group will be in London, on Thursday, December 13 and you can register here. We are always looking out for speakers with interesting and relevant subjects, and if you are interested in talking, please contact us.

All talks were recorded in full and can be found on our ShapeBlue YouTube channel:

Giles Sirett: https://youtu.be/Ls_HakbyxUU

Paul Angus: https://youtu.be/ZVThUKPeC_w

Dag Sonstebo: https://youtu.be/I5I7eduWHRQ

Olivier Lambert: https://youtu.be/KWBCKvwvnUc

Vladimir Melnik: https://youtu.be/aBNMysDoi5w

Boyan Ivanov: https://youtu.be/wt4pqTZ57OY

Thanks for reading, and I hope to see you at the next event!


We’re here in Montreal for the CloudStack Collaboration Conference, and it’s been a fantastic event with more to come! We’ve had two full days of back to back talks over two tracks, with subjects ranging from storage, billing and diagnostics through to containers, automation and monitoring… and everything in between. Mike Tutkowski (CloudStack VP) set the tone with his keynote at the beginning of the first day, asking the question ‘why are we here?’ The answer? To learn, work together​, share ideas​ and share problems. These fundamentals are what makes for a great community, and what makes Apache CloudStack such a great product. We have never really known just how widely adopted CloudStack is, so we have (for the first time) undertaken some in-depth analysis which Mike shared. In the last 12 months CloudStack management server packages were downloaded 116,796 times from 21,202 different IP addresses. We think this means that worldwide there are about 20,000 CloudStack clouds in production! Mike also mentioned several organisations that have recently adopted CloudStack, including Ticketmaster, from whom we saw a talk illustrating how they deployed their global cloud environment using Apache CloudStack.

The CloudStack community is full of smart, committed, talented people passionate about what they do, and this is clear from the quality and delivery of the talks, and the collaboration before and after. They aren’t just repeating facts or reading what has been written for them – they are talking from first hand experience, often about features and functionality they have personally developed and committed to the project. Thanks to the community, CloudStack is constantly being improved and developed by these real-world users and operators.

So we’re into day three, which means no more CloudStack talks. However – as I said, the event is far from finished. Today (Wednesday) we have an all-day hackathon – a room full of people working together on shared goals and ideas, the sole purpose to talk and share new ideas, and make CloudStack even better!

Every time I attend a CloudStack conference, I am privileged to spend time with a community who genuinely enjoy what they do, and I come away having made new friends, and having learnt something new. I am already excited about next year’s event, and seeing some of our new friends in London at our next CloudStack meetup (December 13).

Sincere thanks to the Apache Software Foundation (our conference co-locates every year with Apachecon). It’s always a well organised and well attended event, and we are delighted to be associated with it. Thanks also to the city of Montreal – a beautiful city which I hope to visit again soon.

All the CloudStack talks were recorded and will be published to Apache.org and our YouTube channel very soon.

    

  

 

 

 

 

 

On Thursday, April 19 the CloudStack community joined up with the Ceph community for a combined event in London, and what an event it was! The meetup took place at the Early Excellence Centre at Canada Water, on a beautiful, sunny day (in fact the hottest day of the year so far), and registration started early with people enjoying coffee and pastries by the canal.

CloudStack & Ceph combined morning sessions

Once everyone had enjoyed breakfast, we all settled down as Wido den Hollander (of 42on) took to the podium to welcome everyone, explain a bit about how the event had come about, and talk through both technologies (CloudStack and Ceph), including how well they work together. Having set the scene Wido was ready to deliver the first presentation of the day, which was prepared to appeal to both communities – ‘Building a highly available cloud with Ceph and CloudStack’. In this talk, Wido talked about how he came across and started using Ceph, and how his company utilises both Ceph and CloudStack to offer services. To quote Wido… “We manage tens of thousands of Virtual Machines running with Ceph and CloudStack and we think it is a winning and golden combination”! This first talk really got the event off to a flying start, with plenty of questions and interaction from the floor, and Wido’s slides can be found here:

Next up was John Spray (RedHat) with a talk entitled ‘Ceph in Kubernetes’ John talked through Kubernetes, Rook, and why and how it all works together. Lot’s more detail in John’s slides:

After a short break, it was the turn of Phil Straw (SoftIron) to present. Phil’s talk was ‘Ceph on ARM64’, and covered all the fundamentals including some real world results from SoftIton’s appliance. Take a look through Phil’s slides:

Sebastien Bretschneider (intelligence) was last up before lunch, with ‘Our way to Ceph’ – another great talk for both communities as he talked about his experience using Ceph with CloudStack. Starting with some background, Sebastien talked through lessons learnt and how the platform looks now. More information in Sebastien’s slides:

CloudStack afternoon sessions

After lunch, the day split into two separate tracks, the first dedicated to CloudStack (the CloudStack European User Group – CSEUG), the second to Ceph. As with the morning, both rooms were very busy, with full agendas. As I focussed on CloudStack, I cover the CloudStack talks here (at the end of the article I include links to the Ceph presentations).

Giles Sirett (CSEUG chairman) brought the room to order and started with introductions and CloudStack news, which (as always) there is lots of, including new releases – 4.9.3, 4.10, and the latest LTS release – 4.11. Sticking with 4.11, Giles touched on some of the new features that have been introduced, such as new host HA framework, new CA framework and Prometheus integration (much more detail in Paul’s later presentation). Giles also mentioned that we are moving towards zero-downtime upgrades – a highly anticipated and much needed enhancement! What is exciting about this community is that nearly all of the 4.11 features were contributed or instigated by USERS… and speaking of new features – this was someone’s response to Citrix changing the XenServer licencing model: https://xcp-ng.org/

Giles then advised us about some dates or our diaries – lots of upcoming CloudStack events, and shared how CloudStack is growing in awareness and use – we are seeing 800 downloads per month, just from our (ShapeBlue) repo, and have several new, large scale private CloudStack clouds being deployed this year. Please see Giles slides for much more information:

Once Giles had finished taking questions, he introduced the first presentation, and this was both Wido den Hollander and Mike Tutkowski (SolidFire) for a brief retrospective of Wido’s year as VP
of the CloudStack project, and welcoming Mike as the new VP! This was a quick introduction, and next up was Antoine Coetsier (Exoscale) with his talk entitled ‘Billing the cloud’.

Starting with Exoscale’s background, Antoine talked through pros and cons of different approaches, and explained Exoscale’s solution. Antoine’s slides are here:

As mentioned earlier, next up was Paul Angus (ShapeBlue), with ‘What’s new in ACS 4.11’, which includes 100s of updates, over 30 new features and the best automated test coverage yet. Paul went through user features, operator features and integrations. This demonstrated just how much work and development is going into CloudStack, and if you are interested to see just how much work and development, I urge you to read through Paul’s comprehensive slides:

After a short break, Giles introduced the next talk of the day – Boyan Krosnov (Storpool), ‘Building a software-defined cloud’. After a little background, Boyan started by explaining how a hyper-converged cloud infrastructure looks, and then talked about how Storpool works with CloudStack.

Ivan’s talk was filmed: https://youtu.be/_d_xnqpmY0M, and his slides can be found here:

Next up was Mike Tutkowski (SolidFire), who talked us through managed storage in CloudStack, and then presented a live demo so we could see first-hand how it works. As Mike works for SolidFire this was the storage he used for his demo!

The end of this last talk of the CloudStack track coincided with the end of the last Ceph talk next door, so we all congregated together to hear some last thoughts and acknowledgements from Wido. This was the first collaborative day between Ceph and CloudStack, and it was a resounding success, thanks in no small part to the vibrant communities that support these technologies. Bringing the event to a close, we all moved the conversations to a nearby pub, where the event continued late into a beautiful London evening.

Thanks to our sponsors, without which these great events wouldn’t be possible – 42on, RedHat and ShapeBlue (us!) for providing the venue and refreshments, and to SoftIron for providing the evening drinks.

Ceph afternoon sessions

Wido den Hollander – 10 ways to break your Ceph cluster

Nick Fisk – low latency Ceph

The November EU user group was held in the lovely city of Leipzig in Saxony, Germany. First of all a great thanks to Sven Vogel and his team at Kupper Computer who hosted this event, which had a good turnout from the German CloudStack user community.

Sven started off the afternoon with welcome and introductions, before handing over to first speaker Thomas Heil from Terminal Consulting. Thomas’ gave a very interesting talk on the use of HashiCorp’s Terraform to build infrastructure in CloudStack. He started with an introduction into the basics of Terraform, and how this is configured to work with CloudStack as the cloud backend, before braving a live demo of how to build a full VPC hosting a LAMP stack with front end web server load balancing, all using Terraform. A very useful topic covered in great detail – and well done for the successful live demo. We will update this blog post with Thomas’ slide deck in due course.

Following a break – during which a number of CloudStack features and challenges were discussed over drinks and sandwiches – Dag Sonstebo gave his talk on the CloudStack usage service. This service is used to track all consumption of resources in CloudStack for reporting and billing purposes. Dag went through how the service is installed and configured, before diving deeper into how the service processes data from the CloudStack database into the different usage types (VMs, network usage, storage, etc.), before aggregating this into billable units or time slices in the cloud_usage database. He followed up with a number of examples on how to query and report on this usage data, before looking at general maintenance and troubleshooting of the service. All in all a useful topic showing how the resource usage can be tracked back to CloudStack accounts. More information in Dag’s slide deck below. Also watch out for an accompanying ShapeBlue blog post going into more detail on this topic in the coming weeks.

Finishing off the evening was Sebastian Bretschneider from BIT Group GmbH (part of Itelligence) who gave a very interesting talk on providing end user VM build capability in CloudStack using Ansible. For a larger service provider creating templates for every compute role the end user may want is a huge overhead, and Itelligence are trying to overcome this by using fewer base templates with an automation framework on top which builds the different compute roles, e.g. a web server, a DB server, and so on. Sebastian talked us through how they have overcome the challenges with providing Ansible connectivity into isolated user networks, and how Ansible playbooks are used on demand to automate and build end users’ infrastructure. He also discussed how the solution is being integrated into their custom CloudStack portal to provide the end user with a service catalogue for builds. As always – a very interesting talk from BIT Group, and we are looking forward to seeing more of how their solution works for their customers.

The user group meeting was finished off with an informal discussion on various CloudStack topics – especially the new features in CloudStack 4.10 and upcoming 4.11. We continued the discussion with some good German hospitality in a local pub.

All in all a very successful user group – and we are looking forward to the next one (which may be in Frankfurt in the first part of 2018 – to be confirmed). Again thanks to Sven and Kupper Computing for organising the meetup, for hosting us and providing food and drink.

Last Thursday saw us back at BT, London, for the Summer meetup of the CloudStack European User Group (CSEUG). A full schedule meant a prompt start, and Giles Sirett (CSEUG chairman) brought the meeting to order and started with introductions and CloudStack news.

Since the last meetup, CloudStack 4.9.2 has been released, with 4.9.3 due for early September (4.9.x is the LTS branch). CloudStack continues to develop and add new features, and the recent release of 4.10 introduced OOBM for hosts, user defined roles, additional hypervisor support (including XenServer 7) as well as several other features and improvements – see Giles slides for more detail:

Giles went on to report on recent events – namely the Spring CSEUG meetup in Prague, and the CloudStack Collaboration Conference in Miami – both well attended events that showcased not only the depth of interest in CloudStack, but also the vibrant and continually growing community.
The first speaker was Mike Tutowski (NetApp Solidfire), with a talk entitled ‘Leveraging VMWare Storage I/O Control (SIOC) in CloudStack’. Starting with a high level overview of CloudStack and storage, Mike then focused in on SIOC and how it works with CloudStack. This was a detailed talk full of information, and Mike’s slides can be found here:
Next up was ‘Policy Driven SDN in CloudStack’, a talk from Sigert Goeminne of Nuage Networks. Starting with an introduction to Nuage, SDN and the existing Nuage VSP plugin, Sigert went on to talk through the exciting new features coming with CloudStack 4.11, such as VSD managed subnets, and migrating a native cloud to Nuage. More detail in Sigert’s slides:
After a very brief coffee break (as I said – a full schedule!), Giles welcomed Paul Angus of ShapeBlue, with a talk about Cloudstack and NFV. Paul’s talk stepped through an introduction to NFV, what CloudStack can do with NFV, and some of the exciting work currently going on around NFV in CloudStack… pretty much covering all angles! Paul’s talk was (as always) packed with detail, and you will find much more information in Paul’s slides:

The next talk was all about a new CloudStack UI – given by Ilya Zolotukhin of Bitworks (who joined us all the way from Siberia). Ilya explained why they started the project: based on their experience of the UI they knew about its shortcomings and knew what functionality they needed it to provide. Ilya then compared the native UI with the new one, and went on to talk through the UX goals before going into technical detail and the future direction that the project may take. This was a presentation packed with detail, and I recommend you take a look through Ilya’s slides:

Continuing to be spoilt by the number and quality of talks, next up was ‘Monitoring CloudStack and Components’, presented by Alexander Stock of leading SAP integrators Bit.Group GmbH. This was an in depth talk about how Bit.group use Nagios and a front-end tool called Check_MK, what they monitor in CloudStack, and how the monitoring is distributed an automated. Alexander’s slides are right here:

Time was running short, so we moved right along with the last talk of the day: ‘Running Cloudstack with IPv6’, a talk given Wido Den Hollender of PCExtreme. PCExtreme use CloudStack to deliver their hugely successful Aurora compute service. Wido is not only a CloudStack user and developer, he is also the current CloudStack VP. Opening with some reasons for IPv6 (eg. we are almost out of IPv4 addresses), he went on to state that ‘with the release of CloudStack 4.10 IPv6 is fully supported in basic networking’. Explaining how PCExtreme is currently using IPv4 and IPv6, he also said that they are currently working on providing IPv6 only VMs. Wido’s slides have a lot of information:

Following a Q&A session, the discussions continued seamlessly in a local pub, where the day’s speakers continued to answer questions and discuss CloudStack and future collaboration.

Thanks to all our speakers, to BT for providing lunch and a fabulous venue, and to all attendees for once again proving what a varied and active community CloudStack has. Next event TBC, but should be around November and will be announced on the ShapeBlue website.