Posts

Our first meetup of 2019 saw us at a new venue – Ticketmaster’s London HQ, and if you’re a music lover it certainly takes the prize for coolest meeting venue yet! Walls covered with pictures of rock stars and a stage complete with guitars and Marshall amps (not to mention pinball machines and a bar) created a real buzz of excitement before the meeting had even started. Once everyone had met up with friends, taken photos and finished lunch, Giles Sirett (CSEUG chairman) called the meeting to order, and kicked the day off with CloudStack news.

Giles talked us through the current and upcoming releases of CloudStack, and the new release of Cloudmonkey (6.0), before ‘unofficially’ announcing the new VP of Apache CloudStack – our very own Paul Angus! Moving onto market news, Giles introduced a thought-provoking topic, starting by referencing an article titled ‘What happened to OpenStack?’, before moving onto the different marketing approaches taken by the technologies.

We then heard about upcoming events – the next CSEUG will be in Sofia in June (register here), and we are currently looking for speakers. The CSEUG returns to London in October (and we are working with Ceph on making this another collaboration event), and we have CloudStack Collaboration Conferences in April (Brazil) and September (Las Vegas). Again – the Call For Participation is open for Las Vegas. All the information provided by Giles can be found by watching his talk:

Giles then introduced our first guest speaker onto the stage – Mike Rowell (Director, Platform Infrastructure) of our hosts Ticketmaster, with a talk titled ‘Our journey to a next generation cloud’. Mike did indeed take us on a journey, first explaining what challenges they needed to overcome, and what solutions they initially implemented, before discussing their investigations into a scalable cloud solution. These investigations led them to Apache CloudStack, and Mike went on to share what issues he experienced, as well as what other tools they use, such as Ansible, Terraform and Prometheus in the stack. Mike finished his talk by expanding on some features he would like to see in CloudStack.

Next to the stage was Bobby Stoyanov (ShapeBlue), talking us through some of the new features in CloudStack. These new features include: more sophisticated options for specifying pod and cluster while deploying a VM; running and retrieving diagnostics on the VR; sending additional configuration to VMs; and adding options to cleanup additional data disks when destroying a VM. It’s always great to hear about new features, and to see evidence of the continuing innovation and commitment to the project from the community. Bobby ‘dived deep’ on each feature, so I recommend you watch his talk:

After a short break, we welcomed Wido den Hollander from PCextreme, who talked about flexible networking for scaling a cloud environment,. As Wido explained – regular layer 2 VLANs have their limitations when it comes to scalability, and VXLAN overcomes these limitations, making it easier to scale out your CloudStack deployment. As of CloudStack 4.12, VXLAN can use IPv6, and Wido talked about Advanced networking + IPv6 + VXLAN which he is putting into production right now with the 4.12 release. As usual, Wido covered his topic comprehensively, and if you want to hear more, watch his talk:

We then welcomed Boyan Ivanov (Storpool) with his talk ‘Latency: the #1 metric of your cloud’. As Boyan pointed out – no two clouds are the same. However, the leading clouds all have one thing in common: they deliver on metrics, which matter to the customer. In this session Boyan examined and presented his findings on leading clouds, demonstrating why low latency is the thing that makes a cloud stand out. Watch Boyan’s talk:

Towards the end of Boyan’s talk we weren’t sure whether there would be a fifth talk, or we would be enjoying the hospitality of the Ticketmaster bar a little sooner than anticipated. All day, Grégoire Lamodière (DIMSI) had been struggling to get to London from Paris, due to disruption to Eurostar services. We had already moved his talk to the final slot of the day, and with just a few minutes to spare, he arrived! Grégoire’s talk was ‘Using message broker to extend cloud features’. As he explained, many use cases involve communication between CloudStack admin (provider) and instances (end user) regarding configuration, build and management. Grégoire presented the DIMSI team’s communication framework that enables managing user infrastructure on Windows and Linux systems from a centralized panel. Grégoire’s full talk is on our channel:

After the final ‘official’ talk of the day, Mike (playing the part of bar tender) opened the bar and we enjoyed a couple of drinks and the unofficial discussions started. We were then truly spoiled as Computacenter led us to a nearby pub and carried on buying the drinks! As usual, a fantastic event made so by the CloudStack community. Great attendance from all over Europe (including a heroic effort from Grégoire), and varied, interesting talks (thanks to Mike, Bobby, Wido, Boyan, and Grégoire). Huge thanks to Ticketmaster for hosting and providing a very cool venue (have I mentioned the slide?), and thanks to Computacenter for their generosity. We are already planning the next CSEUG which will be in Sofia, Bulgaria, on June 13 (registration open) – we are looking for talks, so if you want to come along and give a talk, please let me know at steve.roles@shapeblue.com. See you soon!

All the day’s talks were recorded, and are available on the ShapeBlue YouTube channel.

 

Our presenter’s slides can be found on SlideShare:

Giles: https://www.slideshare.net/ShapeBlue/giles-sirett-cloudstack-news

Mike: https://www.slideshare.net/ShapeBlue/mike-rowell-our-journey-to-a-next-generation-cloud

Bobby: https://www.slideshare.net/ShapeBlue/boris-stoyanov-some-new-features-in-apache-cloudstack

Boyan: https://www.slideshare.net/ShapeBlue/boyan-ivanov-latency-the-1-metric-of-your-cloud

Thursday, September 13 saw us back at the Early Excellence Centre, Canada Water, for the (late) Summer meetup of the CloudStack European User Group. As usual, a great turnout and representation of the community and Europe – with attendees traveling from Germany, Switzerland, Bulgaria, Latvia, Poland, and further afield from Ukraine. There were even a few of us there from the UK!

After we’d caught up with old friends and greeted new ones, we had a bite to eat and took our seats for the talks. Giles Sirett (ShapeBlue CEO and chairman of the CloudStack European User Group) was first up, starting with introductions, a run through the day’s agenda, and CloudStack news – and this past few months has seen lots of activity and development, including the release of the latest LTS branch of CloudStack (4.11), with 4.11.2 due soon. CloudStack 4.11 included more than 250 new capabilities, such as new host HA framework and Prometheus integration, whilst the 4.11.1 release brought us a step closer to ‘near zero downtime upgrades’ with a major refactor of the virtual router. Speaking of activity – approximately 800 downloads of CloudStack per month (in the last 6 months) shows continued strong adoption of the technology.

Giles then looked to the future, talking through upcoming events… and we were in Montreal for the CloudStack Collaboration Conference just last week! It was a fabulous event in a great city, and please see my blog for a roundup and some more information. Of course Giles also mentioned our next user group meetup – London, December 13, hosted by our friends at BT (London). Giles finished up with a call for users of CloudStack to talk more about it. For more information on that, and everything Giles talked about, here are his slides:

Giles then introduced our first featured speaker of the day – Paul Angus (VP Technology at ShapeBlue), with his talk: Backup & Recovery in CloudStack. As Paul explained – CloudStack users currently only have snapshots as a form of VM backup. With the Backup and Recovery Framework, end users will be presented with the features and functions that they have come to expect outside of ‘the cloud’, while cloud providers will be able to leverage the advantages of using enterprise backup and recovery products. In this talk, Paul explained some features of the forthcoming backup and recovery feature, the user experience and demonstrated the Veeam plugin working with the backup and recovery framework. This is a highly anticipated feature, and Paul’s slides are a treasure trove of information and detail:

Following Paul, Dag Sonstebo took control of the laser pointer. Dag is a Cloud Architect here at ShapeBlue and had chosen as his topic the CloudStack usage service. Dag started by explaining that the usage service is used to track consumption of resources in Apache CloudStack for reporting and billing purposes, before giving an overview of how the service is installed and configured. Dag then dived deeper into how data is processed from the database into the different usage types (VMs, network usage, storage, etc.), before being aggregated into billable units or time slices in the usage database.

The talk included several examples on how to query and report on this usage data, and looked at general maintenance and troubleshooting of the service. This really was a deep dive, as evidenced by Dag’s extensive slides:

After a brief interlude to grab coffee and some fresh air, next up was Olivier Lambert, the creator of Xen Orchestra and XCP-ng. Starting by talking about Citrix XenServer, Olivier explained why he developed an alternative that is truly open-source. He talked us through Xen Orchestra, before moving onto XCP-ng – a fork of XenServer removing all restrictions that were put in place with the free Citrix version. This is an exciting project, already proven and widely adopted. Olivier and his team continue to develop new functionality with a fast-growing community and have an exciting roadmap in place for future development. Olivier’s slides from his presentation are right here:

After Olivier we welcomed Vladimir Melnik to the podium (all the way from Ukraine, and I think the person who traveled the furthest). Vladimir is a co-founder of the first IaaS provider in Ukraine – Tucha, and his talk was ‘Building a redundant CloudStack management cluster’. Starting with a brief history of Tucha, Vlad covered building and maintaining an open-source-driven clustered environment for the Apache CloudStack management server with GNU Linux, HAProxy, HeartBeat, Bind, OpenLDAP and other tools. Vladimir’s slides are both entertaining and very interesting:

The honour of the last talk of the day fell to Boyan Ivanov of Storpool, providing advice on building software-defined clouds. Boyan posed the question ‘why software defined?’ and went on to answer the question quite comprehensively! Infrastructure is becoming more and more ‘software defined’, and Boyan illustrated how this should mean increased profitability, putting forward the business case for a software defined stack. Boyan was then good enough to provide several tactical tips and a free reference design!

Take a look through Boyan’s slides:

Once Boyan had finished taking questions, we all headed out to the nearest hostelry, and conversation continued into the night. A nice touch (indicative of the great CloudStack community) was that when it was time to say goodbye most people said ‘see you in Montreal’!

Thanks to Early Excellence for providing a first-class venue and refreshments, and huge thanks to the day’s speakers – Paul, Dag, Olivier, Vlad and Boyan, all of whom were good enough to donate their time, and in most cases travel great distances to share their expertise.

The next meetup of the CloudStack European User Group will be in London, on Thursday, December 13 and you can register here. We are always looking out for speakers with interesting and relevant subjects, and if you are interested in talking, please contact us.

All talks were recorded in full and can be found on our ShapeBlue YouTube channel:

Giles Sirett: https://youtu.be/Ls_HakbyxUU

Paul Angus: https://youtu.be/ZVThUKPeC_w

Dag Sonstebo: https://youtu.be/I5I7eduWHRQ

Olivier Lambert: https://youtu.be/KWBCKvwvnUc

Vladimir Melnik: https://youtu.be/aBNMysDoi5w

Boyan Ivanov: https://youtu.be/wt4pqTZ57OY

Thanks for reading, and I hope to see you at the next event!


We’re here in Montreal for the CloudStack Collaboration Conference, and it’s been a fantastic event with more to come! We’ve had two full days of back to back talks over two tracks, with subjects ranging from storage, billing and diagnostics through to containers, automation and monitoring… and everything in between. Mike Tutkowski (CloudStack VP) set the tone with his keynote at the beginning of the first day, asking the question ‘why are we here?’ The answer? To learn, work together​, share ideas​ and share problems. These fundamentals are what makes for a great community, and what makes Apache CloudStack such a great product. We have never really known just how widely adopted CloudStack is, so we have (for the first time) undertaken some in-depth analysis which Mike shared. In the last 12 months CloudStack management server packages were downloaded 116,796 times from 21,202 different IP addresses. We think this means that worldwide there are about 20,000 CloudStack clouds in production! Mike also mentioned several organisations that have recently adopted CloudStack, including Ticketmaster, from whom we saw a talk illustrating how they deployed their global cloud environment using Apache CloudStack.

The CloudStack community is full of smart, committed, talented people passionate about what they do, and this is clear from the quality and delivery of the talks, and the collaboration before and after. They aren’t just repeating facts or reading what has been written for them – they are talking from first hand experience, often about features and functionality they have personally developed and committed to the project. Thanks to the community, CloudStack is constantly being improved and developed by these real-world users and operators.

So we’re into day three, which means no more CloudStack talks. However – as I said, the event is far from finished. Today (Wednesday) we have an all-day hackathon – a room full of people working together on shared goals and ideas, the sole purpose to talk and share new ideas, and make CloudStack even better!

Every time I attend a CloudStack conference, I am privileged to spend time with a community who genuinely enjoy what they do, and I come away having made new friends, and having learnt something new. I am already excited about next year’s event, and seeing some of our new friends in London at our next CloudStack meetup (December 13).

Sincere thanks to the Apache Software Foundation (our conference co-locates every year with Apachecon). It’s always a well organised and well attended event, and we are delighted to be associated with it. Thanks also to the city of Montreal – a beautiful city which I hope to visit again soon.

All the CloudStack talks were recorded and will be published to Apache.org and our YouTube channel very soon.

    

  

The November EU user group was held in the lovely city of Leipzig in Saxony, Germany. First of all a great thanks to Sven Vogel and his team at Kupper Computer who hosted this event, which had a good turnout from the German CloudStack user community.

Sven started off the afternoon with welcome and introductions, before handing over to first speaker Thomas Heil from Terminal Consulting. Thomas’ gave a very interesting talk on the use of HashiCorp’s Terraform to build infrastructure in CloudStack. He started with an introduction into the basics of Terraform, and how this is configured to work with CloudStack as the cloud backend, before braving a live demo of how to build a full VPC hosting a LAMP stack with front end web server load balancing, all using Terraform. A very useful topic covered in great detail – and well done for the successful live demo. We will update this blog post with Thomas’ slide deck in due course.

Following a break – during which a number of CloudStack features and challenges were discussed over drinks and sandwiches – Dag Sonstebo gave his talk on the CloudStack usage service. This service is used to track all consumption of resources in CloudStack for reporting and billing purposes. Dag went through how the service is installed and configured, before diving deeper into how the service processes data from the CloudStack database into the different usage types (VMs, network usage, storage, etc.), before aggregating this into billable units or time slices in the cloud_usage database. He followed up with a number of examples on how to query and report on this usage data, before looking at general maintenance and troubleshooting of the service. All in all a useful topic showing how the resource usage can be tracked back to CloudStack accounts. More information in Dag’s slide deck below. Also watch out for an accompanying ShapeBlue blog post going into more detail on this topic in the coming weeks.

Finishing off the evening was Sebastian Bretschneider from BIT Group GmbH (part of Itelligence) who gave a very interesting talk on providing end user VM build capability in CloudStack using Ansible. For a larger service provider creating templates for every compute role the end user may want is a huge overhead, and Itelligence are trying to overcome this by using fewer base templates with an automation framework on top which builds the different compute roles, e.g. a web server, a DB server, and so on. Sebastian talked us through how they have overcome the challenges with providing Ansible connectivity into isolated user networks, and how Ansible playbooks are used on demand to automate and build end users’ infrastructure. He also discussed how the solution is being integrated into their custom CloudStack portal to provide the end user with a service catalogue for builds. As always – a very interesting talk from BIT Group, and we are looking forward to seeing more of how their solution works for their customers.

The user group meeting was finished off with an informal discussion on various CloudStack topics – especially the new features in CloudStack 4.10 and upcoming 4.11. We continued the discussion with some good German hospitality in a local pub.

All in all a very successful user group – and we are looking forward to the next one (which may be in Frankfurt in the first part of 2018 – to be confirmed). Again thanks to Sven and Kupper Computing for organising the meetup, for hosting us and providing food and drink.