Posts

Introduction

Managing user roles has been a pain for a while, as the model of having a commands.properties file that defines roles and their permissions can be hard to comprehend and use. Due to this, not many CloudStack users made any changes to the default harcoded roles and further enhanced roles. Therefore, ShapeBlue has taken the opportunity to rewrite the Roles-Based Access Control (RBAC) unit into a Dynamic Roles model. The changes allows CloudStack Root Admin to create new roles with customised permissions from the CloudStack UI by allowing / denying specific APIs. It deprecates the old fashioned commands.properties file and transfers all the rules into the DB. This is available in CloudStack 4.9.x and greater.

How it works?

Dynamic RBAC introduces a new tab in the CloudStack Console called Roles. Root Admins by default are able to navigate there and Create / Update all roles. When creating a new role, the Root Admin is able to select the rules that apply to that role, and can define a list of APIs which they could allow or deny for the role. When the user (assigned with a specific role) issues an API request, the backend checks the requested API against configured rules for the assigned role, and the user will only be able to call the API if it’s allowed on the list. If denied or not listed it won’t be possible to call the API.

How to use it?

In this example, let’s assume we want to create a Root Admin that has read-only rights on everything but “Global Settings” in CloudStack.

The following rules configuration shows an example of this custom role, that is only able to view resources. The image bellow shows the rules tab of the custom role called “read-only”. Please observe that only “list*” APIs are allowed, meaning that a user with this role will not be able to delete / update anything within CloudStack, but just use the list APIs. Also, note an addition that is denying any APIs related to configurations (*Configuration). Due to this, the user will not be able to see anything within the “Global Settings”. The order of configuring rules list is also very important – Dynamic Roles Checker iterates the list top-down, so when configuring, it is best practice to shift “Deny” rules to the top. Shifting rules is possible by simply drag-and-dropping the rule by clicking the  button. In this particular case, if “Allow list*” rule was on top of the “Deny *Configurations”, user would be able to see the Global Settings.

When the user hits an API that is Denied, he will be prompted the following generic error message:

 

OK, what happens if we upgrade to 4.9?

Dynamic Roles is available and enabled by default to all new installations after CloudStack 4.9.x release. If a user upgrades from older version (to 4.9.x or greater), Dynamic Roles will be disabled by default and it will follow the old fashioned way of handling RBAC (ie, with command.properties file.) After the upgrade existing deployments of CloudStack can be migrated to Dynamic RBAC by running a migration tool which is part of 4.9 installation. The migration tool is located at the following directory on the management server: /usr/share/cloudstack-common/scripts/util/migrate-dynamicroles.py

When running this tool it will enable Dynamic RBAC, then copy all existing hard-coded roles from commands.properties file, create the same entities in the database following the data format of Dynamic Roles. Finally, it will rename the commands.properties file to “command.properties.deprecated”, as a backup file.

Example:

python /usr/share/cloudstack-common/scripts/util/migrate-dynamicroles.py -u cloud -p cloud -h localhost -p 3306 -f /etc/cloudstack/management/commands.properties

Running this will output the following:
Apache CloudStack Role Permission Migration Tool

(c) Apache CloudStack Authors and the ASF, under the Apache License, Version 2.0 

Running this migration tool will remove any default-role permissions from cloud.role_permissions. Do you want to continue? [y/N]y

The commands.properties file has been deprecated and moved at: /etc/cloudstack/management/commands.properties.deprecated

Static role permissions from commands.properties have been migrated into the db

Dynamic role based API checker has been enabled! 

And you’re all set, no need to restart management servers! There’s a new global setting introduced with this feature called ‘dynamic.apichecker.enabled’. If it is set to “True” it means the Dynamic Roles is enabled. If by any chance if there is any failure with migration it will roll-back the procedure and will revert to the old hardcoded way of handling RBAC.

After the upgrade the rules of Root Admin Role look like this:

…meaning all APIs are allowed.

Other roles have each individual API rule explicitly added (if available). See part of the Domain Admin rules for reference:

Here’s a link to the official CloudStack Admin documentation

About the author

Boris Stoyanov is Software Engineer in testing at ShapeBlue, The Cloud Specialists. Bobby spends his time testing features for the Apache CloudStack Community and for our ShapeBlue clients.

An oft-cited limitation in Apache CloudStack is the lack of granular access controls.  Historically, when creating an account, there have been four built-in roles to choose from: Root Admin, Resource Admin, Domain Admin, and User.  Unfortunately, these built-in roles have been insufficient for the needs of many organizations, who have resorted to various workarounds.  Thankfully, this will change in the upcoming CloudStack 4.9 release with the addition of the Dynamic Role-Based API Access Checker feature.

Read more

CloudStack 4.7 (which is due in the coming weeks) will introduce a new metrics view feature throughout the familiar CloudStack interface. We built this functionality to help system architects and admins comprehend resource utilisation and drill into the data to find performance hotspots. Whilst metrics have always been available via the CloudStack API a lot of information hasn’t been readily available in the GUI, and the idea was to make resource usage more easily accessible on the various CloudStack menus without having to click through multiple pages.

Use cases

  • The metrics view allows easy resource usage monitoring from the GUI – especially when trying to narrow down high resource consumers among VM instances and disks, which in turn helps when making VM and storage migration decisions.
  • The metrics view also assists in longer term capacity planning of the CloudStack infrastructure, giving a better overview of the resource usage in zones and clusters.
  • In larger CloudStack estates the metrics view also gives a quick overview of the number of enabled / disabled zones, clusters and hosts.

 

Overview

The metrics view is implemented on the Zone, Cluster, Hosts, Instances, Primary Storage and Storage pages and is accessed via the new metrics button on the menu bar.

metricsbutton

 

On the metrics view pages there are a number of features to make the function usable:

  • Similar to other CloudStack GUI pages all metrics pages have infinite scrolling – as you scroll down more data is loaded.
  • All columns are sortable by clicking on the column heading.
  • Where CloudStack works with warning and disable limits in the global settings any value exceeding these limits will be flagged amber / red.
  • Since the metrics view pages contain a lot of data each topic heading has a collapse button (<<) which collapses the topic column, thereby allowing more page space for the remaining columns. The collapsed column is expanded again by clicking the (>>) button next to the topic heading.
  • All metrics view entries maintain the familiar Quickview button to access context specific functions.
  • All metrics pages (apart from on the Primary storage pool page) operate in a hierarchy, making it easy to drill down through Zone > Cluster > Host > Instances > Volumes.

 

metricsfunctions

 

Collapsing the “Property” topic heading above allows for better view of the remaining columns:

metricscollapsed

 

Metrics view pages

Zone view

metricszone

The Zone metrics view page gives an overview of resource usage in all zones in the CloudStack infrastructure. (Please note the screen shot above is using offline test data, hence does not represent a true view of resource usage and zone states). As described above this page will also flag any values which exceed the warning or disable threshold configured in global settings.

The Zone view page shows:

  • Zone name: clicking on the zone name will navigate to the Clusters view for that specific zone.
  • State: the enabled state of the zone.
  • State: a count of (enabled clusters)/(total clusters) in the zone.
  • CPU usage:
    • Used: the average percentage of the CPU used metric for the clusters in the zone.
    • Deviation: the max CPU usage deviation of any cluster compared to the average across all clusters.
  • CPU allocation:
    • Allocated: the average CPU allocation percentage across all clusters in the zone.
    • Total: the total CPU allocation in GHz for the zone.
  • Memory usage:
    • Used: the average of the memory used metric per cluster in the zone.
    • Deviation: the max memory usage deviation of any cluster compared to the average across all clusters.
  • Memory allocation:
    • Allocated: average of the memory allocated metric across all clusters.
    • Total: the total memory allocation in GB across all clusters in the zone.

 

Cluster view

metricscluster

Clicking on any zone name in the zone metrics view will bring up the cluster metrics view. The same view can be found under the left hand Infrastructure menu > Clusters. This view provides similar statistics to the zone view – just in the context of each cluster and it’s hosts:

  • Cluster name: clicking on the cluster name will navigate host metrics view for the cluster.
  • State: the enabled state of the cluster.
  • State: a count of (enabled hosts)/(total hosts) in the cluster.
  • CPU usage:
    • Used: the average percentage of the CPU used metric for the hosts in the zone.
    • Deviation: the max CPU usage deviation of any host compared to the average across all hosts.
  • CPU allocation:
    • Allocated: the average CPU allocation percentage across all hosts in the cluster.
    • Total: the total CPU allocation in GHz for the hosts in the cluster.
  • Memory usage:
    • Used: the average of the memory used metric per host in the zone.
    • Deviation: the max memory usage deviation of any host compared to the average across all hosts in the cluster.
  • Memory allocation:
    • Allocated: average of the memory allocated metric across all hosts.
    • Total: the total memory allocation in GB across all hosts in the zone.

 

Host view

metricshosts

The host metrics view is accessed by drilling down from the clusters metrics view, alternatively from the left hand Infrastructure menu > Hosts > metrics button. (Again please note the screen shot above is using offline test data, hence does not represent a true view of resource usage and hosts states).

The host view presents:

  • Host name: clicking on the host name will navigate to the VM instance metrics view for the host.
  • State: the enabled state of the host.
  • Instances: a count of (running VM instances)/(total VM instances) on the host.
  • CPU usage:
    • Cores: total number of CPU cores on the compute host.
    • Total: total number of CPU resources (GHz) provided by the CPUs on the host. The figure in the bracket shows the CPU overprovisioning factor applied.
    • Used: the total of CPU resources (GHz) currently used.
    • Allocated: the total of CPU resources (GHz) allocated to the instances on the host.
  • Memory usage:
    • Total: the total memory for the host. The figure in brackets shows the memory overprovisioning factor.
    • Used: the total amount of memory used by the VM instances running on the host.
    • Allocated: the total memory allocated to all VM instances on the host.
  • Network usage:
    • Read: the cumulated network read bandwidth utilisation in GB.
    • Write: the cumulated network write bandwidth utilisation in GB.

 

VM instance view

metricsinstances

The VM instance view is accessed by drilling down from the hosts view, alternatively via the left hand Instances menu.

The view provides the following data:

  • Name: the VM instance name. Clicking on the VM name will bring up the storage metrics view for the specific VM instance.
  • State: shows the running state of the VM.
  • IP address: the primary IP address for the VM instance.
  • Zone: the name of the zone the VM is running in.
  • CPU usage:
    • Cores: number of CPU cores allocated to the VM.
    • Total: total of CPU resources (GHz) allocated to the VM, i.e. (number of cores) x (CPU speed).
    • Used: current CPU usage (GHz) of the VM.
  • Memory usage:
    • Allocated: the total amount of memory allocated to the VM instance.
  • Network usage:
    • Read: the cumulated network read bandwidth utilisation by the VM (GB).
    • Write: the cumulated network write bandwidth utilisation by the VM (GB).
  • Disk usage: this view will only show data when the underlying storage system provides statistics.
    • Read: accumulated disk reads (MB) for the VM.
    • Write: accumulated disk writes (MB) for the VM.
    • IOPs: total number of IOPs (read + write) for the VM.

 

Volume usage view

metricsvols

The storage volume metrics view can be accessed from the left hand Storage menu, which will show all storage volumes managed by CloudStack, or alternatively by drilling down from the VM instances metrics view, which will show only the volumes for the specific VM.

This view shows:

  • Name: name of the storage volume (there is no further drill down from this page).
  • State: shows the attached state of the volume.
  • VM Name: the VM instance name the volume is attached to. This value is blank when the volume is detached.
  • Size: volume size (GB).
  • Type: VM root disk or a data disk.
  • Storage pool: lists the storage pool where the disk is stored.

 

 

 Primary storage view

metricspools

The Primary Storage metrics view can only be accessed from the left hand Infrastructure menu > Primary storage > metrics button.

The view lists:

  • Name: primary storage pool name. Clicking the pool name will bring up the volume metrics view for all volumes stored on this primary storage pool.
  • Property:
    • State: up/down state of the primary storage pool.
    • Scope: lists the scope of the pool, i.e. either “cluster” or “zone”.
    • Type: storage type for the pool, e.g. NetworkFilesystem, VMFS, etc.
  • Disk:
    • Used: total GB used in the storage pool.
    • Total: the total capacity of the pool. The figured in brackets lists the storage overprovisioning factor.
    • Allocated: the current amount of allocated storage (GB) in the pool.
    • Unallocated: the current amount of unallocated storage (GB) in the pool, taking overprovisioning into account.

 

Summary

All in all we hope the metrics view will be a really useful feature for anyone using CloudStack, especially those with larger CloudStack estates. As always we’re happy to receive feedback , so please get in touch with any comments, questions or suggestions.

 

About The Author

Dag Sonstebo is  a Cloud Architect at ShapeBlue, The Cloud Specialists. Dag spends most of his time designing and implementing IaaS solutions based on on Apache CloudStack.

 

Update – Following community feedback, Timeout Settings have now been added to the script

Update – The HA settings in this post also apply to XenServer 6.5.0 onwards

Warning: If you have applied Hotfix XS62ESP1004 to your XenServer 6.2 infrastructure and have not enabled Pool HA, in the event of your Pool Master going down, a Slave Host will NOT take over as Pool Master and you will lose connectivity via XenCenter. All Hosts will go into Alert state within CloudStack so you will have reduced functionality within your CloudStack Cloud. This article covers how to correctly configure Pool HA and how to manage your XenServers once HA is enabled.

Introduction

Traditionally when using Citrix XenServer with Apache CloudStack / Citrix CloudPlatform (simply referred to CloudStack for the rest of this article) the XenServer HA Feature was not enabled as CloudStack took care of all HA events, however the release of XS62ESP1004 changed a few things.

With the release of XS62ESP1004, the way XenServer handles the loss of its Pool Master has changed and rather than having to manually promote a Slave to become a Pool Master, the XenServer Pool can now ‘Self Heal’ which is great news, however you need to do some additional configuration for this magic to happen.

The important thing to understand is that CloudStack still takes care of all the VM HA events so we do not want to give XenServer control over the VMs. In fact all we are doing is enabling ‘Pool Master HA’ so that in the event of a failure of the Pool Master, a new Pool Master is elected.

Configuring Pool HA

The first step is to ensure all your Hosts in the Pool have XS62ESP1004 installed. If deploying a new batch of XenServers for use with CloudStack its best practise to install all appropriate Hotfixes before adding the Hosts into CloudStack.

Once your Hosts are in a Pool and all networking and bonds are configured etc you need to add a dedicated Storage Repository which will be used only by the Pool Master HA mechanism. I generally name this ‘MGMT-HA’ as its handling the HA of the Pool Management elements etc. This can be a NFS or iSCSI mount but only needs to be 1 GB in size. The important thing is that it is only used for HA and is not added into CloudStack as Primary Storage.

Once the SR is configured and online, the next step is to enable HA. This ‘could’ be enabled using the XenCenter UI, however there is a risk you will inadvertently enable VM HA which could cause you all sorts of problems. In addition you will probably be disabling and re-enabling this on a regular basis so putting together a simple script to enable / re-enable HA when required is a much better option.

To allow for failover of network storage controllers the ha timeout should be set to a minimum of 90 secs, this is to prevent premature self fencing.  If after testing this timeout is not long enough, increase the timeout and test again.

Add the following into a simple bash script and place on your hosts (you can run it from any Host, not just the Pool Master) Note how it references the SR name ‘MGMT-HA’ so if you use a different naming convention simply update the script to match your SR name.

#!/bin/bash
MGMTHAUUID=`xe sr-list name-label=MGMT-HA --minimal`
xe pool-ha-enable heartbeat-sr-uuids=$MGMTHAUUID ha-config:timeout=90
echo "Pool HA is now enabled"

 

Now simply run the script and wait for the confirmation message “Pool HA is now enabled” it can take a couple of minutes so be patient.

When reviewing the settings within XenCenter, you could be forgiven for thinking that HA is not enabled, when in fact it is, but only for the Pool Master and not the VMs which are still protected by CloudStack.

Pool-HA

Having Pool HA enabled unfortunately does add some extra complication to the management of your XenServer Pool so please read on to learn how to deal with failures and how to perform planned restarts etc.

Automatic Handling of Host Failures

If the Pool Master goes down a Slave will take over and all VMs which were running on the failed Pool Master will be automatically restarted on alternate Hosts, but only as long as their Compute Offering had the ‘Offer HA’ feature enabled.

CloudStack still takes care of the restarting of the VMs and will initiate the restart only after the new Pool Master has taken over and the time out controlled by the global setting ‘alert.wait’ has expired. In the system used for the following tests, alert.wait was set to 60 seconds. The default value is ‘blank’ which results in a delay of 30 mins once the Host has been detected as being down by CloudStack.

The following timings were observed on a production system which was undergoing testing. The system was running Apache CloudStack 4.3.2, the Hosts were CISCO UCS running XenServer 6.2 with all available updates applied (up to XS62ESP1014). Storage was provided by SolidFire and networking was 10GB CISCO. Ten Windows 2008 R2 VMs were running on the system along with the usual array of System VMs. Each Cluster had 4 Hosts and the tests were performed on two Clusters at the same time. All VMs were running on the Pool Masters to cause maximum impact from the simulated failure. Timings are in Minutes and Seconds from the start of the test:

Start – Pool Master Failure simulated by killing power to Hosts via CIMC

01:15 – XenCenter detected host failures and lost connection

03:00 – XenCenter reconnected to new Pool Masters

03:15 – CloudStack confirmed failure of Hosts

03:30 – CloudStack initiated HA as VMs have now been down for over the configured 60 secs set by alert.wait

05:30 – All ten Windows VMs and System VMs were reported by CloudStack as ‘Running’

So in summary, following a failure of the XenServer Pool Masters in two Clusters, which is a worst case scenario, within 5 mins 30 secs all 10 Windows VMs, 4 Virtual Routers, and 3 System VMs were back online without any administrator input.

Recovering from Host Failures

Once the ‘failed’ pool masters are restored by simply powering them back on, they should automatically reconnect to the Pool

However if the host fails to automatically reconnect, and is not accessible via XenCenter, to re-enable them you need to disable HA on the Pool and also disable HA on the Host, which still thinks it is a Pool Master and will have gone into ‘Emergency Mode’.   To do this first disable HA on the Pool using XenCenter, then run the following command on the recovered Host

xe host-emergency-ha-disable --force

 

After a short delay the Host will reconnect to the Pool as a Slave and then come back online within CloudStack.

If a Slave Host goes down, after it is restored, before it will re-connect to the Pool, HA on the Pool ‘may’ need to be disabled, but there is no need to force disable HA on the Host as it should not go into Emergency Mode as the Pool Master will be accessible when the Host boots.

Once the failed hosts is back online, the Pool HA should be re-enabled by re-running the script which enables HA

Managing XenServers when Pool HA is enabled

Occasionally you will need to restart or Shutdown a XenServer Host which is part of a CloudStack Cluster and belongs to a Pool which has Pool HA enabled. The following sections list the correct actions to be taken:

To perform a controlled restart of a Slave Host:

1. Place into Maintenance Mode within CloudStack
2. Restart the Host
3. Exit Maintenance Mode in CloudStack

To perform a controlled restart of a Pool Master Host:

1. Place into Maintenance Mode within CloudStack
2. Disable Pool HA
3. Place the Host into Maintenance Mode with XenCenter to force the promotion of a Slave to Pool Master
4. Restart the Host
5. Once Host has fully booted and is online in XenCenter re-enable Pool HA
6. Exit Maintenance Mode in XenCenter
7. Exit Maintenance Mode in CloudStack

To shutdown a Host for an extended period of time: (Pool Masters should be demoted to a Slave)

1. Place into Maintenance Mode within CloudStack
2. Disable Pool HA
3. Place into Maintenance Mode with XenCenter
4. Shutdown the Host
5. Enable Pool HA
6. Exit Maintenance Mode in CloudStack

When you wish to bring the Host back on line

a. Disable Pool HA
b. Power on Host, then once Host has fully booted and is online in XenCenter re-enable Pool HA
c. Exit Maintenance Mode in XenCenter
d. Exit Maintenance Mode in CloudStack

Summary

After installing XS62ESP1004 you MUST enable HA on your XenServer Pools, and this results in a different approach to managing your XenServer resources.

About the Author

Geoff Higginbottom is CTO of ShapeBlue, the strategic cloud consultancy. Geoff spends most of his time designing private & public cloud infrastructures for telco’s, ISP’s and enterprises based on CloudStack.

 

ha-config:timeout=90

The Apache CloudStack community recently released CloudMonkey 5.3.0. In this post, Rohit Yadav Software Architect at ShapeBlue talks about this release and his work on the new server profile feature. For more information on CloudMonkey and its usage click here.

At ShapeBlue we offer CloudStack infrastructure support and in doing so we rely heavily on tools such as CloudMonkey which is the official Apache CloudStack command line interface and client. CloudMonkey allows us to quickly slice and dice API outputs, and to find the issue without using the CloudStack web UI.

This release aimed to become the greatest release ever with supporting unicode strings handling, server profile improvements and bugfixes, better argument processing, error handling and reporting, non-zero exit codes when an API call fails,  network and security handling and above all autocompletion of API arguments and filters in interpreter mode. I think users will love the new release as the tool becomes more mature, reliable and easy to use.

To get started, install python and pip, then install CloudMonkey:

    $ pip install cloudmonkey

If you’re an existing user, please upgrade CloudMonkey:

    $ pip install --upgrade cloudmonkey
    $ cloudmonkey set paramcompletion true

cm-install

With two new autocompletion heuristics introduced in 5.3.0, CloudMonkey automatically tries to search an API which should complete the argument and then it calls the API in the background when a user hits tab tab on an API argument or a filter. It further speeds up rendering by caching list API results from previous calls with a low cache burst timeout of 10-15 mins. This allows users to completely simply use CloudMonkey for getting UUIDs instead of using both the UI and CloudMonkey in tandem.

cm-complete

Unicode support in CloudMonkey brings maturity in the tool which would be useful for CloudStack users who want to use non-ascii characters in strings such as username, description etc. Since all the strings in CloudMonkey are now UTF-8 encoded, it’s also possible to do things like shell execution and pipe/greps using UTF-8 strings.

cm-unicode

Hope you’ll find the latest release useful and it will make you more productive!

About the author

Rohit Yadav is a Software Architect at ShapeBlue and an Apache CloudStack committer. He is the author and maintainer of CloudMonkey, and has contributed and worked on various parts of CloudStack such as API layer, authentication, database, accounts, KVM, virtual router, systemvms, maven build system. He was also behind DevCloud2, API discovery service, and  developed methods to build and export systemvms using VirtualBox.

 

In this post, Rohit Yadav, Software Architect at ShapeBlue talks about  his work on the recent implementation of SAML 2.0 based Single Sign-On (SSO) and Single Log-Out (SLO) for Apache CloudStack

As part of the ShapeBlue Software Engineering Team, I work on both CloudStack feature  requests for our customers and vendor integrations for CloudStack. However, we  do sometimes identify features that we feel are just simply missing from CloudStack.  They’re missing because no one customer/user has ever had a strong enough need to either develop them or to ask a company like ours to develop them;  these are the “nice to haves”.  So, we’ve recently been working internally here to develop some of these as part of our overall commitment to the Apache CloudStack project.

Apache CloudStack has its own cookie and key based authentication, and a special pre-shared secret key SSO mechanism but has no support for standard authentication protocols or mechanism such as SAML 2.0, OAuth2, etc. Many large organisations who may use such open and standard single sign on mechanisms cannot natively integrate the with CloudStack . This is not the first time this has come up: last year John Burwell also gave a talk on integrating SSO in CloudStack.

So, we set to work on fixing this.

SAML 2.0 is a widely used standard for exchanging authentication and authorization data between security domains. We found that a lot of CloudStack enterprise users have SAML 2.0 compliant authentication infrastructures so we started refactoring the CloudStack authentication layer to support integration of organisation specific custom authentication mechanisms which can then be implemented as CloudStack plugins.

Refactoring the authentication layer was a pretty straight forward task and it is briefly documented on the project wiki and we  then used the re-factored authentication layer to implement a SAML service provider as a plugin, the functional spec of this work can be read here

In SAML terminology, CloudStack is a Service Provider (SP) that means it’s the service that users would want to access and the organization would have their own Identity Provider (IdP) which is an authentication and authorization service backed by a user listing/store such as LDAP, AD etc. For the implementation of the plugin, we used OpenSAML which is an opensource Java library that provides building blocks to implement SAML IdP or SP. For testing the service provider implementation we used Feide OpenIdP. The flow diagram below explains the SAML 2.0 authentication workflow that uses SAML 2.0’s HTTP redirection binding.

acssaml

The work can be found in the master branch of Apache CloudStack and will be available in Apache CloudStack 4.5.0 release, in future. However, we could back-port it if we see demand.  You may read the latest administration docs for configuring and using SAML 2.0 authentication.

About the Author

Rohit Yadav is a Software Architect at ShapeBlue and an Apache CloudStack Committer. He is the author and maintainer of CloudMonkey, and has contributed and worked on various parts of CloudStack such as Auth layer and SAML plugin, API layer, authentication, database, accounts, KVM, virtual router, systemvms and the  maven build system. He was also behind DevCloud2, the API discovery service and  developed methods to build and export systemvms using VirtualBox.

Find out more about ShapeBlue’s CloudStack Software Engineering Services

The Apache CloudStack community recently released CloudMonkey 5.2.0. In this post, Rohit Yadav Software Architect at ShapeBlue talks about this release and his work on the new server profile feature. For more information on CloudMonkey and its usage click here.

At ShapeBlue we offer CloudStack infrastructure support and in doing so we rely heavily on tools such as CloudMonkey which is the official Apache CloudStack command line interface and client. CloudMonkey allows us to quickly slice and dice API outputs, and to find the issue without using the CloudStack web UI.

We use CloudMonkey to interact with different CloudStack deployments but, prior to this release,  every time we needed to interact with a new management server, we had to change the management server related options and user credentials. So we wanted to solve this problem with server profiles. With this motivation, I started a discussion thread on the CloudStack dev mailing list to work on the next CloudMonkey release and asked for any feature or improvements the users and admins would want to have in the next release, and one of the community members, Lucian, suggested this feature as well.

So, we implemented support for multiple server profiles in CloudMonkey that would allow a user to toggle between multiple server specific configurations using a single command while having different servers configurations stored in the same CloudMonkey configuration file in separate sections. The latest release of CloudMonkey 5.2.0 was shipped with this feature among other improvements and bugfixes.

Using CloudMonkey 5.2.0

To get started, install python and pip, then install CloudMonkey:

    $ pip install cloudmonkey

If you’re an existing user, please upgrade CloudMonkey:

    $ pip install --upgrade cloudmonkey

Now, launch CloudMonkey and you’ll see something like the following:

cloudmonkey-start

If no “profile” is found in the config, CloudMonkey creates and uses a default profile by the name “local” with default values. To create a server profile, you run a “set profile ” command which creates a new section profile section in the config file with the given name and sets default values for other options such as url, username, password etc. If a profile already exists with provided name, it’s simply loaded from the configuration file. Any “set” command for options such as url, username, password, apikey, secretkey etc. will set these options only on the current server profile. The following screenshot illustrates the default server profile “[local]” in the CloudMonkey config:

cloudmonkey-server-profile

 Hope you’ll find the latest release of CloudMonkey and this feature useful!

About the author

Rohit Yadav is a Software Architect at ShapeBlue and an Apache CloudStack committer. He is the author and maintainer of CloudMonkey, and has contributed and worked on various parts of CloudStack such as API layer, authentication, database, accounts, KVM, virtual router, systemvms, maven build system. He was also behind DevCloud2, API discovery service, and  developed methods to build and export systemvms using VirtualBox.

 

The CloudStack API is a very powerful tool which once mastered provides Administrators access to all of CloudStack’s features, many of which are not available via the GUI.

During a number of recent Projects the ShapeBlue consulting team needed to leverage these features in order to deliver the advanced solutions our clients required.

This article will demonstrate completing the following tasks using only the API:

  • Creating a new Network Offering with No DHCP and DNS Services
  • Activating the Network Offering
  • Creating an Isolated Network based on the new Network Offering
  • Creating a Guest Instance with a pre-determined IP Address

For the sake of this blog article, a greatly simplified example infrastructure is used, however the concepts covered here have been used with large complex Clouds.

Using the API for Advanced Network Management

The simple example infrastructure consists of the following:

  • CloudStack Server on IP 192.168.1.1 with API Port 8096 enabled
  • Default Guest CIDR of 10.1.1.1/24
  • A physical infrastructure separate to CloudStack with a CIDR of 192.168.250.0/24 using VLAN 102
  • Active Directory, DHCP and DNS Servers
  • A Client Specific Physical Firewall on IP 192.168.250.1

Advanced Networking Use Case Example

Imagine a Hosting Provider who is addition to offering Cloud Services based on CloudStack, has clients who also have dedicated servers running Active Directory, DNS, DHCP etc and a dedicated Hardware Firewall in their Data Centre. The client now wants to start using Guest Instances hosted on the CloudStack platform, but connected to the existing AD Domain on the dedicated physical infrastructure. Due to the integration with Active Directory the default Guest IP Address range and DNS Servers offered by the CloudStack Virtual Router will not be suitable as they are common to the whole Zone.

In order to create a CloudStack Network which utilises the required IP Address Range, and provides access to the DNS Servers within the Active Directory Domain, a new Isolated Network is required. Once created, this Isolated Network will be linked to the Physical Architecture via a VLAN Trunk.

The default CloudStack Network Offerings are not suitable for this as they all provide DNS and DHCP Services, so a new Network Service Offering will need to be created.

Using the API

Only some of the required steps can be completed via the GUI so the only way to configure and implement this architecture is by using the API.

For testing and evaluation, the API commands can be submitted manually by simply pasting them into a browsers Address Bar. A live deployment which requires this type of advanced management should create a set of custom management tools to streamline and automate these processes.

API commands are normally submitted with user specific security credentials, however Administrators with direct access to the CloudStack Server can use the API Port to simplify the commands.

Prior to the release of V3.0.1 this API Port was enabled by default and used port 8096. If you have upgraded from V3.0 it will still be enabled, clean installs of V3.0.1 and later will need to enable this in Global Settings. If enabled you should change the default setting to increase security.

Creating a Custom Network Offering

We need a new Network Offering with no DHCP or DNS so we can use the physical DHCP and DNS Servers, the base command for this is:

http://192.168.1.1:8096/client/api?command=createNetworkOffering

We then append the various options such as Name, Description etc

&name=Name

To use spaces in text in API commands, we need to use %20, so for the description we would use

&displaytext=Offering%20Description

As we want to use this Network Offering with a single account, we need to create an Isolated Network Offering

&guestiptype=Isolated

We do not really need any Supported Services so we will use the bare minimum of just UserData, but as it is simply a coma separated list multiple services can be added if required

&supportedservices=UserData

The Traffic Type in V3 of CloudStack must be Guest

&traffictype=guest

We want to change the default 200Mb Network speed to 1Gb

&networkrate=1000

We want to connect this network to a specific VLAN, so we need to be able to specify the VLAN ID when we create the Network using this Network Offering

&specifyvlan=true

We have to specify IP Ranges when creating an Isolated network, even though we will be using an external DHCP Server

&specifyipranges=true

The full command will therefore look like this:

http://192.168.1.1:8096/client/api?command=createNetworkOffering&name=Name&displaytext=Offering%20Description&guestiptype=Isolated&supportedservices=UserData&traffictype=guest&networkrate=1000&specifyvlan=true&specifyipranges=true

Once submitted, this will create a new Network Offering and provide an XML response in the browser with the results including the ID of the new offering, we must capture this ID for use in later steps.

Activating the newly created Network Offering

New Network Offerings created with the API are initially disabled, but using the ID recorded from the previous step we can activate it using the following command:

http://192.168.1.1:8096/client/api?command=updateNetworkOffering&id=XXXXX&state=Enabled

Create an Isolated Network

Only a ROOT level Administrator can create new Networks which use a specific VLAN ID. The command will need a number of IDs which will need to be obtained first using the following commands:

http://192.168.1.1:8096/client/api?command=listZones to obtain the Zone ID

http://192.168.1.1:8096/client/api?command=listDomainChildren to obtain the Domain ID

The base command to then create the new Network is:

http://192.168.1.1:8096/client/api?command=createNetwork

The options required for this example are:

&name= Network%20Name

&displaytext=Network%20Description

The Network Offering ID was obtained when we created the new Network Offering

&networkofferingid=XXX

&zoneid=XXX

An IP Range, Netmask & Gateway has to be specified

&startip=192.168.250.100

&endip=192.168.250.150

&gateway=192.168.250.1

&netmask=255.255.255.0

The VLAN of the existing physical architecture to enable Guest Traffic to be exchanged

&vlan=102

The ID of the Domain the Account belongs to

&domainid=XXX

The Name of the Account, and not the ID is used here, highlighting the importance that the method used for generating Account Names, creates truly unique ones.

&account=XXX

We want to restrict access to this Network to a specific Account

&acltype=Account

The final command will therefore look something like this:

http://192.168.1.1:8096/client/api?command=createNetwork&name=Network%20Name&displaytext=Network%20Description&networkofferingid=XXX&zoneid=XXX&startip=192.168.250.100&endip=192.168.250.150&gateway=192.168.250.1&netmask=255.255.255.0&vlan=102&domainid=XXX&account=XXX&acltype=Account

Deploy Guest Instance

The only way to deploy a new Guest Instance with a pre-determined IP Address is to do so via the API. As we will be connecting this Instance to an existing Active Directory Domain there will probably be an IP Schema already in place.

This IP address will not actually be assigned by CloudStack as we have created a Network Service Offering without DHCP, but as there will be an IP Address displayed in the GUI, we should allocate it to prevent confusion when using the CloudStack GUI. The IP Address will actually be assigned by the Physical DHCP Server, or configured manually within the Guest Instance.

Before we can deploy a new Guest Instance we need to obtain the IDs of the chosen Service Offering and Template

http://192.168.1.1:8096/client/api?command=listServiceOfferings

http://192.168.1.1:8096/client/api?command=listTemplates&templatefilter=featured

The base command to then create the Guest Instance is:

http://192.168.1.1:8096/client/api?command=deployVirtualMachine

The options required for this example are:

The Service Offering and Template IDs obtained by the previous commands

&serviceofferingid=XXX

&templateid=XXX

Zone, Domain and Account information used earlier

&zoneid=XXX

&domainid=XXX

&account=XXX

Unique Instance Display Name

&displayname=Instance%20Display%20Name

&ipaddress=192.168.250.110

Multiple Network IDs can be assigned, hence the ‘Pluralisation’ of this option

&networkids=XXX

The complete command will look like:

http://192.168.1.1:8096/client/api?command=deployVirtualMachine&serviceofferingid=XXX&templateid=XXX&displayname=Instance%20Display%20Name&ipaddress=192.168.250.110&networkids=XXX

The DeployVirtualMachine command is Asynchronous, to the feedback after issuing the command consists of only two IDs. To check the status of Asynchronous commands, take the Job ID from the results and use with the following command

command=queryAsyncJobResult&jobId=XXX

Once the Asynchronous command has completed, the results returned by the queryAsyncJohbResult command will be updated and provide a very detailed output.

Summary

This article has demonstrated some of the advanced features available to CloudStack Administrators, and introduced management using the API rather than the GUI. By employing these techniques, complex Cloud architectures can be created and managed.

To simplify this type of management, scripts written using Ajax should be used as these can interact with the XML responses issued by the CloudStack API allowing advanced workflows to be created.

About the Author

Geoff Higginbottom is CTO of ShapeBlue, the strategic cloud consultancy. Geoff spends most of his time designing private & public cloud infrastructures for telco’s, ISP’s and enterprises based on CloudStack.