IoTs have gained interest over recent times. In this article, Rohit Yadav, Principal Engineer at ShapeBlue, explores and shares his personal experience of setting up an Apache CloudStack based IaaS cloud on Raspberry Pi4, a popular single-board ARM64 IoT computer that can run GNU/Linux kernel with KVM. The article presents the use case of Apache CloudStack on RaspberryPi4 with Ubuntu 20.04 and KVM.

CloudStack support for ARM64/RaspberryPi4 is available from version 4.13.1.0+. This guide uses a custom CloudStack 4.16 repository that was created and tested specifically against the new RaspberryPi4 and Ubuntu 20.04 arm64 to set up an IaaS cloud computing platform.

Getting Started

By default, KVM is not enabled in the arm64 pre-built images. I found this during my research where I built Linux kernel with KVM enabled on the ARM64 RPi4 board and shared my findings with the Ubuntu kernel team and influence the build team to have KVM enabled by default. With the changes, now Ubuntu 19.10+ ARM64 builds have KVM enabled.

To get started you need the following:

• RPi4 board (cortex-72 quad-core 1.5Ghz processor) 4GB/8GB RAM model
• Ubuntu 20.04.2 arm64 image installed on a Samsung EVO+ 128GB micro sd card (any 4GB+ class 10 u3/v30 sdcard will do).
• (Optional) An external USB-based SSD storage with high iops for storage

Flash the image to your microSD card:

$ xzcat ubuntu-20.04.2-preinstalled-server-arm64+raspi.img.xz | sudo dd bs=4M of=/dev/mmcblk0
0+381791 records in
0+381791 records out
3259499520 bytes (3.3 GB, 3.0 GiB) copied, 131.749 s, 24.7 MB/s

Eject and insert the microSD card again to initiate volume mounts, then create an empty /boot/ssh file to enable headless ssh:

# find the mount point
mount -l | grep /dev/mmcblk0
# cd to the writable mount point, for example:
cd /media/rohit/writable
# create an empty ssh file
sudo touch boot/ssh

Next, check and ensure that 64-bit mode is enabled:

cd /media/rohit/system-boot

# Edit config.txt to have this:
[all]
arm_64bit=1
device_tree_address=0x03000000
dtoverlay=vc4-fkms-v3d
enable_gic=1

# Save file and unmount to safely eject the microSD card
sync
sudo umount /dev/mmcblk0p1
sudo umount /dev/mmcblk0p2

Next, eject and insert the microSD card in your Raspberry Pi4 and power on. Find the device via your router dhcp clients list and ssh into it using username ubuntu and password ubuntu.

ssh ubuntu@<ip>

Allow the root user for ssh access using password, fix /etc/ssh/sshd_config and set PermitRootLogin yes and restart ssh using systemctl restart ssh. Change and remember the root password:

passwd root

Next, install basic packages and setup time as the root user:

apt-get update
apt-get install ntpdate openssh-server sudo vim htop tar iotop
ntpdate time.nist.gov # update time
hostnamectl set-hostname cloudstack-mgmt

Ensure that KVM is available at /dev/kvm or by running kvm-ok:

# apt install cpu-checker

# kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

Disable automatic upgrades and unnecessary packages:

apt-get remove --purge unattended-upgrades snapd cloud-init
# Edit the files at /etc/apt/apt.conf.d/* with following
APT::Periodic::Update-Package-Lists "0";
APT::Periodic::Unattended-Upgrade "1";

Tip: In case you suspect IO load, to reduce load on RaspberryPi4 micrSD card change the fs commit duration, for example in /etc/fstab: (however this adds risk of potential data loss)

LABEL=writable  /        ext4   defaults,commit=60      0 0
LABEL=system-boot       /boot/firmware  vfat    defaults        0       1

Network Setup

Next, setup host networking using Linux bridges that can handle CloudStack’s public, guest, management and storage traffic. For simplicity, a single bridge cloudbr0 to be used for all traffic types on the same physical network. Install bridge utilities:

apt-get install bridge-utils

Note: This part assumes that you’re in a 192.168.1.0/24 home network which is a typical RFC1918 private network.

Admins can now use netplan to configure networking with Ubuntu 20.04. The default installation creates a file at /etc/netplan/50-cloud-init.yaml which you can comment, and create a file at /etc/netplan/01-netcfg.yaml applying your network specific changes:

 network:
   version: 2
   renderer: networkd
   ethernets:
     eth0:
       dhcp4: false
       dhcp6: false
       optional: true
   bridges:
     cloudbr0:
       addresses: [192.168.1.10/24]
       gateway4: 192.168.1.1
       nameservers:
         addresses: [8.8.8.8]
       interfaces: [eth0]
       dhcp4: false
       dhcp6: false
       parameters:
         stp: false
         forward-delay: 0

Tip: If you want to use VXLAN based traffic isolation, make sure to increase the MTU setting of the physical nics by 50 bytes (because VXLAN header size is 50 bytes). For example:

  ethernets:
    enp2s0:
      match:
        macaddress: 00:01:2e:4f:f7:d0
      mtu: 1550
      dhcp4: false
      dhcp6: false
    enp3s0:
      mtu: 1550

Save the file and apply network config, finally reboot:

netplan generate
netplan apply
reboot

Management Server Setup

Install MySQL server: (run as root)

apt-get install mysql-server

Make a note of the MySQL server’s root user password. Configure InnoDB settings in /etc/mysql/mysql.conf.d/mysqld.cnf:

[mysqld]

server_id = 1
sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
innodb_rollback_on_timeout=1
innodb_lock_wait_timeout=600
max_connections=1000
log-bin=mysql-bin
binlog-format = 'ROW'

default-authentication-plugin=mysql_native_password

Restart database:

systemctl restart mysql

Installing management server may give dependency errors, so download and manually install few packages as follows:

apt-get install python2
wget http://mirrors.kernel.org/ubuntu/pool/universe/m/mysql-connector-python/python-mysql.connector_2.1.6-1_all.deb
dpkg -i python-mysql.connector_2.1.6-1_all.deb

# Install management server
echo deb [trusted=yes] https://download.cloudstack.org/rpi4/4.16 / > /etc/apt/sources.list.d/cloudstack.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 484248210EE3D884
apt-get update
apt-get install cloudstack-management cloudstack-usage

# Stop the automatic start after install
systemctl stop cloudstack-management cloudstack-usage

Setup database:

cloudstack-setup-databases cloud:cloud@localhost --deploy-as=root: -i 192.168.1.10

Storage Setup

In my setup, I’m using an external USB SSD for storage. I plug in the USB SSD storage, with the partition formatted as ext4, here’s the /etc/fstab:

LABEL=writable  /        ext4   defaults        0 0
LABEL=system-boot       /boot/firmware  vfat    defaults        0       1
UUID="91175b3a-ee2c-47a7-a1e5-f4528e127523" /export ext4 defaults 0 0

Then mount the storage as:

mkdir -p /export
mount -a

Install NFS server:

apt-get install nfs-kernel-server quota

Create exports:

echo "/export  *(rw,async,no_root_squash,no_subtree_check)" > /etc/exports
mkdir -p /export/primary /export/secondary
exportfs -a

Configure and restart the NFS server:

sed -i -e 's/^RPCMOUNTDOPTS="--manage-gids"$/RPCMOUNTDOPTS="-p 892 --manage-gids"/g' /etc/default/nfs-kernel-server
sed -i -e 's/^STATDOPTS=$/STATDOPTS="--port 662 --outgoing-port 2020"/g' /etc/default/nfs-common
echo "NEED_STATD=yes" >> /etc/default/nfs-common
sed -i -e 's/^RPCRQUOTADOPTS=$/RPCRQUOTADOPTS="-p 875"/g' /etc/default/quota
service nfs-kernel-server restart

Seed systemvm template from the management server:

wget https://download.cloudstack.org/rpi4/systemvmtemplate/4.16/systemvmtemplate-4.16.0-kvm-arm64.qcow2
/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt \
          -m /export/secondary -f systemvmtemplate-4.16.0-kvm-arm64.qcow2 -h kvm \
          -o localhost -r cloud -d cloud

KVM Host Setup

Install KVM and CloudStack agent, configure libvirt:

echo deb [trusted=yes] https://download.cloudstack.org/rpi4/4.16 / > /etc/apt/sources.list.d/cloudstack.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 484248210EE3D884
apt-get update
apt-get install qemu-kvm cloudstack-agent
systemctl stop cloudstack-agent

Enable VNC for console proxy:

sed -i -e 's/\#vnc_listen.*$/vnc_listen = "0.0.0.0"/g' /etc/libvirt/qemu.conf

Fix security driver issue:

echo 'security_driver = "none"' >> /etc/libvirt/qemu.conf

Enable libvirtd in listen mode:

sed -i -e 's/.*libvirtd_opts.*/libvirtd_opts="-l"/' /etc/default/libvirtd

Configure default libvirtd config:

echo 'listen_tls=0' >> /etc/libvirt/libvirtd.conf
echo 'listen_tcp=1' >> /etc/libvirt/libvirtd.conf
echo 'tcp_port = "16509"' >> /etc/libvirt/libvirtd.conf
echo 'mdns_adv = 0' >> /etc/libvirt/libvirtd.conf
echo 'auth_tcp = "none"' >> /etc/libvirt/libvirtd.conf

The traditional socket/listen based configuration may not be supported, we can get the old behaviour as follows:

systemctl mask libvirtd.socket libvirtd-ro.socket libvirtd-admin.socket libvirtd-tls.socket libvirtd-tcp.socket
systemctl restart libvirtd

Ensure the following options in the /etc/cloudstack/agent/agent.properties:

guest.cpu.arch=aarch64
guest.cpu.mode=host-passthrough
host.reserved.mem.mb=640

Configure Firewall

Configure firewall:

# configure iptables
NETWORK=192.168.1.0/24
iptables -A INPUT -s $NETWORK -m state --state NEW -p udp --dport 111 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 111 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 2049 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 32803 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p udp --dport 32769 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 892 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 875 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 662 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 8250 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 8080 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 9090 -j ACCEPT
iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 16514 -j ACCEPT

apt-get install iptables-persistent

# Disable apparmour on libvirtd
ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/
ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/
apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd
apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper

Launch Cloud

Start your cloud:

cloudstack-setup-management
systemctl status cloudstack-management
tail -f /var/log/cloudstack/management/management-server.log

After management server is UP, proceed to http://192.168.1.10:8080/client/primate (change the IP suitably) and log in using the default credentials – username admin and password password.

NOTE: I found some DB issue while deploying a zone, please check/run the following until the bug is fixed upstream, using MySQL client:

use cloud;
ALTER TABLE nics MODIFY COLUMN update_time timestamp NULL;

Example Setup

The following is an example of how you can set up an advanced zone in the 192.168.1.0/24 network.

Setup Zone

Go to Infrastructure > Zone and click on add zone button, select advanced zone and provide the following configuration:

Name - any name
Public DNS 1 - 8.8.8.8
Internal DNS1 - 192.168.1.1
Hypervisor - KVM

Setup Network

Use the default, which is VLAN isolation method on a single physical nic (on the host) that will carry all traffic types (management, public, guest etc).

Note: If you’ve iproute2 installed and host’s physical NIC MTUs configured, you can use VXLAN as well.

Public traffic configuration:

Gateway - 192.168.1.1
Netmask - 255.255.255.0
VLAN/VNI - (leave blank for vlan://untagged or in case of VXLAN use vxlan://untagged)
Start IP - 192.168.1.20
End IP - 192.168.1.50

Pod Configuration:

Name - any name
Gateway - 192.168.1.1
Start/end reserved system IPs - 192.168.1.51 - 192.168.1.80

Guest traffic:

VLAN/VNI range: 500-800

Add Resources

Create a cluster with the following:

Name - any name
Hypervisor - Choose KVM

Add your default/first host:

Hostname - 192.168.1.10
Username - root
Password - <password for root user, please enable root user ssh-access by password on the KVM host>

Note: root user ssh-access is disabled by default, please enable it.

Add primary storage:

Name - any name
Scope - zone-wide
Protocol - NFS
Server - 192.168.1.10
Path - /export/primary

Add secondary storage:

Provider - NFS
Name - any name
Server - 192.168.1.10
Path - /export/secondary

Next, click Launch Zone which will perform the following actions:

Create Zone
Create Physical networks:
  - Add various traffic types to the physical network
  - Update and enable the physical network
  - Configure, enable and update various network provider and elements such as the virtual network element
Create Pod
Configure public traffic
Configure guest traffic (vlan range for physical network)
Create Cluster
Add host
Create primary storage (also mounts it on the KVM host)
Create secondary storage
Complete zone creation

Finally, confirm and enable the zone. Wait for the system VMs to come up, then you can proceed with your IaaS usage.

You can build your own arm64 guest templates or deploy VMs using these guest templates at: https://download.cloudstack.org/rpi4/templates

About the Author

Rohit Yadav is a Principal Engineer at ShapeBlue. He has been involved with Cloudstack since 2012 when he was first made a committer in the Apache CloudStack project and has made more code commits to Cloudstack than any other developer in its history. He has been a PMC member of the project since 2015. Rohit is the author & maintainer of the Cloudstack CloudMonkey project and has been instrumental in the development of many of CloudStack’s flagship features.

Rohit spends most of his time designing and implementing features in Apache CloudStack. In his free time, he has a passion for renewable energy technology and is always willing to help people passionate about open-source technologies.

Read more articles from Rohit on his personal blog.

While, in my opinion VMware’s vSphere is the best performing and most stable hypervisor available, vCenter obstinately remains a single point of failure when using vSphere and it’s no different when leveraging vCenter in a CloudStack environment.  Therefore, very occasionally there is a requirement to rebuild a vCentre server which was previously running in your CloudStack environment.  Working with one of our clients we found out (the hard way) how to do this.

Replacing a Pre-Existing vCenter Instance

Recreating a vCenter Instance

The recovery steps below apply to both the Windows-based vCenter server or an appliance.

The first step, which I won’t cover in detail here, is to build a new vCenter server and attach the hosts to it.  Follow VMware standard procedure when building your new vCenter server.  To save some additional steps later on, reuse the host name and IP address of the previous vCenter server. Ensure that the permissions have been reapplied allowing CloudStack to connect at the vCentre level with full administrative privileges and that datacenter and cluster names have been recreated accurately.  When you re-add the hosts, the VMs running on those hosts will automatically be pulled into the vCentre inventory.

Networking

The environment that we were working on was using standard vSwitches; therefore the configuration of those switches was held on each of the hosts independently and pulled into the vCenter inventory when the hosts were re-added to vCenter.  Distributed vSwitches (dvSwitches) have their configuration held in the vCenter.  We did not have to deal with this complication and so the recreation of dvSwitches is not covered here.

Roll-back

The changes that are to be made can easily be undone, as they make no change and trigger no change in the physical environment.  However, it is never a bad idea to have a backup, you shouldn’t ever need to roll back the whole database, but it’s good to have a record of your before/after states.

Update vCenter Password in CloudStack

If you have changed the vCenter password, it’ll need updating in the CloudStack database in its encrypted form.  The following instructions are taken from the CloudStack documentation:

• Generate the encrypted equivalent of your vCenter password:

$ java -classpath /usr/share/cloudstack-common/lib/jasypt-1.9.0.jar org.jasypt.intf.cli.JasyptPBEStringEncryptionCLI encrypt.sh input="_your_vCenter_password_" password="`cat /etc/cloudstack/management/key`" verbose=false

• Store the output from this step, we need to add this in cluster_details table and vmware_data_center tables in place of the plain text password
• Find the ID of the row of cluster_details table that you have to update:

$ mysql -u <USERNAME> -p<PASSWORD>
select * from cloud.cluster_details;

• Update the password with the new encrypted one

update cloud.cluster_details set value = '_ciphertext_from_step_1_' where id = _id_from_step_2_;

• Confirm that the table is updated:

select * from cloud.cluster_details;

• Find the ID of the correct row of vmware_data_center that you want to update

select * from cloud.vmware_data_center;

• update the plain text password with the encrypted one:

update cloud.vmware_data_center set password = '_ciphertext_from_step_1_' where id = _id_from_step_5_;

• Confirm that the table is updated:

select * from cloud.vmware_data_center;

Reallocating the VMware Datacenters to CloudStack

The next step is to recreate the cloud.zone property within vSphere which is used to track VMware datacenters which have been connected to CloudStack.   Adding a VMware datacenter to CloudStack creates this custom attribute and sets it to “true”.  CloudStack will check for this attribute and value and show an error on the management log if it attempts to connect to a vCenter server which does not have this attribute set.  While the property can be edited using the full vSphere client, it must be created using PowerCLI utility.

Using the PowerCLI utility do the following:

Connect-VIServer <VCENTER_SERVER_IP>

(you’ll be prompted for a username and password with sufficient permissions for the vCenter Server)

New-CustomAttribute -Name "cloud.zone" -TargetType Datacenter

Get-Datacenter <DATACENTER_NAME> | Set-Annotation -CustomAttribute "cloud.zone" -Value true

Reconnecting the ESXi Hosts to CloudStack

CloudStack will now be connected to the vCenter server, but the hosts will still probably all be marked as down.  This is because vCenter uses an internal identifier for hosts, so that names and IP addresses can change, but it still can keep track of which host is which.

This identifier appears in two places in CloudStack’s database; The ‘guid’ in the cloud.host table and also in the cloud.host_details table against the ‘guid’ value. The host table can be queried as follows:

SELECT id,name,status,guid FROM cloud.host WHERE hypervisor_type = 'VMware' AND status != 'Removed';

The guid takes the form:

HostSystem<HOST_ID>@<VCENTER_NAME>

You can find the new Host ID either though the PowerCLI again, or via a utility like Robware’s RVTools.  RVTools is a very light-weight way of viewing information from a vCenter. It also allows export to csv so is very handy for manual reporting and analysis. The screen grap below shows a view called vHosts; note the Object ID.  This is vCenter’s internal ID of the host.

Through PowerCLI you would use:

Connect-VIServer <VCENTER_SERVER_IP>;

Get-VMHost | ft -Property Name,Id -autosize

Now you can update the CloudStack tables with the new host ids for each of your hosts.

cloud.host.guid for  host ‘10.0.1.32’ on vCenter ‘10.0.1.11’ might previously have read:

HostSystem:host-99@10.0.1.11

should be updated to:

HostSystem:host-10@10.0.1.11

Remember to check the host and host_details table.

Once that you have updated the entries for all of the active hosts in the zone, you can start the management service again now and all of the hosts should reconnect, as their pointers have been fixed in the CloudStack database.

Summary

This article explains how to re-insert a lost or failed vCenter server into CloudStack.  This blog describes the use of PowerCLI to both set and retrieve values from vCenter.

This process could be used to migrate an appliance based vCenter to a Windows based environment in order to support additional capacity as well as replacing a broken vCenter installation.

About The Author

Paul Angus is VP Technology & Cloud Architect at ShapeBlue, The Cloud Specialists. He has designed and implemented numerous CloudStack environments for customers across 4 continents, based on Apache Cloudstack, Citrix Cloudplatform and Citrix CloudPortal.

When not building Clouds, Paul likes to create Ansible playbooks that build clouds

Recently we’ve seen a few clients being tripped up by the System VM upgrades during the CloudStack upgrades in multi-zone deployments.

The issue occurs when the System VM is pre-deployed individually to multiple zones, rather than being a single template deployed to multiple zones. This may be done in error or because the specific version of CloudStack/CloudPlatform which you are upgrading from does not allow you to provision the template as zone-wide.

The result is that only the system VMs in one zone recreate properly after the upgrade and the other zones do not recreate the SSVM or CPVM when they are destroyed.

The vm_template table will have entries like this for a given System VM template:

 SELECT id,name,type,cross_zones,state FROM cloud.vm_template WHERE name like '%systemvm-xenserver%' AND removed IS NULL; 

 SELECT template_store_ref.id,template_store_ref.store_id,template_store_ref.template_id,image_store.data_center_id,image_store.name FROM template_store_ref JOIN image_store ON template_store_ref.store_id=image_store.id WHERE template_store_ref.destroyed=0; 

The above example assumes that there are 3 zones, with 1 secondary storage pool in each zone, if there are multiple secondary storage pools, then only one in each zone will have initially received the template.

You could edit the path in the template_store_ref to point to the location where the template has been downloaded to on the secondary storage, however, this will leave the system in a non-standard state.

The safest way forward is to set one of the templates to cross_zones=1 and then coax CloudStack into creating an SSVM in each zone (one at a time), which will then download that cross-zone template.

Remediation

In this example I’ll use template 634 to be the ultimate region-wide template.

In order to download the cross_zone template we need to make one template the region-wide template at a time. So assuming that zone relating to secondary storage pool 3 upgraded properly, we first make template 635 the region-wide template. By changing the database to this:

UPDATE cloud.vm_template SET state='Inactive', cross_zones=1 WHERE id=634; UPDATE cloud.vm_template SET state='Active' WHERE id=635; UPDATE cloud.vm_template SET state='Inactive' WHERE id=636;

UPDATE cloud.vm_template SET state='Active' WHERE id=634; UPDATE cloud.vm_template SET state='Inactive' WHERE id=635;

The SSVM in zones 2 and 3 will need their cloud service restarting in order to prompt the re-scanning of the templates table. This can be done by a restart or stopping then starting the cloud service on the SSVM – DO NOT DESTROY the SSVM in zones 2 or 3 as they don’t have the correct template yet. The SSVMs in zones 2 and 3 will now start downloading template id 634 to their secondary storage pools.

Once the template 634 has downloaded to zones 2 and 3 (it was already in zone 1), CloudStack will be able to recreate system VMs in any zones.

Summary

This article explains how to recover from the upgrade failure scenario when system VMs will not recreate in zones in a multi-zone deployment.

About The Author

Paul Angus is VP Technology & Cloud Architect at ShapeBlue, The Cloud Specialists. He has designed and implemented numerous CloudStack environments for customers across 4 continents, based on Apache Cloudstack ,Citrix Cloudplatform and Citrix Cloudportal.

When not building Clouds, Paul likes to create Ansible playbooks that build clouds

Introduction

If you are new to Apache CloudStack and want to learn the concepts but do not have all the equipment required to stand-up a test environment, why not use your existing PC and VirtualBox.

VirtualBox is a cross platform virtualisation application which runs on OSX, Windows, Linux and Solaris, meaning no matter what OS you are running, you should be able to run VirtualBox.

The aim of this exercise is to build an Apache CloudStack environment which is as close to a Production deployment as possible, within the obvious constraints of running it all on a laptop. This deployment will support the following key functions of Apache CloudStack:

Production Grade Hypervisor: Citrix XenServer 6.2 with full VLAN support
Apache CloudStack on CentOS 6.5
NFS for Primary and Secondary Storage – each on a dedicated VLAN
Console Proxy and Secondary Storage VM
All Advanced Networking features such as Firewall, NAT, Port Forwarding, Load Balancing, VPC

To achieve all of this we need to deploy two VMs on VirtualBox, a CentOS VM for Apache Cloudstack, and a Citrix XenServer VM for our Hypervisor. The CloudStack VM will also act as our MySQL Server and NFS Server.

A key requirement of this test environment is to keep it completely self-contained so it can be used for training (insert link to Bootcamp) and demos etc.  To achieve this, and maintain the ability to deploy a new Zone and download the example CentOS Template to use in the system, we simulate the CloudStack Public Network and host the Default CentOS Template on the CloudStack Management Server VM using NGINX.

VirtualBox Configuration

Download and install the appropriate version from https://www.virtualbox.org/wiki/Downloads

Once VirtualBox is installed we need to configure it ready for this environment. The defaults are used where possible, but if you have been using VirtualBox already, you may have different settings which need to be adjusted.

We will be using three ‘Host Only’ networks, one ‘Nat’ network, and an ‘Internal’ network. By default VirtualBox has only one ‘Host Only’ network so we need to create two more.

1. From the ‘file’ menu (windows) or VirtualBox menu (OSX), select ‘Preferences’ then ‘Network’ then ‘Host-only Networks’
2. Add two more networks so you have at least 3 which we can use
3. Setup the IP Schema for the 1st two networks as follows:

The naming conventions for Host Only Networks differs depending on the Host OS, I will simply refer to these as

‘Host Only Network 1’, 2 and 3 etc so please refer to the following comparison matrix to identify the correct Network.

Host Only Network 1:

IPv4 Address: 192.168.56.1
IPv4 Network Mask: 255.255.255.0

DHCP Server is optional as we don’t use it, but ensure the range does not clash with the static IPs we will be using which are 192.168.56.11 & 192.168.56.101

Host Only Network 2:

IPv4 Address: 172.30.0.1
IPv4 Network Mask: 255.255.255.0

By setting up these IP ranges, we ensure our host laptop has an IP on these Networks so we can access the VMs connected to them. We don’t need an IP on ‘Host Only Network 3’ as this will be used for storage and will also be running VLANs.

We use a NAT Network so that we can connect the CloudStack Management VM to the internet to enable the installation of the various packages we will be using.

Configure the VirtualBox ‘NatNetwork’ to use the following settings:

Network Name: NatNetwork
Network CIDR: 10.0.2.0/24

We disable DHCP as we cannot control the range to exclude our statically assigned IPs on our VMs.

Whilst this article focuses on creating a single CloudStack Management Server, you can easily add a second, and I have found that the DHCP allocated IPs from the NAT Network can change randomly, so setting up NAT Rules can be problematic, hence I always use statically assigned IPs.

The ‘Internal’ Network requires no configuration.

CloudStack VM

Create a VM for CloudStack Manager using the following Settings:

Name: CSMAN 4.4.1
Type: Linux
Version: Red Hat (64 bit)
RAM: 2048 (you cannot go lower than this for initial setup)
Hard Drive: VDI – Dynamic – 64 GB (we allocate this much as it will act as NFS Storage)

Note: VirtualBox seems to mix up the networks if you add them all at the same time so we add the 1st Network and install CentOS, then once fully installed, we add the additional networks, rebooting in-between. This appears to be a bug in the latest versions of VirtualBox (4.3.18 at the time of writing)

Modify the settings and assign ONLY the 1st network Adapter correct networks as follows:

Install CentOS 6.5 64-bit minimal, set the Hostname to CSMAN, and IP address to 192.168.56.11/24 with a gateway of 192.168.56.1, and ensure the network is set to start on boot. Set DNS to public servers such as 8.8.8.8 & 8.8.4.4

Once the install is completed reboot the VM and confirm eth0 is active, then shutdown the VM and add the 2nd Network Adapter

Boot the VM so it detects the NIC, then shut down and add the 3rd Adapter

Boot the VM so it detects the NIC, then shut down and add the 4th Adapter

Finally, boot the VM so it detects the last adapter and then we can configure the various interfaces with the correct IP schemas.

ifcfg-eth0

DEVICE=eth0
TYPE=Ethernet
IPADDR=192.168.56.11
PREFIX=24
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME=MGMT

ifcfg-eth1

DEVICE=eth1
TYPE=Ethernet
IPADDR=10.0.2.11
GATEWAY=10.0.2.1
PREFIX=24
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
DEFROUTE=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME=NAT

ifcfg-eth2

DEVICE=eth2
TYPE=Ethernet
IPADDR=172.30.0.11
PREFIX=24
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME=PUBLIC 

ifcfg-eth3

DEVICE=eth3
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MTU=9000
VLAN=yes
USERCTL=no
MTU=9000

ifcfg-eth3.100

DEVICE=eth3.100
TYPE=Ethernet
IPADDR=10.10.100.11
PREFIX=24
ONBOOT=yes
BOOTPROTO=none
NAME=PRI-STOR
VLAN=yes
USERCTL=no
MTU=9000 

ifcfg-eth3.101

DEVICE=eth3.101
TYPE=Ethernet
IPADDR=10.10.101.11
PREFIX=24
ONBOOT=yes
BOOTPROTO=none
NAME=SEC-STOR
VLAN=yes
USERCTL=no
MTU=9000

Restart networking to apply the new settings, then apply all the latest updates

service networking restart
yum update -y

You can now connect via SSH using Putty to continue the rest of the configuration so you can copy and paste commands and settings etc

Installation and Configuration

With the base VM built we now need to install Apache CloudStack and all the other services this VM will be hosting. First we need to ensure the VM has the correct configuration.

Selinux

Selinux needs to be set to ‘permissive’, we can achieve this by running the following commands:

setenforce permissive
sed -i "/SELINUX=enforcing/ c\SELINUX=permissive" /etc/selinux/config

Hostname

The CloudStack Management Server should return its FQDN when you run hostname –fqdn, but as we do not have a working DNS installation it will probably return ‘unknown-host’ To resolve this we simply add an entry into the Hosts file, and while we are there, we may as well add one for the xenserver as well. Update /etc/hosts with the following, then reboot for it to take effect.

127.0.0.1 localhost localhost.cstack.local
192.168.56.11 csman.cstack.local csman
192.168.56.101 xenserver.cstack.local xenserver

Speed up SSH Connections

As you will want to use SSH to connect to the CloudStack VM its worth turning off the DNS Check to speed up the connection. Run the following commands

sed -i "/#UseDNS yes/ c\UseDNS no" /etc/ssh/sshd_config
service sshd restart

NTP

It’s always a good idea to install NTP so let’s add it now, and set it to start on boot (you can always configure this VM to act as the NTP Server for the XenServer, but that’s out of scope for this article)

yum install -y ntp
chkconfig ntpd on
service ntpd start

CloudStack Repo

Setup the CloudStack repo by running the following command:

echo "[cloudstack]
name=cloudstack
baseurl=http://packages.shapeblue.com/cloudstack/main/centos/4.4
enabled=1
gpgcheck=1" > /etc/yum.repos.d/cloudstack.repo

Import the ShapeBlue gpg release key: (Key ID 584DF93F, Key fingerprint = 7203 0CA1 18C1 A275 68B1 37C4 BDF0 E176 584D F93F)

yum install wget -y
wget http://packages.shapeblue.com/release.asc
sudo rpm --import release.asc

Install CloudStack and MySQL

Now we can install CloudStack and MySQL Server

yum install -y cloudstack-management mysql-server

Setup NFS Server

As the CSMAN VM will also be acting as the NFS Server we need to setup the NFS environment. Run the following commands to create the folders for Primary and Secondary Storage and then export them to the appropriate IP ranges.

mkdir /exports
mkdir -p /exports/primary
mkdir -p /exports/secondary
chmod 777 -R /exports
echo "/exports/primary 10.10.100.0/24(rw,async,no_root_squash)" > /etc/exports
echo "/exports/secondary 10.10.101.0/24(rw,async,no_root_squash)" >> /etc/exports
exportfs -a

We now need to update /etc/sysconfig/nfs with the settings to activate the NFS Server. Run the following command to update the required settings

sed -i -e '/#MOUNTD_NFS_V3="no"/ c\MOUNTD_NFS_V3="yes"' -e '/#RQUOTAD_PORT=875/ c\RQUOTAD_PORT=875' -e '/#LOCKD_TCPPORT=32803/ c\LOCKD_TCPPORT=32803' -e '/#LOCKD_UDPPORT=32769/ c\LOCKD_UDPPORT=32769' -e '/#MOUNTD_PORT=892/ c\MOUNTD_PORT=892' -e '/#STATD_PORT=662/ c\STATD_PORT=662' -e '/#STATD_OUTGOING_PORT=2020/ c\STATD_OUTGOING_PORT=2020' /etc/sysconfig/nfs

We also need to update the firewall settings to allow the XenServer to access the NFS exports so run the following to setup the required settings

sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 111 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p udp -m udp --dport 111 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 2049 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p udp -m udp --dport 2049 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 2020 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 32803 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p udp -m udp --dport 32769 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 892 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p udp -m udp --dport 892 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 875 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p udp -m udp --dport 875 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 662 -j ACCEPT" /etc/sysconfig/iptables
sed -i -e "/:OUTPUT/ a\-A INPUT -p udp -m udp --dport 662 -j ACCEPT" /etc/sysconfig/iptables
service iptables restart

Then we set the nfs service to autostart on boot, and also start it now

chkconfig nfs on
service nfs start

Setup MySQL Server

The following command will adjust the MySQL Configuration for this environment

sed -i -e '/datadir/ a\innodb_rollback_on_timeout=1' -e '/datadir/ a\innodb_lock_wait_timeout=600' -e '/datadir/ a\max_connections=350' -e '/datadir/ a\log-bin=mysql-bin' -e "/datadir/ a\binlog-format = 'ROW'" -e "/datadir/ a\bind-address = 0.0.0.0" /etc/my.cnf

Then we set the mysqld service to autostart on boot, and also start it now

chkconfig mysqld on
service mysqld start

It’s always a good idea to secure a default install of MySQL and there is a handy utility to do this for you. Run the following command, setting a new password when prompted, (the current password will be blank) and accept all of the defaults to remove the anonymous user, test database and disable remote access etc.

mysql_secure_installation

Now we will login into MySQL and assign all privileges to the root account, this is so it can be used to create the ‘cloud’ account in a later step

mysql -u root -p  (enter password when prompted)
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;
mysql> quit

Setup Databases

With MySQL configured we can now setup the CloudStack Databases by running the following two commands, substituting your root password you setup earlier

cloudstack-setup-databases cloud:<password>@127.0.0.1 --deploy-as=root:<password>
cloudstack-setup-management

Nginx

There is a default example template which gets downloaded from the cloud.com web servers, but as this test system has no real public internet access we need to provide a way for the Secondary Storage VM to download this template. We achieve this by installing NGINX on the CSMAN VM, and use it to host the Template on our simulated ‘Public’ network.

First create the NGINX repo by running the following command:

echo "[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/\$releasever/\$basearch/
gpgcheck=0
enabled=1" > /etc/yum.repos.d/nginx.repo

Then install NGINX by running the following command

yum install nginx -y

Now we download the example CentOS Template for XenServer by running the following two commands

cd /usr/share/nginx/html
wget -nc http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2

We need to add a firewall rule to allow access via port 80 so run the following two commands

sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT" /etc/sysconfig/iptables
service iptables restart

Finally we start the nginx service, then test it by accessing http://192.168.56.11/ from the Host laptop

service nginx start

XenServer vhd-util

As we will be using Citrix XenServer as our Hypervisor we need to download a special utility which will get copied to every XenServer when it is added to the system. Run the following lines to download the file and update the permissions.

cd /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/
wget http://download.cloud.com.s3.amazonaws.com/tools/vhd-util
chmod 755 /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/vhd-util

Seed the CloudStack Default System VM Template

With now we need to seed the Secondary Storage with the XenServer System VM Template so run the following command

/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m /exports/secondary -u http://packages.shapeblue.com/systemvmtemplate/4.4/4.4.1/systemvm64template-4.4.1-7-xen.vhd.bz2 -h xenserver -F

CloudStack Usage Server

An optional step is to install the CloudStack Usage Service, to do so run the following command

yum install cloudstack-usage -y
service cloudstack-usage start

Customise the Configuration

For this test system to work within the limited resources available on a 4GB RAM Laptop, we need to make a number of modifications to the configuration.

Firstly we need to enable the use of a non HVM enabled XenServer. When you install XenServer on VirtualBox it warns you that it will only support PV and not HVM. To get around this we run the following SQL update command to add a new line into the Configuration table in the Cloud Database (remember to substitute your own MySQL Cloud password you used when you setup the CloudStack Database)

mysql -p<password> cloud -e \ "INSERT INTO cloud.configuration (category, instance, component, name, value, description) VALUES ('Advanced', 'DEFAULT', 'management-server', 'xen.check.hvm', 'false', 'Shoud we allow only the XenServers support HVM');"

The following MySQL commands update various global settings, and change the resources allocated to the system VMs so they will work within the limited resources available.

mysql -u cloud -p<password>
UPDATE cloud.configuration SET value='8096' WHERE name='integration.api.port';
UPDATE cloud.configuration SET value='60' WHERE name='expunge.delay';
UPDATE cloud.configuration SET value='60' WHERE name='expunge.interval';
UPDATE cloud.configuration SET value='60' WHERE name='account.cleanup.interval';
UPDATE cloud.configuration SET value='60' WHERE name='capacity.skipcounting.hours';
UPDATE cloud.configuration SET value='0.99' WHERE name='cluster.cpu.allocated.capacity.disablethreshold';
UPDATE cloud.configuration SET value='0.99' WHERE name='cluster.memory.allocated.capacity.disablethreshold';
UPDATE cloud.configuration SET value='0.99' WHERE name='pool.storage.capacity.disablethreshold';
UPDATE cloud.configuration SET value='0.99' WHERE name='pool.storage.allocated.capacity.disablethreshold';
UPDATE cloud.configuration SET value='60000' WHERE name='capacity.check.period';
UPDATE cloud.configuration SET value='1' WHERE name='event.purge.delay';
UPDATE cloud.configuration SET value='60' WHERE name='network.gc.interval';
UPDATE cloud.configuration SET value='60' WHERE name='network.gc.wait';
UPDATE cloud.configuration SET value='600' WHERE name='vm.op.cleanup.interval';
UPDATE cloud.configuration SET value='60' WHERE name='vm.op.cleanup.wait';
UPDATE cloud.configuration SET value='600' WHERE name='vm.tranisition.wait.interval';
UPDATE cloud.configuration SET value='60' WHERE name='vpc.cleanup.interval';
UPDATE cloud.configuration SET value='4' WHERE name='cpu.overprovisioning.factor';
UPDATE cloud.configuration SET value='8' WHERE name='storage.overprovisioning.factor';
UPDATE cloud.configuration SET value='192.168.56.11/32' WHERE name='secstorage.allowed.internal.sites';
UPDATE cloud.configuration SET value='192.168.56.0/24' WHERE name='management.network.cidr';
UPDATE cloud.configuration SET value='192.168.56.11' WHERE name='host';
UPDATE cloud.configuration SET value='false' WHERE name='check.pod.cidrs';
UPDATE cloud.configuration SET value='0' WHERE name='network.throttling.rate';
UPDATE cloud.configuration SET value='0' WHERE name='vm.network.throttling.rate';
UPDATE cloud.configuration SET value='GMT' WHERE name='usage.execution.timezone';
UPDATE cloud.configuration SET value='16:00' WHERE name='usage.stats.job.exec.time';
UPDATE cloud.configuration SET value='true' WHERE name='enable.dynamic.scale.vm';
UPDATE cloud.configuration SET value='9000' WHERE name='secstorage.vm.mtu.size';
UPDATE cloud.configuration SET value='60' WHERE name='alert.wait';
UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='domainrouter';
UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='elasticloadbalancervm';
UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='secondarystoragevm';
UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='internalloadbalancervm';
UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='consoleproxy';
UPDATE cloud.vm_template SET removed=now() WHERE id='2';
UPDATE cloud.vm_template SET url='http://192.168.56.11/centos56-x86_64.vhd.bz2' WHERE unique_name='centos56-x86_64-xen';
quit

service cloudstack-management restart

To enable access to the Un-Authenticated API which we have enabled on the default port of 8096, we need to add a firewall rule. Run the following commands to allow port 8096 through the firewall

sed -i -e "/:OUTPUT/ a\-A INPUT -p tcp -m tcp --dport 8096 -j ACCEPT" /etc/sysconfig/iptables
service iptables restart

Test the UI

Allow 1-2 mins for the cloudstack-management service to fully restart then login into the UI which should be accessible from the Host Laptop on http://192.168.56.11:8080/client/

The default credentials are

Username: admin
Password: password
Domain: <blank>

Create Compute Offering

The default Compute Offerings are not suitable for this limited environment so we need to create a new compute offering using the following settings:

Name: Ultra Tiny
Description: Ultra Tiny – 1vCPU, 128MB RAM
Storage Type: Shared
Custom: No
# of CPU Cores: 1
CPU (in MHz): 500
Memory (in MB): 128
Network Rate (Mb/s): null
QoS Type: null
Offer HA: Yes
Storage Tags: null
Host Tags: null
CPU Cap: No
Public: Yes
Volatile: No
Deployment Planner: null
Planner mode: null
GPU: null

Reduce the amount of RAM

Following a successful login to the UI, the Databases will be fully deployed so now we can reduce the RAM to 1GB to free up memory for our XenServer VM. Shutdown the VM and change the settings to 1024 MB of RAM.

XenServer VM

To configure the XenServer you will need XenCenter running on your local Host if you are running Windows, but if your Host is running OSX or Linux, then you need to add a Windows VM which can run XenCenter. You can download XenCenter from http://downloadns.citrix.com.edgesuite.net/akdlm/8160/XenServer-6.2.0-XenCenter.msi

Create a VM for XenServer using the following settings:

Name: XenServer
Type: Linux
Version: Red Hat (64 bit)
vCPU: 2
RAM: 1536 (If your host has 8GB of RAM, consider allocating 3072)
Hard Drive: VDI – Dynamic – 24 GB

Note: VirtualBox seems to mix up the networks if you add them all at the same time so we add the 1st Network and install XenServer, then once fully installed, we add the additional networks, rebooting in-between. This appears to be a bug in the latest versions of VirtualBox (4.3.18 at the time of writing)

Modify the settings and assign ONLY the 1st network Adapter correct networks as follows:

Note how we have set the ‘Promiscuous Mode’ to ‘Allow All’

Now install XenServer 6.2 by downloading the ISO from http://downloadns.citrix.com.edgesuite.net/akdlm/8159/XenServer-6.2.0-install-cd.iso and booting the VM.

The XenServer installation wizard is straightforward, but you will get a warning about the lack of hardware virtualisation support, this is expected as VirtualBox does not support this. Accept the warning and continue.

Choose the appropriate regional settings and enter the following details when prompted: (we enter the IP of the CSMAN VM for DNS and NTP, whilst this guide does not cover setting up these services on the CSMAN VM, this gives you the option of doing so at a later date)

Enable Thin Provisioning: Yes
Install source: Local media
Supplemental Packs: No
Verification: Skip
Password: <password>
Static IP: 192.168.56.101/24 (no gateway required)
Hostname: xenserver
DNS: 192.168.56.11
NTP: 192.168.56.11

Once the XenServer Installation has finished, detach the ISO and reboot the VM.

We now need to change the amount of RAM allocated to Dom0 to its minimum recommended amount which is 400MB, we do this by running the following command on the XenServer console

 /opt/xensource/libexec/xen-cmdline --set-xen dom0_mem=400M,max:400M 

XenServer Patches

It’s important to install XenServer Patches and whilst XenCenter will inform you of the required patches, as we are using the OpenSource version of XenServer we have to install Patches via the command line. Fortunately there are a number of ways of automating this process.

Personally I always use PXE to deploy XenServer and the installation of patches is built into my deployment process. However that is out of scope for this article, but Tim Mackey has produced a great blog article on how to do this: http://xenserver.org/discuss-virtualization/virtualization-blog/entry/patching-xenserver-at-scale.html

Whilst Tim’s method of rebooting after every patch install is best practice, it can take a long time to install all Patches so an alternative approach I use in these non-production test environments is detailed here https://github.com/amesserl/xs_patcher  This installs all patches and requires only a single reboot.

The configuration file ‘clearwater’ is now a little out of date, and should contain the following (and the cache folder should contain the associated patch files):

XS62E014|78251ea4-e4e7-4d72-85bd-b22bc137e20b|downloadns.citrix.com.edgesuite.net/8736/XS62E014.zip|support.citrix.com/article/CTX140052

XS62ESP1|0850b186-4d47-11e3-a720-001b2151a503|downloadns.citrix.com.edgesuite.net/8707/XS62ESP1.zip|support.citrix.com/article/CTX139788

XS62ESP1003|c208dc56-36c2-4e91-b8d7-0246575b1828|downloadns.citrix.com.edgesuite.net/9031/XS62ESP1003.zip|support.citrix.com/article/CTX140416

XS62ESP1005|1c952800-c030-481c-a0c1-d1b45aa19fcc|downloadns.citrix.com.edgesuite.net/9058/XS62ESP1005.zip|support.citrix.com/article/CTX140553

XS62ESP1009|a24d94e1-326b-4eaa-8611-548a1b5f8bd5|downloadns.citrix.com.edgesuite.net/9617/XS62ESP1009.zip|support.citrix.com/article/CTX141191

XS62ESP1013|b22d6335-823d-43a6-ba26-28793717125b|downloadns.citrix.com.edgesuite.net/9703/XS62ESP1013.zip|support.citrix.com/article/CTX141480

XS62ESP1014|4fc82e62-b938-407d-a2c6-68c8922f3ec2|downloadns.citrix.com.edgesuite.net/9708/XS62ESP1014.zip|support.citrix.com/article/CTX141486

Once you have your XenServer fully patched shut it down and then add the 2nd Adapter, again note how we have set the ‘Promiscuous Mode’ to ‘Allow All’

Boot the VM and then using XenCenter perform a ‘Rescan’ on the NICs to detect this new NIC, then shutdown and add the 3rd Adapter, again note how we have set the ‘Promiscuous Mode’ to ‘Allow All’

Boot the VM and then using XenCenter perform a ‘Rescan’ on the NICs to detect this new NIC, then shutdown and add the 4th Adapter, again note how we have set the ‘Promiscuous Mode’ to ‘Allow All’

Boot the VM and then using XenCenter perform a ‘Rescan’ on the NICs to detect this final NIC, then one final reboot to make sure they are all activated and connected.

Configure XenServer Networks

Now we are ready to configure the XenServer Networks. We should have the following four networks present, and it’s worth just checking the MACs line up with the Adapters in VirtualBox.

We need to rename the networks using a more logical naming convention, and also create the two Storage Networks, and assign their VLANs etc.

First of all start by renaming them all setting the MTU of the Storage Network to 9000 (the rest remain at the default of 1500)

Network 0 – MGMT
Network 1 – GUEST
Network 2 – PUBLIC
Network 3 – STORAGE (and MTU of 9000)

Next we add the Primary Storage Network using the following settings:

Type: External Network
Name: PRI-STORAGE
NIC: NIC 3
VLAN: 100
MTU: 9000

Then the Secondary Storage Network:

Type: External Network
Name: SEC-STORAGE
NIC: NIC 3
VLAN: 101
MTU: 9000

Finally we add the IP addresses for the Primary and Secondary Storage Networks so the XenServer can access them

Name: PRI-STOR
Network: PRI-STORAGE
IP address: 10.10.100.101
Subnet mask: 255.255.255.0
Gateway: <blank>

Name: SEC-STOR
Network: SEC-STORAGE
IP address: 10.10.101.101
Subnet mask: 255.255.255.0
Gateway: <blank>

That is all the configuration required for XenServer so now we can proceed with deploying our first Zone. However before we do, it’s worth taking a snapshot of both of the VMs so you can roll back and start again if required.

Zone Deployment

We now add an Advanced Zone by going to ‘Infrastructure/Zones/Add Zone’ and creating a new Zone of type ‘Advanced’ without Security Groups

Zone Name – Test
IPv4 DNS1 – 8.8.8.8
Internal DNS 1 – 192.168.56.11
Hypervisor – XenServer
Guest CIDR – 10.1.1.0/24

Next we need to setup the XenServer Traffic Labels to match the names we allocated to each Network on our XenServer, and we also need to add the optional Storage Network by dragging it onto the Physical Network.

Edit each Traffic Type and set the following Labels:

Management Network – MGMT
Public Network – PUBLIC
Guest Network – GUEST
Storage Network – SEC-STORAGE

Then continue through the add zone wizard using the following settings

Public Traffic

Gateway – 172.30.0.1
Netmask – 255.255.255.0
VLAN – <blank>
Start IP – 172.30.0.21
End IP -172.30.0.30

POD Settings

POD Name – POD1
Reserved System Gateway – 192.168.56.1
Reserved System Netmask – 255.255.255.0
Start Reserved System IP – 192.168.56.21
End Reserved System IP – 192.168.56.30

Guest Traffic

VLAN Range – 600 – 699

Storage Traffic

Gateway – 10.10.101.1
Netmask – 255.255.255.0
VLAN – <blank>
Start IP – 10.10.101.21
End IP – 10.10.101.30

Cluster Settings

Hypervisor – XenServer
Cluster Name – CLU1

Host Settings

Host Name – 192.168.56.101
Username – root
Password – <password>

Primary Storage Settings

Name – PRI1
Scope – Cluster
Protocol – nfs
Server – 10.10.100.11
Path – /exports/primary
Provider: DefaultPrimary
Storage Tags: <BLANK>

Secondary Storage Settings

Provider – NFS
Name – SEC1
Server – 10.10.101.11
Path – /exports/secondary

At the end of it, activate the Zone, then allow approx. 5 minutes for the System VMs to deploy and the default CentOS Template to be ‘downloaded’ into the system. You are now ready to deploy your first Guest VM.