Citrix’s CloudStack 3.0 -Advanced Zone Setup

CloudStack 3.0, a product by Cloud.com (recently acquired by Citrix), implements IaaS (Infrastructure as a service) style private clouds. It is an open source product for the deployment and management of large networks of virtual machines to create a scalable cloud computing platform. 

Recently, in one of our projects we needed a Cloudstack 3.0 setup with advanced mode networking options. Cloudstack has two options of networking setup viz basic and advanced. Setting up Cloudstack in basic mode is pretty simple! But, for setting up Cloudstack with advance mode one needs a clear understanding of internals of networking in advanced mode. This post will cover most of the concepts required along with step by step guide of advanced mode Cloudstack setup.

CloudStack 3.0 basic mode does not have VLAN enabled i.e. all instances created under basic mode have a public IP assigned. Users outside the platform can access these instances directly using the public IP associated with those instances. Also, basic mode doesn’t provide flexibility in networking. Setting up advanced mode provides such flexibility. In advanced mode, admins can customize networking options such as VPN access, load balancing, firewall and port forwarding for particular account. Virtual routers (setup using VLANs, refer diagram) in advance mode serves as abstraction for all the instances associated with that account and helps in routing the network traffic.

Network diagram-Cloudstack Advanced zone setup

Lets first discuss the common terminology used in CloudStack setup -


Common terminologies:

Hosts:

Hosts are basic physical blocks of the Cloudstack platform. A host can be single XenServer or ESX server. The number of guest virtual machines that can be hosted on Cloudstack can be determined by number of hosts and capacity of each host. Hosts are not visible to the end user. An end user cannot determine which Host their guest has been assigned to.

CloudStack resource hierarchy

Clusters:

Clusters are second level of physical scaling in Cloudstack platform. Cluster is a group of hosts (for Ex. XenServer or VSphere) that have same hypervisor type and share the primary storage. Size of cluster is limited by underlying hypervisor type. In a particular cluster, instances can migrate from one host to another.

Clusters can be Cloud Managed or Externally Managed. For Ex. ‘Xenserver’ based clusters are ‘Cloudmanaged’ however, for ‘VMWare’ based clusters are managed by vCenter servers.

Pods:

Pod is a collection of different types of clusters. It is a third level of physical scaling in CloudStack platform. The pod may contain only one cluster or multiple clusters with different base hypervisor type. A Pod is frequently mapped to a single rack with a layer-2 switch.

Zones:

Zones are fourth level of physical scaling in Cloudstack platform. Zones can also provide physical isolation from other zones. Users have option to choose zone while deploying their guest VMs. Admins can setup private zones which is only accessible to specific domain. Each node in zone shares the secondary storage and network.

CloudStack Availability zones

Primary storage:

Guest VM’s root disks and other additional data disks are stored on primary storage. Instances in a particular cluster have same primary storage for them. Speed of primary storage directly impact guest VM performance.

Root volumes are created when guest VM is created on Cloudstack platform. Additional data disks can be added later or they can be added while guest VM deployment. Root disks are deleted when guest VM is destroyed. but, data disks are not deleted from primary storage when VM is destroyed which is a main difference between two types.

Secondary Storage:

Secondary storage is used for storing templates, ISO images and snapshots on cloudstack platform. Submissions to secondary storage go through the Secondary Storage VM (System VM). The Secondary Storage VM can retrieve templates and ISO images from URLs.

Each zone can have multiple secondary storage devices of type NFS added to them.


Different types of Networks in CloudStack platform

Before we start discussing the setup & configurations for advance mode, lets first understand the types of network used in Cloudstack platform. CloudStack has three networks – Public, Private and Link local network.

Every account has a internal network associated with it called a “link local network”. All guest VMs associated with the account communicate on this network and guest VMs on two different clusters (for Ex. XenServer and VSphere) communicate on a private network. Through public network, other users can access the CloudStack resources. [refer diagram below]

CloudStack Networking

Public network

Users of Internet can access Cloudstack resources through Public network. For every account source NAT is created and a public IP is allocated to that network. Resources belonging to this account, if public, are accessible through the public IP address. Multiple public IPs can be assigned to single resource.

Private network

Two instances on different hosts communicate with each other using private network. For Ex. The client has instances on two different hypervisors- ‘Xenserver’ and ‘VMWare’ and if these instances want to communicate in between them then they can communicate on private network.

Link local network

The Management Server automatically creates a Virtual Router for each guest virtual network. A virtual router is a special virtual machine that runs on the Hosts. This router communicates with other resources of the account on link local network as shown in diagram above.

We will now discuss how to setup Advance zone with VLANs-

Setting Up Zone in Advanced mode

Advanced mode provides the most flexibility in allowing administrators to provide custom network offerings such as providing Firewall, VPN, or Load Balancer support as well as enabling direct vs virtual networking. Advanced mode of CloudStack setup can be setup in two modes – with VLANs or with Security groups.

Step 1: Login to admin console.

Step 2: Navigate to menu: System > Zones from let side bar menues

Step 3: Select Add zone and choose ‘Advanced’. Click next.

Step 4: Enter name and other details provided by your network provider.

Step 5: In CloudStack 3.0, for every NIC present on management server, a separate physical network is should be created. If the management server has only one NIC, click next else add additional physical networks.

Step 6: CloudStack’s resources like virtual machines are accessible to outside world through domain router. Every domain router is associated with a public IP which is picked from pool of IP addresses provided at time of zone setup. Add appropriate public IP range for CloudStack setup.

NOTE: If VLAN ID is specified, all the CloudStack requests will be tagged with that VLAN ID. For communication with CloudStack’s resources, requests should be tagged with VLAN ID specified. If VLAN ID is not specified, outside resources can communicate with CloudStack without any restrictions.

Step 7: The next step is to add pod specific settings. The reserved System IPs are used for creation of private network.

Step 8: Enter the VLAN Range which will be used by CloudStack for it’s internal communication. A separate VLAN range should be specified for every physical network created in step 5.

Step 9: Add Cluster, host configurations with primary and secondary storage servers. Click next. CloudStack will create physical network, pods and clusters inside new zone. System VMs will come up after successfully adding host, primary and secondary storage in the zone. Wait for their state to change to ‘Running’ state.

Step 10: Check status of BUILTIN templates from templates menu. If they are in downloading state, wait till the downloading is complete.

That’s it! You have setup a Cloudstack with advanced networking options and now you can spawn your first VM instance.

  • Geoff

    Great attempt at an introduction to the two networking models in CloudStack V3.0.0, however there are a number of fundamental errors in this post.

    When using the Basic Networking option, you are building a flat network architecture, and all Guest Instances share the same IP range as the Hypervisors and the Management Server. As these will all be on a Private IP Range behind a Firewall, the Guest VMs do not have Public IPs assigned, but Private IPs. The Guest VMs also have no security segmentation so this is only to be used for a Private Cloud.

    The Secondary Storage sits at the Availability Zone level, and not at the POD level. The Secondary Storage provides storage for all the Templates, ISO images and Snapshots, and is accessible to every Pod, Cluster and Host in that Zone. A new feature of V3.0.0 is the ability to use OpenStack Object Storage (Swift) which effectively moves the Secondary Storage Layer up to the Cloud Level, meaning it can be accessed from ANY Availability Zone, POD, Cluster, Host etc.

    There are actually four main network types in V3.0.0 

    The Management Network is used by the CloudStack Management Server to communicate with the Hosts. 

    The Guest Network is used by Guest VM Instances to communicate with other VM Instances belonging to the same account, or Project, and should utilise Tagged VLANs for optimum security and functionality. 

    The Public Network which should also use Tagged VLANs provides the connectivity to Public Internet via the accounts dedicated Virtual Router, which can have multiple Public IP addresses. Users can also configure VPNs, Firewall Rules, NAT, Load Balancing etc directly from the CloudStack interface.

    The optional Storage Network can be dedicated to traffic between the Primary and Secondary Storage devices. If no Storage Network is configured, the Management Network is used for actions relating to Primary and Secondary Storage. 

    It is recommended to Team or Bond Dedicated NICs used on the Guest and Storage Networks to improve performance and reliability. 

    The Link-Local Network is used only by XenServer Hosts to communicate with the System VMs, providing a capacity of 65,000 private IPs, whilst VMWare uses IPs from the Management Network Range, which if configured incorrectly could limit the number of Guest VMs to less than 250.

    When setting up a new Availability Zone you need to navigate to ‘Infrastructure’ tab on the left hand menu, then select ‘View More’ under the Zones section then click ‘+ Add Zone’ The ‘System / Physical Resources’ options were in Version 2 and some V3 Betas, but not the production version of V3.0.0

    When configuring the Public Traffic section of the Add New Zone Dialogue, a Tagged VLAN should be configured, for example 500, this ensures that the Public Traffic is segmented from other Public Networks created at a later date, and that the Layer 3 Switches can route the traffic to the Internet or other Availability Zones as required.

    The POD configuration (not shown in this post) requires Reserved IPs to be assigned for use by the System VMs, a range of approx. 7 will suffice, but they must be unique throughout the whole Cloud, and be in the same IP range used by the Hosts in the POD.

    The recommended range for Guest VLANs is 600-999, you can only use higher ranges if your switch architecture supports Transparent VLANs. Each Account will be allocated a unique VLAN ID, and this ensures all communication between VM Instances within that Account remains secure.

    When adding Primary Storage (not shown) a number of Protocol options are available, NFS, iSCSI or Pre-Setup. The Pre-Setup option allows any storage supported by the chosen HyperVisor being used, it just needs to be pre-configured on each Host before it is added to the Cluster. 

    If the System VMs are not created review the log files for problems relating to either Secondary or Primary Storage as often the devices security layer is preventing access.

  • Geoff

    It appears we are not allowed to add constructive critism, it will just be deleted!

    • http://www.tutkiun.com/ Mayur

      @fa493c4d121aa555d853ed65d14346f9:disqus  – Thanks for dropping by!

      Looks like the commenting system auto filtered the comment and put ‘em in junk folder. I just removed it from junk folder.