Easy OpenStack Folsom with VirtualBox and Vagrant

Testing OpenStack is now as easy thanks to VirtualBox and Vagrant. To run a mini test environment with Compute, Cinder, Keystone and Horizon you just need the following tools:

  • VirtualBox
  • Vagrant
  • Git client

Getting Ready

To set up a sandbox environment within VirtualBox to run OpenStack Folsom you will need to download:

Installation of these tools are simple – follow the on-screen prompts.

When ready we need to configure the VirtualBox “Host-Only” Networking. This networking mode allows us to communicate with our VirtualBox guest and our underlying host.
We will set up the following:

  • Host-Only Network: IP 172.16.0.254; Network 172.16.0.0/255.255.0.0; Disable DHCP
  • Host-Only Network #2: IP 10.0.0.254; Network 10.0.0.0/255.0.0.0; Disable DHCP

(Hint: there is a bash script @ https://raw.github.com/uksysadmin/OpenStackInstaller/folsom/virtualbox/vbox-create-networks.sh to create these for you).

How To Do It

To create a VirtualBox VM, running Ubuntu 12.04 with OpenStack Folsom from Ubuntu’s Cloud Archive, carry out the following

1. Clone the GitHub OpenStackInstaller scripts

git clone https://github.com/uksysadmin/OpenStackInstaller.git

2. Make the scripts the ‘folsom’ branch

cd OpenStackInstaller
git checkout folsom

3. Run ‘vagrant’ to launch your OpenStack instance which will come up with IP 172.16.0.201

cd virtualbox
vagrant up

4. After a short while your instance will be ready. Note that on the first run, Vagrant will download a 384Mb Precise64 “box”. Subsequent launches will not require this step.

Launch a web browser at http://172.16.0.201/horizon and log in with:

Username: admin
Password: openstack

(Note, to edit the IP it is assigned, modify virtualbox/vagrant-openstack-bootstrap.sh (Warning its a bit of a sed hack!).

Advertisements

Running OpenStack under VirtualBox – A Complete Guide (Part 2)

In the previous post, we very simply got OpenStack running under VirtualBox. This next part takes this further by getting multiple compute nodes installed to spread the load of your instances. It also paves the way for Part 3 where we will then start to look at Swift, OpenStack’s storage implementation of S3.

Part 2 – OpenStack on a multiple VirtualBox VMs with OpenStack instances accessible from host

We will be using the same set up as for Part 1.

The proposed environment

  • OpenStack “Public” Network: 172.241.0.0/25
  • OpenStack “Private” Network: 10.0.0.0/8
  • Host has access to its own LAN, separate to this on 192.168.0.0/16 and not used for this guide
  • One VirtualBox VM running the software needed for the controller
  • One VirtualBox VM running the software needed for a compute node

This guide will assume you have followed Part 1. We are simply adding in a compute node now, so that will make the virtual machine you created in Part 1 will become the Cloud Controller (CC). Also, OpenStack has been designed so that any part of the environment can run on a separate server. For this guide we will have the following

  • Cloud Controller running MySQL, RabbitMQ, nova-network, nova-scheduler, nova-objectstore, nova-api, nova-compute
  • Compute node running: nova-compute

The Guide

  • Add a new VirtualBox Guest
    • Name: cloud2
      • OS Type: Linux
      • Version: Ubuntu (64-Bit)
    • 2048Mb Ram
    • Boot Hard Disk
      • Dynamically Expanding Storage
      • 8.0Gb
    • After this initial set up, continue to configure the guest
      • Storage:
        • Edit the CD-ROM so that it boots Ubuntu 10.10 Live or Server ISO
        • Ensure that the SATA controller has Host I/O Cache Enabled (recommended by VirtualBox for EXT4 filesystems)
      • Network:
        • Adapter 1
          • Host-only Adapter
          • Name: vboxnet0
        • Adapter 2
          • NAT
          • This will provide the default route to allow the VM to access the internet to get the updates, OpenStack scripts and software
      • Audio:
        • Disable (just not required)
    • Boot the guest and install Ubuntu as per normal
  • Assign static IPs to the cloud controller
    • Ensure that the Cloud Controller you created in Part 1 has static addresses for eth0 and eth1.
      • For the sake of this guide, I’m assuming you have assigned the following
        • eth0: 172.241.0.101/255.255.255.128.  This address is your Cloud Controller address (CC_ADDR) and will be the API interface address you will be communicating on from your host.
        • eth1: stays as dhcp as it is only used for NAT’d access to the real world
    • Your compute nodes don’t need to be set statically, but for the rest of this guide it is assumed the addresses are as follows
      • Cloud2
        • eth0: 172.241.0.102/255.255.255.128
        • eth1: stays as dhcp as it is only used for NAT’d access to the real world
  • Grab this script to install OpenStack
    • This will set up a repository (ppa:nova/trunk) and install MySQL server where the information regarding your cloud will be stored
    • The options specified on the command line match the environment described above
    • wget --no-check-certificate \
      https://github.com/uksysadmin/OpenStackInstaller/raw/master/OSinstall.sh
  • Run the script (as root/through sudo) specifying that you want a compute node and that you’ve specified the IP address of the Cloud Controller
    • sudo bash ./OSinstall.sh -A $(whoami) -T compute -C 172.241.0.101
  • No further configuration is required.
  • Once completed, ensure that the Cloud Controller knows about this new compute node
    • On the Cloud Controller run the following
      • mysql -uroot -pnova nova -e 'select * from services;'
      • sudo nova-manage service list
      • You should see your new compute node listed under hosts
      • If you don’t have a DNS server running that resolves these hosts add your new compute node to /etc/hosts
        • 172.241.0.102 cloud2
  • On the new compute node run the following
    • sudo nova-manage db sync
  • As you copied your credentials from the cloud controller created in Part 1, you should just be able to continue to use this from your host – but this time you can spin up more guests.
    • If you changed the eth0 address of your cloud controller, ensure your cloud/creds/novarc environment file has the correct IP.
  • Repeat the steps above to create further compute nodes in your environment, scaling seamlessly

Running OpenStack under VirtualBox – A Complete Guide (Part 1)

UPDATE: I’ve been working on a new version of the script which can be used to create an OpenStack host running on Ubuntu 12.04 Precise Pangolin and the Essex release.
I’ve now got a video to accompany this which is recommended over this guide
Head over to  ‎https://uksysadmin.wordpress.com/2012/03/28/screencast-video-of-an-install-of-openstack-essex-on-ubuntu-12-04-under-virtualbox/

Running OpenStack under VirtualBox allows you to have a complete multi-node cluster that you can access and manage from the computer running VirtualBox as if you’re accessing a region on Amazon.
This is a complete guide to setting up a VirtualBox VM running Ubuntu, with OpenStack running on this guest and an OpenStack instance running, accessible from your host.

Part 1 – OpenStack on a single VirtualBox VM with OpenStack instances accessible from host

The environment used for this guide

  • A 64-Bit Intel Core i7 Laptop, 8Gb Ram.
  • Ubuntu 10.10 Maverick AMD64 (The “host”)
  • VirtualBox 4
  • Access from host running VirtualBox only (so useful for development/proof of concept)

The proposed environment

  • OpenStack “Public” Network: 172.241.0.0/25
  • OpenStack “Private” Network: 10.0.0.0/8
  • Host has access to its own LAN, separate to this on 192.168.0.0/16 and not used for this guide

The Guide

  • Download and install VirtualBox from http://www.virtualbox.org/
  • Under Preferences… Network…
  • Add/Edit Host-only network so you have vboxnet0. This will serve as the “Public interface” to your cloud environment
    • Configure this as follows
      • Adapter
        • IPv4 Address: 172.241.0.100
        • IPv4 Network Mask: 255.255.255.128
      • DHCP Server
        • Disable Server
    • On your Linux host running VirtualBox, you will see an interface created called ‘vboxnet0’ with the address specified as 172.241.0.100. This will be the IP address your OpenStack instances will see when you access them.
    • Create a new Guest
      • Name: Cloud1
        • OS Type: Linux
        • Version: Ubuntu (64-Bit)
      • 1024Mb Ram
      • Boot Hard Disk
        • Dynamically Expanding Storage
        • 8.0Gb
      • After this initial set up, continue to configure the guest
        • Storage:
          • Edit the CD-ROM so that it boots Ubuntu 10.10 Live or Server ISO
          • Ensure that the SATA controller has Host I/O Cache Enabled (recommended by VirtualBox for EXT4 filesystems)
        • Network:
          • Adapter 1
            • Host-only Adapter
            • Name: vboxnet0
          • Adapter 2
            • NAT
            • This will provide the default route to allow the VM to access the internet to get the updates, OpenStack scripts and software
        • Audio:
          • Disable (just not required)
    • Power the guest on and install Ubuntu
    • For this guide I’ve statically assigned the guest with the IP: 172.241.0.101 for eth0 and netmask 255.255.255.128.  This will be the IP address that you will use to access the guest from your host box, as well as the IP address you can use to SSH/SCP files around.
    • Once installed, run an update (sudo apt-get update&&sudo apt-get upgrade) then reboot
    • If you’re running a desktop, install the Guest Additions (Device… Install Guest Additions, then click on Places and select the VBoxGuestAdditions CD and follow the Autorun script), then Reboot
    • Install openssh-server
      • sudo apt-get -y install openssh-server
    • Grab this script to install OpenStack
      • This will set up a repository (ppa:nova/trunk) and install MySQL server where the information regarding your cloud will be stored
      • The options specified on the command line match the environment described above
      • wget https://github.com/uksysadmin/OpenStackInstaller/raw/master/OSinstall.sh
    • Run the script (as root/through sudo)
      • sudo bash ./OSinstall.sh -A $(whoami)
    • Run the post-configuration steps
      • ADMIN=$(whoami)
        sudo nova-manage user admin ${ADMIN}
        sudo nova-manage role add ${ADMIN} cloudadmin
        sudo nova-manage project create myproject ${ADMIN}
        sudo nova-manage project zipfile myproject ${ADMIN}
        mkdir -p cloud/creds
        cd cloud/creds
        unzip ~/nova.zip
        . novarc
        cd
        euca-add-keypair openstack > ~/cloud/creds/openstack.pem
        chmod 0600 cloud/creds/*

    Congratulations, you now have a working Cloud environment waiting for its first image and instances to run, with a user you specified on the command line (yourusername), the credentials to access the cloud and a project called ‘myproject’ to host the instances.

    • You now need to ensure that you can access any instances that you launch via SSH as a minimum (as well as being able to ping) – but I add in access to a web service and port 8080 too for this environment as my “default” security group.
      • euca-authorize default -P tcp -p 22 -s 0.0.0.0/0
        euca-authorize default -P tcp -p 80 -s 0.0.0.0/0
        euca-authorize default -P tcp -p 8080 -s 0.0.0.0/0
        euca-authorize default -P icmp -t -1:-1
    • Next you need to load a UEC image into your cloud so that instances can be launched from it
      • image="ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz"
        wget http://smoser.brickies.net/ubuntu/ttylinux-uec/$image
        uec-publish-tarball $image mybucket
    • Once the uec-publish-tarball command has been run, it will present you with a line with emi=, eri= and eki= specifying the Image, Ramdisk and Kernel as shown below. Highlight this, copy and paste back in your shell
      Thu Feb 24 09:55:19 GMT 2011: ====== extracting image ======
      kernel : ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz
      ramdisk: ttylinux-uec-amd64-12.1_2.6.35-22_1-initrd
      image  : ttylinux-uec-amd64-12.1_2.6.35-22_1.img
      Thu Feb 24 09:55:19 GMT 2011: ====== bundle/upload kernel ======
      Thu Feb 24 09:55:21 GMT 2011: ====== bundle/upload ramdisk ======
      Thu Feb 24 09:55:22 GMT 2011: ====== bundle/upload image ======
      Thu Feb 24 09:55:25 GMT 2011: ====== done ======
      emi="ami-fnlidlmq"; eri="ami-dqliu15n"; eki="ami-66rz6vbs";
    • To launch an instance
      • euca-run-instances $emi -k openstack -t m1.tiny
    • To check its running
      • euca-describe-instances
      • You will see the Private IP that has been assigned to this instance, for example 10.0.0.3
    • To access this via SSH
      • ssh -i cloud/creds/openstack.pem root@10.0.0.3
      • (To log out of ttylinux, type: logout)
    • Congratulations, you now have an OpenStack instance running under OpenStack Nova, running under a VirtualBox VM!
    • To access this outside of the VirtualBox environment (i.e. back on your real computer, the host) you need to assign it a “public” IP
      • Associate this to the instance id (get from euca-describe-instances and will be of the format i-00000000)
        • euca-allocate-address
        • This will return an IP address that has been assigned to your project so that you can now associate to your instance, e.g. 172.241.0.3
        • euca-associate-address -i i-00000001 172.241.0.3
      • Now back on your host (so outside of VirtualBox), grab a copy of cloud/creds directory
        • scp -r user@172.241.0.101:cloud/creds .
      • You can now access that host using the Public address you associated to it above
        • ssh -i cloud/creds/openstack.pem root@172.241.0.3

    CONGRATULATIONS! You have now created a complete cloud environment under VirtualBox that you can manage from your computer (host) as if you’re managing services on Amazon. To demonstrate this you can terminate that instance you created from your computer (host)

    • sudo apt-get install euca2ools
      . cloud/creds/novarc
      euca-describe-instances
      euca-terminate-instances i-00000001

    Credits

    This guide is based on Thierry Carrez’ blog @ http://fnords.wordpress.com/2010/12/02/bleeding-edge-openstack-nova-on-maverick/

  • Next: Part 2 – OpenStack on a multiple VirtualBox VMs with OpenStack instances accessible from host

Running OpenStack under VirtualBox

This page has been superseded by Running OpenStack under VirtualBox – A Complete Guide.

(There’s still some good things on here though)

Running OpenStack under VirtualBox is detailed on many pages on the internet.  The Wiki at OpenStack.org has an intro on getting this going and why you would want to do this.  One big reason is that getting this great cloud software running under virtual hardware means you can set up multi-node clusters without a big outlay in hardware. This allows you to develop your cloud environment under the safety and convenience of your own machine.

The steps below are a mixture of instructions from http://fnords.wordpress.com/2010/12/02/bleeding-edge-openstack-nova-on-maverick/ and http://wiki.openstack.org/NovaInstall/.

What you will be setting up

Instructions

  1. Install VirtualBox
  2. Create a Ubuntu 64 Guest
    • 1vCPU
    • 1024Mb Ram
    • 8Gb Disk
    • Enable Hardware VT-x/AMD-V if available
    • Add in an extra NIC, Host-only Adapter
  3. Once installed, run the updates (and optionally install the Guest Additions if you’re running the desktop version) and reboot.
  4. Assign a static IP to your eth1 interface (Host-only) – you will use this to access the guest from your host.
  5. To install OpenStack follow the instructions http://fnords.wordpress.com/2010/12/02/bleeding-edge-openstack-nova-on-maverick/

Warning for nested virtualization

Since you are running virtualization software under virtualization software (nested virtualization) some words of warning:

Intel VT-x: KVM does NOT currently support nested virtualization

To run instances under OpenStack under VirtualBox, you must specify that software emulation be used

sudo apt-get install qemu

Edit /etc/nova/nova.conf to enable qemu (software virtualization) support

--libvirt_type=qemu

AMD-V: Enable nested KVM virtualization support

To enable AMD’s KVM support create a file /etc/modprobe.d/kvm_amd.conf with the following in

options kvm_amd nested=1

And restart all the OpenStack services

service libvirt-bin restart; service nova-network restart; service  nova-compute restart; service nova-api restart; service nova-objectstore  restart; service nova-scheduler restart

Known Issues

Currently, at the time of writing, there is a bug in some of the Python scripts used to launch instances that may cause the following error to be thrown: KeyError: ‘imageId’ when creating instance with EC2/S3. A patch is available.

wget http://launchpadlibrarian.net/64364074/x.patch -O /tmp/KeyError_imageId.patch
cd /usr/lib/pymodules/python2.6
sudo patch -p0 < /tmp/KeyError_imageId.patch
service libvirt-bin restart; service nova-network restart; service   nova-compute restart; service nova-api restart; service nova-objectstore   restart; service nova-scheduler restart

Installing OpenSUSE 11.3 under Virtual Box 3.2

You will need

  1. VirtualBox
  2. OpenSUSE Live CD

Instructions

Guest Additions

  1. Update the packages
    zypper up
  2. Reboot
  3. Install Kernel Development Packages
    sudo zypper in -t pattern devel_kernel
  4. Mount the VirtualBox Guest Additions CD[VirtualBox Menu] Devices… Install Guest Additions
  5. Run the installer
    sudo /media/VBOXADDITIONS_3.2.10_66523/VBoxLinuxAdditions-x86.run
  6. Reboot

VirtualBox Windows Host, Shared Folders and Linux Guest

VirtualBox comes with a feature to give you access to the hosts’s filesystem through a CIFS server.  This is available after you’ve installed the Guest Additions drivers in the Guest.  More details are available here (specific to Ubuntu but can apply to all versions of Linux): https://help.ubuntu.com/community/VirtualBox

There are more details at that page on how to set up the share.

Frustratingly, the CIFS (Shared Folder) server name you access a hosts file system on a Linux Guest is different to one that you use under Windows so as a quick overview

Windows Guest

net use x: vboxsvrshare

Linux Guest

mount -t vboxsf share mountpoint

Note the change from vboxsvr under Windows to vboxsf under Linux.

Run Windows XP from raw partition under Linux

Introduction
I have managed to successfully boot my Windows XP raw partition from within Linux (Ubuntu, Karmic Koala Alpha 4) using Sun’s Virtual Box so I thought I’d share my successes and gotchas with you.

Warnings and expectations!
I accept no responsibility for the loss or damage to your data or computer by following these instructions!

First up – these instructions worked for me and were collated from lots of googling and sanitising other instructions found on the internet. I’m hoping these are a little more straightforward to follow than some I’ve seen.
Secondly, although I got this going, it wasn’t necessarily the ideal set up I was hoping for. The computer I was using for this was an Compaq NC6400 with 4Gb Ram and a very slow hard drive. There are no hardware virtualisation features on my model so although it was cool to get it working, the speed of disk access was nearly enough to see me go mad.

Disk Layout
My hard drive disk layout was your pretty average Windows XP partition followed by some Linux partitions:

Disk /dev/sda: 60.0 GB, 60022480896 bytes
255 heads, 63 sectors/track, 7297 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x7e987e98

Device Boot Start End Blocks Id System
/dev/sda1 * 1 5338 42877453+ 7 HPFS/NTFS
/dev/sda2 5339 5363 200812+ 83 Linux
/dev/sda3 5364 5736 2996122+ 82 Linux swap / Solaris
/dev/sda4 5737 7297 12538732+ 83 Linux

Pre-Requisite – Windows XP Preparation
There are a couple of things that need to be done under Windows XP prior to starting this. This includes running the MergeIDE utility which preps a system to be moved to another piece of hardware (in our case, a virtual machine) as Windows XP isn’t so friendly when it comes across different hardware. Also you need to create hardware profiles – you will be booting your original Windows XP that has been running under your specific hardware, and not Virtual Box’s virtual hardware.

  1. Download, extract and run the MergeIDE utility from this site
  2. Create a hardware profile under Windows XPRight Click on My Computer and go to Properties, then click on the Hardware tab. From there you can create new hardware profiles which you can select on boot.


Virtual Box and Windows XP raw partition
Sun’s VBox can be downloaded from Sun’s Virtual Box Website or by using your Linux distribution’s software installer. As I was running Ubuntu the instructions will be specific to that.

  1. Ensure your user is a member of the ‘disk’ group as your normal unprivileged user needs access to the partitions that Linux can seesudo usermod -a -G disk username

    You will need to log out and log back in again for the whole X desktop (e.g. Gnome) session to see the changes.

  2. sudo aptitude install virtualbox-ose mbrmbr package provides a means of creating a new boot record that will be used to boot your partition

    the virtualbox-ose package is the Open Source Edition as packaged up by Ubuntu

  3. mkdir -p ~/.VirtualBox&&cd ~/.VirtualBox
  4. install mbr –force MBR.mbrthis creates a new master boot record for you to use to boot your partition
  5. Refer to the disk partitioning layout and recall in the example above that /dev/sda, partition 1 (/dev/sda1) has my Windows XP install. Also note that /dev/sda has my original MBR so we will need to directly reference partition 1 to get access to Windows XP directly.Check your disk layout by doing the following

    fdisk -l /dev/sda

    Note normally you’d expect to prepend that with sudo (or be root) to view this, but this is a good check that your permissions are working as expected. If not – check you have set up your user permissions correctly and have logged out and back in again.
    Still not working after that? Do an ls -l /dev/sda and check the group that has write permissions on the partition. Add your username to that.

  6. IMPORTANT – UNMOUNT THE WINDOWS PARTITION FROM LINUXYou don’t want to have multiple OSes accessing the same device simultaneously. You have been warned!
  7. Create the disk file that VBox will use to access the partition:VBoxManage internalcommands createrawvmdk -filename ./WindowsXP.vmdk -rawdisk /dev/sda -partitions 1 -mbr ./MBR.mbr -relative -register


    This says create a file called WindowsXP.vmdk in the current dir, the disk is /dev/sda and the partition number is 1. We’re using the MBR file created in step number 4. The register option adds this into the Virtual Media Manager in VBox.user@localhost:~/.VirtualBox$ VBoxManage internalcommands createrawvmdk -filename ./WindowsXP.vmdk -rawdisk /dev/sda -partitions 1 -mbr ./MBR.mbr -relative -register
    VirtualBox Command Line Management Interface Version 3.0.4_OSE
    (C) 2005-2009 Sun Microsystems, Inc.
    All rights reserved.

    RAW host disk access VMDK file ./WindowsXP.vmdk created successfully.

  8. If you get permission denied, double check those user permissions. If you are getting access denied at this stage, there isn’t a chance that VBox will! So review from step 1 again…
  9. Fire up VirtualBox OSE and create a new Windows XP Virtual Machine, choosing the new hard disk you created in step 7
  10. Important: Now go back into the new Virtual Machine’s settings to enable IO-Apic


  11. Now boot it up!

Troubleshooting
I encountered a rather frustrating issue that prevented me booting up my Windows XP partition:

A disk read error occurred
Press Ctrl+Alt+Del to restart

To circumvent this, after lots of searching and reading I realised that the people that have successfully booted had something in common in their vmdk file. Edit the vmdk file you created in Step 7 and change

ddb.geometry.biosHeads=”255″ to ddb.geometry.biosHeads=”240

Save and watch proudly as Windows XP boots from a raw partition under Linux

Booting your native Windows XP – Keyboard and Mouse not working after installing Guest Additions: Uninstall Guest Additions.
Unfortunately I’ve not come across a nice solution to this. After booting into Windows XP from within Linux, after getting frustrated if the speed isn’t what you expect, there may be a reason to boot back into Windows XP natively. The problem you may come across though is that after installing Virtual Box Guest Additions, it replaced your mouse and keyboard drivers. This is great for running Windows XP under Linux, but not so great when that hardware disappears!
So – uninstalling Virtual Box is all I’ve come across so far. This seems correct as of version 3.0.4. Reports of it working with older (2.2.2) have been unconfirmed by me.

Thanks
After some googling the following helped me set up my raw Windows XP from within Linux
http://www.phyordia.com/ricardo/?p=370