The #OpenStack Cloud Computing Cookbook by @itarchitectkev and @cody_bunch

7587_CoverIt’s out! Cody and I managed to get the book out the door recently after a few months of hard slog upgrading the 1st Edition of the OpenStack Cloud Computing Cookbook to work with Grizzly. The 2nd Edition, published by Packt Publishing, can be found at the list of locations below – and being Amazon, check your local version for rates in your country:

Looking to purchase?
Amazon UK Print: http://amzn.to/1624lXL
Amazon UK Kindle: http://amzn.to/17aFAJx

Amazon US Print: http://amzn.to/1cB8bZi
Amazon US Kindle: http://amzn.to/162S84N

Available direct from publisher in many non-DRM eBook formats: http://bit.ly/1bi5I8M

You can read an interview about the book with Rackspace with Cody and myself here: http://bit.ly/1hfjVng

What is the OpenStack Cloud Computing Cookbook?

As I go through the process of editing the chapters based on the excellent feedback a team of technical reviewers have given us, there are certain themes that are standing out.  One more so is what exactly is the OpenStack Cloud Computing Cookbook, or what is a cookbook in general?

In a traditional setting, a cookbook is a set of cooking recipes on how to bake a cake, make a jacket potato or a lamb rogan josh for example.  These steps include an overview of what you’re about to make, the ingredients, any preparation required and the method by which to create this culinary masterpiece.

These recipes are usually grouped together with some common theme.  It could be starters, mains and desserts or lamb dishes, chicken dishes, etc.

A chef, or keen boyfriend wanting to impress is other half, would find the recipe that they want to follow and go from the beginning of that recipe for that scrumptious Tiramisu. Then voila, after following the 10 steps to create this masterpiece they have their dessert and eat it.

This is the exact same analogy with the OpenStack Cloud Computing Cookbook – or any technical cookbook series: a user locates the scenario they want instructions to follow and, step-by-step, they can recreate the end goal that is laid out for them.  This might be creating a new service role under the OpenStack Identity Service, or deleting SDN networks.

Optionally, and this is the crucial thing, the user can choose to follow from Page 1 to Page 300+ in an attempt to go from bare-metal to more advanced features – but it’s not within the bounds of a cookbook to assume that just because you want a starter, main and dessert that you necessarily have to have chosen a particular starter in order to make the main course more palatable.

What I tried to do with the 1st Edition was to arrange the chapters into a logical flow – the Starters, Mains and Desserts.  In the 2nd Edition we have the following chapters (we’re currently in an editing phase, so these reflect the lay of the land at the time of writing and are subject to change):

  • Chapter 1: Starting OpenStack Identity Service
  • Chapter 2: Starting OpenStack Image Service
  • Chapter 3: Starting OpenStack Compute Service
  • Chapter 4: Administering OpenStack Compute
  • Chapter 5: Starting OpenStack Storage
  • Chapter 6: Using OpenStack Storage
  • Chapter 7: Administering OpenStack Storage
  • Chapter 8: Starting OpenStack Volume Service
  • Chapter 9: Starting OpenStack Dashboard
  • Chapter 10: Using OpenStack Networking
  • Chapter 11: Automating OpenStack Installations
  • Chapter 12: Highly Available OpenStack
  • Chapter 13: Monitoring and Troubleshooting

As you can see there isn’t a chapter on how to plan an installation, what do I have to take into consideration, how to install “OpenStack” (rather than how do I install and configure Keystone, how do I install and configure Glance: these are recipes in the book), how does Glance interact with Keystone and Horizon… if you want a book on these, you need to download or purchase the excellent OpenStack Operations Guide or books that are on Mastering OpenStack.  Guides provide the extra words and reasoning behind OpenStack, cookbooks answer your questions on how to do something.  But the chapters are laid out in the typical order of services that are installed to get OpenStack up and running. One of my pet hates with some documentation is that we ensure continuity from one recipe and chapter to the next so that people wishing to consume from Page 1 to Page 300+ end up in the same place Cody and I expect you to be.

I hope this serves as a useful piece of information for anyone wishing to purchase the book and technical reviewers currently providing feedback to Cody and myself.

OpenStack Cloud Computing Cookbook, Second Edition – Updates

I live with first world problems: I’ve been too busy to blog because I’m too busy in work and busy co-authoring the new OpenStack Cloud Computing Cookbook that’s based on OpenStack Grizzly.

With the long tunnel beginning to shed some form of daylight, this second edition is on the final curve before the sprint.

To keep up to date, visit this website: http://www.openstackcookbook.com/

For the adventurous, we’ve been developing the supporting scripts for the book which can be found at https://github.com/uksysadmin/OpenStackCookbook. To use this – ensure you have the following:

  • An x86 PC with at least 8Gb Ram
  • VirtualBox installed (4.2 branch)
  • Vagrant Installed (Currently tested with 1.1.5)
  • Git client installed

Simply:

git clone https://github.com/uksysadmin/OpenStackCookbook.git
cd OpenStackCookbook
vagrant up

This gives you an OpenStack Grizzly release running on Ubuntu 12.04 LTS with Ubuntu Cloud Archive packages with Quantum. Spin it up and have a play!

Easy OpenStack Folsom with VirtualBox and Vagrant

Testing OpenStack is now as easy thanks to VirtualBox and Vagrant. To run a mini test environment with Compute, Cinder, Keystone and Horizon you just need the following tools:

  • VirtualBox
  • Vagrant
  • Git client

Getting Ready

To set up a sandbox environment within VirtualBox to run OpenStack Folsom you will need to download:

Installation of these tools are simple – follow the on-screen prompts.

When ready we need to configure the VirtualBox “Host-Only” Networking. This networking mode allows us to communicate with our VirtualBox guest and our underlying host.
We will set up the following:

  • Host-Only Network: IP 172.16.0.254; Network 172.16.0.0/255.255.0.0; Disable DHCP
  • Host-Only Network #2: IP 10.0.0.254; Network 10.0.0.0/255.0.0.0; Disable DHCP

(Hint: there is a bash script @ https://raw.github.com/uksysadmin/OpenStackInstaller/folsom/virtualbox/vbox-create-networks.sh to create these for you).

How To Do It

To create a VirtualBox VM, running Ubuntu 12.04 with OpenStack Folsom from Ubuntu’s Cloud Archive, carry out the following

1. Clone the GitHub OpenStackInstaller scripts

git clone https://github.com/uksysadmin/OpenStackInstaller.git

2. Make the scripts the ‘folsom’ branch

cd OpenStackInstaller
git checkout folsom

3. Run ‘vagrant’ to launch your OpenStack instance which will come up with IP 172.16.0.201

cd virtualbox
vagrant up

4. After a short while your instance will be ready. Note that on the first run, Vagrant will download a 384Mb Precise64 “box”. Subsequent launches will not require this step.

Launch a web browser at http://172.16.0.201/horizon and log in with:

Username: admin
Password: openstack

(Note, to edit the IP it is assigned, modify virtualbox/vagrant-openstack-bootstrap.sh (Warning its a bit of a sed hack!).

What to expect from the OpenStack Folsom Release

We are nearing the most complete and feature-rich OpenStack release to date, so given that we’ve all be running stable Diablo and Essex releases, what’s in Folsom that will get people reaching for their favorite package manager?

The OpenStack Folsom release is the community’s sixth release. With each release, the community gets larger and more exciting as more companies come on board and more developers commit their code; with each release comes greater stability and new features that allow us greater flexibility when deploying our cloud environments.

It’s hard to talk about OpenStack Folsom without mentioning two notable new projects: Quantum and Cinder. Quantum provides a feature-rich and extensible API for programmatically defining networks. This allows for far richer network topologies to be defined than currently available such as configuring multi-tier applications and the subnets they’re allocated. It also allows for more advanced configurations at the backend by utilizing plugins, such as utilizing the excellent Open vSwitch software or implementing QoS and security functions. Quantum gives users of OpenStack the ability to completely control all aspects of their cloud compute environment, without compromising the underlying infrastructure and security of the underlying OpenStack environment; Quantum brings true multi-tenancy without any restrictions of VLANs.

Cinder is the OpenStack Block Storage project that was formerly known as nova-volume. This re-birth into its own project allows this crucial piece of core functionality to be developed independently of Nova (Compute) itself. To a user, this means faster development, quicker bug fixes and newer features to meet the demands of today’s hosting requirements.

Other notable features included in Folsom are Microsoft’s hypervisor, Hyper-V, making a comeback after it was removed due to the code being unmaintained for the Essex release and the highly available OpenStack Storage, Swift, being ramped up to 1.7.2. For those upgrading Swift, read the advice here https://lists.launchpad.net/openstack/msg16188.html as this comes with a speed improvement to the rings that is backwards compatible – but downgrading at a later date using the new rings won’t be possible.

Put all the elements of Folsom together and you have a cloud software suite that allows you to easily scale out your secure compute environment and place highly available object storage within your datacenter. With a feature rich API fronting all aspects of an OpenStack controlled datacenter, management of this environment brings cost savings through efficiency and automation. Everything becomes code.

So how do you get the magical OpenStack Folsom installed onto the pristine that adorns your network? Over the last couple of months, there have been great strides in this area with the release of Rackspace’s Private Cloud Edition and Piston’s Airframe, both allowing swift installation of OpenStack without the overhead of some of the complexities previously experienced with an installation. Expect Folsom versions of easy to install methods soon. For those installing on Red Hat based and Ubuntu distros are left to stitch some of the required magic together, with some nuances to be mindful of when moving from a previous release – such as keystone and glance command line syntax changes. In fact, moving from a previous release isn’t as straight-forward as an apt-get upgrade, although with the Canonical starting to fulfil their promise of supporting future OpenStack releases on the latest LTS release via their Cloud Archive – this is quickly becoming a reality. Behind the scenes moving from one version to the next makes this process complex, so for those looking at moving from Diablo or Essex are best looking at migrating workloads to your new environment rather than expecting an hands-free upgrade at this stage. This “want” will become mandatory as OpenStack matures further to remove people from the endless conversations and confusions for those entering OpenStack and the sticking point being the version to accept as stable.

Essex laid the foundations for OpenStack production clouds, Folsom builds on this success and comes brandishing new weapons. With performance enhancements in Compute and Storage, Nova Volumes moving to its own Cinder project and Quantum added to its arsenal – OpenStack just made the choice easier when deciding on what cloud software you should deploy in your datacenter.

For more information on the Folsom release, take a look at Emilien Macchi’s deeper dive summary here: http://my1.fr/blog/whats-new-in-openstack-folsom/

Screencast / Video of an Install of OpenStack Essex on Ubuntu 12.04 under VirtualBox

A 12 minute screencast showing an installation of OpenStack ‘Essex’ on Ubuntu 12.04 running on VirtualBox

OpenStack Essex Installation Screencast

OpenStack Essex Installation Screencast

Note that this screencast has no sound.

  1. Configure VirtualBox with the following
    Network Interfaces:
    eth0 (nat)
    eth1 host-only: 172.16.0.0/16
    eth2 host-only: 10.0.0.0/8
    Memory: 1536Mb
    Hard Disk: 20Gb
    System Processor (optional but recommended): Increase CPU from 1
  2. Install Ubuntu 12.04, specifying eth0 as your default interface
  3. Configure networking:
    eth1 is your public network set to be 172.16.0.0/16
    eth2 is your private VLAN
  4. Run an update on the machine, and reboot
  5. Install Git which allows you to pull down a script to perform the installation of OpenStack
  6. Grab the script using the following:
    git clone https://github.com/uksysadmin/OpenStackInstaller.git
  7. Ensure you’re running the ‘essex’ version of the script by running: git checkout essex
  8. Run the script with the following:
    ./OSinstall.sh -F 172.16.1.0/24 -f 10.1.0.0/16 -s 512 -t demo -v qemu
  9. Upload an image using the test supplied script:
    ./upload_ubuntu.sh -a admin -p openstack -t demo -C 172.16.0.1
  10. Log into the Dashboard: http://172.16.0.1/ with ‘demo/openstack’
  11. Create your access keys
  12. Edit the default security group (add in SSH access and ability to ping)
  13. Launch your instance
  14. Log into your new instance!

OpenStack Cloud Computing Cookbook – Book Available for Pre-Order

I’m very pleased to announce that you can pre-order my book on OpenStack: OpenStack Cloud Computing Cookbook

http://www.packtpub.com/openstack-cloud-computing-cookbook/book

Understand, install and configure Nova, OpenStack’s Cloud Compute resource
Learn how to install Swift, how it operates—with practical recipes to troubleshooting and maintaining OpenStack’s Object Storage service
Configure Keystone, OpenStack’s Identity Service which underpins the authentication of all OpenStack services
Manage cloud computing images using the OpenStack Image Service, Glance
Learn how to create custom Windows and Linux images for use in your private cloud environment
Configure and install Horizon, the OpenStack Dashboard service for managing your cloud environment
Learn how to secure your private cloud and the instances running on them
Learn how to troubleshoot, monitor and deploy OpenStack environments beyond test environments and into real-world datacenters

Available around September 2012

Updated OpenStackInstaller script for Precise and Essex installs

I’ve updated the OpenStackInstaller script which now gives you a Development (Trunk) OpenStack Essex installation on Ubuntu Precise (Currently Alpha 2) with the following

Nova Compute (and associated services)
Keystone
Glance

This set up allows you to use nova client tools to launch instances

Install Ubuntu Precise
apt-get update
apt-get dist-upgrade
reboot

(as root)

  1. git clone https://github.com/uksysadmin/OpenStackInstaller.git
  2. cd OpenStackInstaller
  3. git checkout essex
  4. ./OSinstall.sh

A lot of this wouldn’t be possible without the help of people in #openstack on freenode.org.
For an equally awesome installation from scripts for a Diablo release view these scripts: https://github.com/managedit/openstack-setup

My response to “OpenStack is overstretched” – The Register

What do we get from writing an opinion online? Noteriety? Fame? Publicity?  I’m doing it because sometimes I need to fill in the blank space between the last post and this one.
I’ve just finished reading a Register article entitled “OpenStack is overstretched” in which a Enrico Signoretti writes that due to the popularity of OpenStack it seems like everyone needs to stick their awe in – good and bad – and the outcome is too many cooks.
Whilst I don’t question the burden of having to juggle a large number of vendors interests into a popular Open Source Cloud “Operating System” is a large one – isn’t this where Open Source projects shine? Sub groups work on particular code that may have a small group of people’s interests at heart. But The source code is available to do that right? What makes that feature get into the final cut? Does it matter so long as the core platform exists to serve the general public?
There is no doubt that OpenStack is a challenging product to use and implement and equally one that must keep the release manager busy and community as a whole – but their interests are to make a fantastic Open Source product available to all that want to take it in any direction they want.
We live in a world where we expect products to be able to have one-click installs – and in fact, a part of my job is to get to this stage in all aspects by automate everything. But sometimes it isn’t possible – but we can make it easier and certainly strive to get there. What we can do is to contribute back rather than sit back and watch a product mature enough to then add your own SKU that you can resell to keep your shareholders happy.
It sounds like people are frustrated because OpenStack has the potential to be big but its not ready to shine yet. There are companies using it, there are companies investigating it and there are companies that are contributing to it.

This is Open Source. You get the choice of what you want to do with it.