Oct 25 2013

The #OpenStack Cloud Computing Cookbook by @itarchitectkev and @cody_bunch

7587_CoverIt’s out! Cody and I managed to get the book out the door recently after a few months of hard slog upgrading the 1st Edition of the OpenStack Cloud Computing Cookbook to work with Grizzly. The 2nd Edition, published by Packt Publishing, can be found at the list of locations below – and being Amazon, check your local version for rates in your country:

Looking to purchase?
Amazon UK Print: http://amzn.to/1624lXL
Amazon UK Kindle: http://amzn.to/17aFAJx

Amazon US Print: http://amzn.to/1cB8bZi
Amazon US Kindle: http://amzn.to/162S84N

Available direct from publisher in many non-DRM eBook formats: http://bit.ly/1bi5I8M

You can read an interview about the book with Rackspace with Cody and myself here: http://bit.ly/1hfjVng

Jun 23 2013

What is the OpenStack Cloud Computing Cookbook?

As I go through the process of editing the chapters based on the excellent feedback a team of technical reviewers have given us, there are certain themes that are standing out.  One more so is what exactly is the OpenStack Cloud Computing Cookbook, or what is a cookbook in general?

In a traditional setting, a cookbook is a set of cooking recipes on how to bake a cake, make a jacket potato or a lamb rogan josh for example.  These steps include an overview of what you’re about to make, the ingredients, any preparation required and the method by which to create this culinary masterpiece.

These recipes are usually grouped together with some common theme.  It could be starters, mains and desserts or lamb dishes, chicken dishes, etc.

A chef, or keen boyfriend wanting to impress is other half, would find the recipe that they want to follow and go from the beginning of that recipe for that scrumptious Tiramisu. Then voila, after following the 10 steps to create this masterpiece they have their dessert and eat it.

This is the exact same analogy with the OpenStack Cloud Computing Cookbook – or any technical cookbook series: a user locates the scenario they want instructions to follow and, step-by-step, they can recreate the end goal that is laid out for them.  This might be creating a new service role under the OpenStack Identity Service, or deleting SDN networks.

Optionally, and this is the crucial thing, the user can choose to follow from Page 1 to Page 300+ in an attempt to go from bare-metal to more advanced features – but it’s not within the bounds of a cookbook to assume that just because you want a starter, main and dessert that you necessarily have to have chosen a particular starter in order to make the main course more palatable.

What I tried to do with the 1st Edition was to arrange the chapters into a logical flow – the Starters, Mains and Desserts.  In the 2nd Edition we have the following chapters (we’re currently in an editing phase, so these reflect the lay of the land at the time of writing and are subject to change):

  • Chapter 1: Starting OpenStack Identity Service
  • Chapter 2: Starting OpenStack Image Service
  • Chapter 3: Starting OpenStack Compute Service
  • Chapter 4: Administering OpenStack Compute
  • Chapter 5: Starting OpenStack Storage
  • Chapter 6: Using OpenStack Storage
  • Chapter 7: Administering OpenStack Storage
  • Chapter 8: Starting OpenStack Volume Service
  • Chapter 9: Starting OpenStack Dashboard
  • Chapter 10: Using OpenStack Networking
  • Chapter 11: Automating OpenStack Installations
  • Chapter 12: Highly Available OpenStack
  • Chapter 13: Monitoring and Troubleshooting

As you can see there isn’t a chapter on how to plan an installation, what do I have to take into consideration, how to install “OpenStack” (rather than how do I install and configure Keystone, how do I install and configure Glance: these are recipes in the book), how does Glance interact with Keystone and Horizon… if you want a book on these, you need to download or purchase the excellent OpenStack Operations Guide or books that are on Mastering OpenStack.  Guides provide the extra words and reasoning behind OpenStack, cookbooks answer your questions on how to do something.  But the chapters are laid out in the typical order of services that are installed to get OpenStack up and running. One of my pet hates with some documentation is that we ensure continuity from one recipe and chapter to the next so that people wishing to consume from Page 1 to Page 300+ end up in the same place Cody and I expect you to be.

I hope this serves as a useful piece of information for anyone wishing to purchase the book and technical reviewers currently providing feedback to Cody and myself.

Jun 21 2013

OpenStack Cloud Computing Cookbook, Second Edition – Updates

I live with first world problems: I’ve been too busy to blog because I’m too busy in work and busy co-authoring the new OpenStack Cloud Computing Cookbook that’s based on OpenStack Grizzly.

With the long tunnel beginning to shed some form of daylight, this second edition is on the final curve before the sprint.

To keep up to date, visit this website: http://www.openstackcookbook.com/

For the adventurous, we’ve been developing the supporting scripts for the book which can be found at https://github.com/uksysadmin/OpenStackCookbook. To use this – ensure you have the following:

  • An x86 PC with at least 8Gb Ram
  • VirtualBox installed (4.2 branch)
  • Vagrant Installed (Currently tested with 1.1.5)
  • Git client installed

Simply:

git clone https://github.com/uksysadmin/OpenStackCookbook.git
cd OpenStackCookbook
vagrant up

This gives you an OpenStack Grizzly release running on Ubuntu 12.04 LTS with Ubuntu Cloud Archive packages with Quantum. Spin it up and have a play!

Oct 30 2012

Upgrade Ubuntu 12.04 Precise to 12.10 Quantal causes black screen

Disappointingly, I’ve just upgraded my 12.04 desktop to Ubuntu 12.10 Quantal release and the upgrade was not smooth. X failed to run. Given I use ATI’s proprietary driver, I assumed this was the issue. Unfortunately this didn’t seem to be the case as running the “Failsafe X” resulted in a black screen.

Recalling the pain when upgrading from 11.04 to 11.10 (https://uksysadmin.wordpress.com/2011/10/14/upgrade-to-ubuntu-11-10-problem-waiting-for-network-configuration-then-black-screen-solution/) and getting similar symptoms (although those issues related to networking) I ran through the same steps and I had X back again. The steps I did where:

  1. Boot into a safe session by selected Advanced Ubuntu Options from the Grub menu then choosing recovery mode
  2. Drop to a root shell
  3. rm -rf /var/run /var/lock
  4. ln -s /run /var
  5. ln -s /run/lock /var
  6. reboot

I then had X back and lightdm working. I then went to log in, but Unity didn’t seem to work (I still need to troubleshoot and will update here). My current work around was to install Gnome.

  1. Get to a console (try CTRL+ALT F1, although this mightn’t work – reboot into a root shell again)
  2. (If you don’t have networking, as the root user run the following: dhclient eth0)
  3. apt-get update
  4. apt-get install gnome
  5. reboot or restart lightdm (service lightdm restart)
  6. Under your username, choose the Ubuntu symbol above the > symbol to choose window managers and select Gnome
  7. Log in

To get Unity back:

  1. Fire up a console
  2. sudo apt-get remove –purge unity
  3. sudo apt-get install unity ubuntu-desktop

(thanks @EwanToo!)

I have no option to install proprietary hardware now – so I ‘ll see how my ATI FireGL card performs under the GPL driver…

Oct 9 2012

Easy OpenStack Folsom with VirtualBox and Vagrant

Testing OpenStack is now as easy thanks to VirtualBox and Vagrant. To run a mini test environment with Compute, Cinder, Keystone and Horizon you just need the following tools:

  • VirtualBox
  • Vagrant
  • Git client

Getting Ready

To set up a sandbox environment within VirtualBox to run OpenStack Folsom you will need to download:

Installation of these tools are simple – follow the on-screen prompts.

When ready we need to configure the VirtualBox “Host-Only” Networking. This networking mode allows us to communicate with our VirtualBox guest and our underlying host.
We will set up the following:

  • Host-Only Network: IP 172.16.0.254; Network 172.16.0.0/255.255.0.0; Disable DHCP
  • Host-Only Network #2: IP 10.0.0.254; Network 10.0.0.0/255.0.0.0; Disable DHCP

(Hint: there is a bash script @ https://raw.github.com/uksysadmin/OpenStackInstaller/folsom/virtualbox/vbox-create-networks.sh to create these for you).

How To Do It

To create a VirtualBox VM, running Ubuntu 12.04 with OpenStack Folsom from Ubuntu’s Cloud Archive, carry out the following

1. Clone the GitHub OpenStackInstaller scripts

git clone https://github.com/uksysadmin/OpenStackInstaller.git

2. Make the scripts the ‘folsom’ branch

cd OpenStackInstaller
git checkout folsom

3. Run ‘vagrant’ to launch your OpenStack instance which will come up with IP 172.16.0.201

cd virtualbox
vagrant up

4. After a short while your instance will be ready. Note that on the first run, Vagrant will download a 384Mb Precise64 “box”. Subsequent launches will not require this step.

Launch a web browser at http://172.16.0.201/horizon and log in with:

Username: admin
Password: openstack

(Note, to edit the IP it is assigned, modify virtualbox/vagrant-openstack-bootstrap.sh (Warning its a bit of a sed hack!).

Sep 23 2012

What to expect from the OpenStack Folsom Release

We are nearing the most complete and feature-rich OpenStack release to date, so given that we’ve all be running stable Diablo and Essex releases, what’s in Folsom that will get people reaching for their favorite package manager?

The OpenStack Folsom release is the community’s sixth release. With each release, the community gets larger and more exciting as more companies come on board and more developers commit their code; with each release comes greater stability and new features that allow us greater flexibility when deploying our cloud environments.

It’s hard to talk about OpenStack Folsom without mentioning two notable new projects: Quantum and Cinder. Quantum provides a feature-rich and extensible API for programmatically defining networks. This allows for far richer network topologies to be defined than currently available such as configuring multi-tier applications and the subnets they’re allocated. It also allows for more advanced configurations at the backend by utilizing plugins, such as utilizing the excellent Open vSwitch software or implementing QoS and security functions. Quantum gives users of OpenStack the ability to completely control all aspects of their cloud compute environment, without compromising the underlying infrastructure and security of the underlying OpenStack environment; Quantum brings true multi-tenancy without any restrictions of VLANs.

Cinder is the OpenStack Block Storage project that was formerly known as nova-volume. This re-birth into its own project allows this crucial piece of core functionality to be developed independently of Nova (Compute) itself. To a user, this means faster development, quicker bug fixes and newer features to meet the demands of today’s hosting requirements.

Other notable features included in Folsom are Microsoft’s hypervisor, Hyper-V, making a comeback after it was removed due to the code being unmaintained for the Essex release and the highly available OpenStack Storage, Swift, being ramped up to 1.7.2. For those upgrading Swift, read the advice here https://lists.launchpad.net/openstack/msg16188.html as this comes with a speed improvement to the rings that is backwards compatible – but downgrading at a later date using the new rings won’t be possible.

Put all the elements of Folsom together and you have a cloud software suite that allows you to easily scale out your secure compute environment and place highly available object storage within your datacenter. With a feature rich API fronting all aspects of an OpenStack controlled datacenter, management of this environment brings cost savings through efficiency and automation. Everything becomes code.

So how do you get the magical OpenStack Folsom installed onto the pristine that adorns your network? Over the last couple of months, there have been great strides in this area with the release of Rackspace’s Private Cloud Edition and Piston’s Airframe, both allowing swift installation of OpenStack without the overhead of some of the complexities previously experienced with an installation. Expect Folsom versions of easy to install methods soon. For those installing on Red Hat based and Ubuntu distros are left to stitch some of the required magic together, with some nuances to be mindful of when moving from a previous release – such as keystone and glance command line syntax changes. In fact, moving from a previous release isn’t as straight-forward as an apt-get upgrade, although with the Canonical starting to fulfil their promise of supporting future OpenStack releases on the latest LTS release via their Cloud Archive – this is quickly becoming a reality. Behind the scenes moving from one version to the next makes this process complex, so for those looking at moving from Diablo or Essex are best looking at migrating workloads to your new environment rather than expecting an hands-free upgrade at this stage. This “want” will become mandatory as OpenStack matures further to remove people from the endless conversations and confusions for those entering OpenStack and the sticking point being the version to accept as stable.

Essex laid the foundations for OpenStack production clouds, Folsom builds on this success and comes brandishing new weapons. With performance enhancements in Compute and Storage, Nova Volumes moving to its own Cinder project and Quantum added to its arsenal – OpenStack just made the choice easier when deciding on what cloud software you should deploy in your datacenter.

For more information on the Folsom release, take a look at Emilien Macchi’s deeper dive summary here: http://my1.fr/blog/whats-new-in-openstack-folsom/

Aug 21 2012

#OpenStack Cloud Computing Cookbook: Table Of Contents

I’m nearing the end of my latest project – my book: OpenStack Cloud Computing Cookbook. A few more cuts here and there and it will be released to the general public in September. This is predominantly the cause of the 4 month blog update starvation.

About the book

The book is designed specifically to help people get up to speed with OpenStack and give you the confidence and understanding to roll this out into your own datacenters. The book covers installation of OpenStack in a sandbox environment on local equipment running VirtualBox to recipes covering topics that allow you to move out to the datacenter, and sections covering some of the tools and techniques to make for a robust installation of OpenStack in your production environment.

The book will be out shortly and available from Packtpub, Amazon, Amazon UK, (and on Kindle), and Barnes & Noble.

The recipes

So a sneaky peak at the recipes to tease you until publish time:

  • Creating a sandbox environment with VirtualBox
  • Installing OpenStack Compute Packages
  • Configuring Database Services
  • Configuring OpenStack Compute
  • Stopping and starting OpenStack Compute services
  • Creating a cloudadmin account and project
  • Installation of command line tools
  • Checking OpenStack Compute services
  • Uploading a sample machine image
  • Launching your first cloud instance
  • Terminating your instance
  • Creating and modifying user accounts
  • Managing security groups
  • Creating and managing keypairs
  • Using Public Cloud Images
  • Alternative upload method using euca2ools
  • Creating Custom Windows Images
  • Creating Custom CentOS Images
  • Installing the OpenStack Identity Service
  • Creating tenants
  • Creating roles
  • Adding users
  • Defining service endpoints
  • Creating the service tenant and service users
  • Configuring OpenStack Image Service
  • Configuring OpenStack Compute
  • Using OpenStack Compute with the OpenStack Identity Service
  • Creating an OpenStack Storage Sandbox Environment
  • Installing OpenStack Storage
  • Configuring Storage
  • Configuring Replication
  • Configuring OpenStack Storage Service
  • Configuring OpenStack Storage Proxy Server
  • Configuring Account Server
  • Configuring Container Server
  • Configuring Object Server
  • Making the rings
  • Stopping and starting OpenStack Storage
  • Testing OpenStack Storage
  • Installing the swift client tool
  • Creating containers
  • Uploading objects
  • Uploading large objects
  • Listing containers and objects
  • Downloading objects
  • Deleting containers and objects
  • Using Swift ACLs
  • Preparing drives for OpenStack Storage
  • Managing OpenStack Storage cluster with swift-­init
  • Checking cluster health
  • OpenStack Storage Benchmarking
  • Managing capacity
  • Removing nodes from a cluster
  • Replacing failed hard drives
  • Collecting usage statistics
  • Installing OpenStack Image Service
  • Configuring OpenStack Image Service with MySQL
  • Configuring OpenStack Image Service with OpenStack Storage
  • Configuring OpenStack Compute with OpenStack Image Service
  • Managing Images
  • Registering a Remotely Stored Image
  • Configuring Nova-­Volume Services
  • Configure OpenStack Compute for Nova-­Volume
  • Creating Volumes
  • Attaching volumes to instances
  • Detaching volumes from an instance
  • Deleting volumes
  • Installing the OpenStack Dashboard
  • Key Management using the OpenStack Dashboard
  • Security Group management using the OpenStack Dashboard
  • Launching instances using the OpenStack Dashboard
  • Terminating instances using the OpenStack Dashboard
  • Connecting to instances using the OpenStack Dashboard and VNC
  • Adding new tenants using the OpenStack Dashboard
  • User management using the OpenStack Dashboard
  • Configuring Flat Networking
  • Configuring Flat Networking with DHCP
  • Configuring VLAN Networking
  • Configuring Per-­Project (Tenant) IP ranges
  • Automatically assigning fixed networks to tenants
  • Modifying a tenant’s fixed network
  • Manually associating floating IPs to instances
  • Manually removing floating IPs from instances
  • Automatically assigning floating IPs
  • Installing MAAS for bare-­metal provisioning
  • Using MAAS for bare-­metal provisioning
  • Installing and configuring Juju
  • Installing OpenStack Services using Juju
  • Increasing OpenStack Compute capacity
  • MySQL clustering using Galera
  • Configuring HA Proxy for MySQL Galera Load Balancing
  • Configuring highly-­available OpenStack services
  • Bonding Network Interfaces for Redundancy
  • Monitoring Compute Services with Munin
  • Monitoring Instances using Munin and collectd
  • Monitoring Storage Service using StatsD/Graphite
  • Monitoring MySQL with Hyperic
  • Checking OpenStack Compute Services
  • Understanding logging
  • Troubleshooting OpenStack Compute services
  • Troubleshooting OpenStack Storage
  • Troubleshooting OpenStack Authentication
  • Submitting bugs
  • Asking help from the community

Where possible I explain both Euca2ools and Nova Client – so whichever floats your boat, the recipe will apply to you where applicable!

I’ll save all my thanks for those that have donated their time in helping me write this when the book is out. In the meantime – a big thank you to everyone in the OpenStack community that has helped me in some shape or form!