Category Archives: Linux

Oct 14 2011

Upgrade to Ubuntu 11.10 problem: Waiting for network configuration then black screen solution

Have you just upgraded to Ubuntu 11.10 Oneiric Ocelot and now getting the “Waiting for network configuration” message followed by “Waiting up to 60 seconds more for network”? This then might be accompanied by a black blank screen.

[update] I’ve updated this post to reflect the copy step mentioned in the bug post below is surplus as /run is mounted tmpfs – the refined steps are below. The fix is removing the old /var/run and /var/lock then pointing those old locations to /run and /run/lock respectively. I’m suspecting this bug only comes about after an upgrade from your existing session (e.g. apt-get dist-upgrade) where it must have trouble removing these directories because existing services have files needed in there.

[update 8th March 2012] Ubuntu 12.04 is just around the corner. I strongly advise you resist upgrading to 11.10 at this stage when 12.04 is to be released next month.

The bug is here (https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/858122) and the fix is based on this: https://bugs.launchpad.net/ubuntu/+source/dbus/+bug/811441/comments/24 :

  1. Hit Ctrl+Alt+F1 at the blank screen to get you to a non-X terminal (tty1)
  2. Login in with your username and password
  3. Change to root with: sudo -i and enter your password
  4. mkdir -p /run /run/lock
  5. rm -rf /var/run /var/lock
  6. ln -s /run /var
  7. ln -s /run/lock /var
  8. reboot

You should have 11.10 back again.

Sep 27 2011

OpenStack Diablo, updates and work in progress!

It has been a while since I blogged, and in that time OpenStack has come on leaps and bounds with Diablo being the latest official release. This will change as I work pretty much full-time on testing OpenStack as an end-user (and day job as architect) based on Diablo. This will also help with some book projects that are in the pipe-line for which I’m very humbled and excited about. I’ll blog my experiences as I go along – after all, it’s the reason you’ve stumbled upon this corner of the internet in the first place to learn from my experiences in using OpenStack.
The project I’m working on will be based on Ubuntu running the latest release of OpenStack, Diablo (2011.3). I’ll be investigating Crowbar from Dell to see how remote bare-metal provisioning of OpenStack is coming along – a crucial element for this to be adopted in established enterprises where it is the norm to roll-out enterprise class software in this way. I’ll try to squeeze in JuJu too. Most importantly though is playing catch up on the raft of projects that are flowing through OpenStack from Keystone for authentication, Quantum (although probably more relevant to Essex as this develops) as well as playing catch up on where Swift, Glance and the Dashboard are.

Mar 21 2011

Protecting SSH against brute force attacks

Running a public AWS instance is always asking for unexpected trouble from script kiddies and bots trying to find a vector in to compromise your server.
Sshguard (www.sshguard.net) monitors your log and alters your IPtables firewall accordingly to help keep persistent brute force attackers at bay.

1. Download the latest version from http://www.sshguard.net @ http://freshmeat.net/urls/6ff38f7dc039f95efec2859eefe17d3a

wget -O sshguard-1.5.tar.bz2

http://freshmeat.net/urls/6ff38f7dc039f95efec2859eefe17d3a

2. Unpack

tar jxvf sshguard-1.5.tar.bz2

3. Configure + Make

cd sshguard-1.5
./configure --with-firewall=iptables
make

4. Install (to /usr/local/sbin/sshguard)

sudo make install

5. /etc/init.d/sshguard (chmod 0755)

! /bin/sh
# this is a concept, elaborate to your taste
case $1 in
start)
/usr/local/sbin/sshguard -a 4 -b 5:/var/sshguard/blacklist.db -l
     /var/log/auth.log &
;;
stop)
killall sshguard
;;
*)
echo "Use start or stop"
exit 1
;;
esac

6. /etc/iptables.up.rules

# Firewall
*filter
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:INPUT DROP [0:0]
-N sshguard
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp --dport http -j ACCEPT
-A INPUT -p tcp --dport ftp-data -j ACCEPT
-A INPUT -p tcp --dport ftp -j ACCEPT
-A INPUT -p tcp --dport ssh -j sshguard
-A INPUT -p udp --source-port 53 -d 0/0 -j ACCEPT
-A OUTPUT -j ACCEPT
-A INPUT -j DROP
COMMIT
# Completed

7. Read in the IPtables rules

iptables-restore < /etc/iptables.up.rules

8. Start Sshguard

mkdir /var/sshguard&&/etc/init.d/sshguard start

Verification

tail -f /var/log/auth.log
iptables -L -n

Mar 3 2011

OpenStack Nova CentOS Instance

I’ve been working on tweaking a CentOS 5.3 image you can download from http://open.eucalyptus.com/wiki/EucalyptusUserImageCreatorGuide_v1.6 as there seems to be a big bias towards running Ubuntu under OpenStack. This is great for getting OpenStack up and running, but for us evangelists that operate a RHEL family house, its crucial to be able to demonstrate like-for-like offerings against what you currently run to help promote its use.

This guide should get you to a point where you have a usable, useful CentOS image for your environment. When I get around to it I’ll upload my version for use in your environment with the modifications laid out in this blog post.

The Guide

  • Start off by downloading a compatible image from Eucalyptus: http://open.eucalyptus.com/wiki/EucalyptusUserImageCreatorGuide_v1.6. I’ll work on the 64-Bit CentOS 5.3 image for this guide.
  • mkdir cloud/images and unpack the tarball here 
    • mkdir -p cloud/images
    • cd cloud/images
    • tar zxvf <path_to_tarball>/euca-centos-5.3-x86_64.tar.gz
    • cd euca-centos-5.3-x86_64
    • At this stage we’d normally upload the image to OpenStack but some modifications are needed such as increasing the size of the image to accommodate some new packages so we must first mount the image (read-only because we’re not needing to make edits to this yet) as follows 
      • mkdir image
      • sudo mount centos.5-3.x86-64.img image -o loop,ro
    • Increase the size of the image as follows and copy the contents 
      • dd if=/dev/zero of=newcentos.img bs=1M count=2048
      • mkfs.ext3 newcentos.img
      • mkdir newcentos
      • sudo mount newcentos.img newcentos -o loop,rw
      • sudo cp -pR image/* newcentos/
      • sudo umount image
  • Modify the image as follows
  • IMPORTANT! (ensure you’re chrooted as described below to your mounted image and you have verified that you’re not modifying your running environment – I accept no responsibility because you can’t read) 
    • sudo su -
    • chroot ~/cloud/images/euca-centos-5.3-x86_64/newcentos
    • mount proc -t proc /proc
  • Now to modify the image and install some new packages… 
    • yum update
    • yum install redhat-lsb sudo enhanced-vim
    • Remove /etc/udev/rules.d/* to stop the lengthy wait on boot
    • edit /etc/sysconfig/network and disable ZEROCONF (your instance will fail to download meta data from OpenStack nova-api otherwise) 
      • NOZEROCONF=yes
    • Edit /etc/profile.d/vim.sh 
      • if [ -n "$BASH_VERSION" -o -n "$KSH_VERSION" -o -n "$ZSH_VERSION" ]
then
[ -x /usr/bin/id ] || return
tmpid=$(/usr/bin/id -u)
[ "$tmpid" = "" ] && tmpid=0
# for bash and zsh, only if no alias is already set
alias vi >/dev/null 2>&1 || alias vi=vim
alias view >/dev/null 2>&1 || alias view='vim -R'
fi
    • Ensure /dev/null is writeable by all 
      • chmod 777 /dev/null
  • That’s the modifications done, but feel free to add your own to suit your own environment so to wrap it up 
    • umount /proc
    • logout
    • logout
    • sudo umount newcentos
    • To make things neat rename it appropriately 
      • mv newcentos.img centos-5.5-x86_64.img

Upload CentOS image to OpenStack

  • Now you have a CentOS image suitable for OpenStack you need to upload it to OpenStack.
  • The tarball ships with 2 lots of kernels and ramdisks. I’ll assume you’ll be using KVM, but change the instructions to suit a Xen hypervisor.
    • Upload the kernel and make note of the ami 
      • euca-bundle-image -i kvm-kernel/vmlinuz-2.6.28-11-generic
--kernel true
      • euca-upload-bundle -b mybucket
-m /tmp/vmlinuz-2.6.28-11-generic.manifest.xml
      • euca-register mybucket/vmlinuz-2.6.28-11-generic.manifest.xml
    • Upload the ramdisk and make a note of the ami 
      • euca-bundle-image -i kvm-kernel/initrd.img-2.6.28-11-generic
--ramdisk true
      • euca-upload-bundle -b mybucket
-m /tmp/initrd.img-2.6.28-11-generic.manifest.xml
      • euca-register mybucket/initrd.img-2.6.28-11-generic.manifest.xml
    • Upload the machine image you modifed above, specifying the ami values from the steps above to specify the kernel and ramdisk to load with this 
      • euca-bundle-image -i centos-5.5-x86_64.img
--kernel aki-XXXXXXXX --ramdisk ari-XXXXXXXX
      • euca-upload-bundle -b mybucket
-m /tmp/centos-5.5-x86_64.img.manifest.xml
      • euca-register mybucket/centos-5.5-x86_64.img.manifest.xml
  • That’s it done (you may have to wait a short while whilst it uploads to the nova-objectstore server) – you should now see your new AMI available 
    • euca-describe-images
      • IMAGE    ami-reey5wk5    mybucket/centos.5-5.x86-64.img.manifest.xml   
myproject    available    private        x86_64    machine    ami-f4ks8moj   
ami-jqxvgtmd
  • You can now use this to launch an instance 
    • euca-run-instances ami-reey5wk5 -k openstack -t m1.tiny
Nov 9 2010

Apache, FancyIndexing and PHP 5 (mod_autoindex)

Introduction
The default Directory Listing in Apache is pretty much awful, but I had a need to present some files through a web browser. Rather than produce something with PHP alone I decided to enhance the Apache FancyIndexing option as it is designed for exactly this purpose.
I came across a nice PHP enhancement (update to include link and credit) to the FancyIndexing that added guided navigation to the directory listing, as well as improve the default font and general styling thanks to effective use of CSS.

Instructions

1. Edit httpd.conf and add or modify the following

AccessFileName .htaccess
<Files ~ “^\.ht”>
Order allow,deny
Deny from all
</Files>

<Directory /your/directory>
AllowOverride all
</Directory>

2. In the directory you want the listing of add the following .htaccess file

Options +Indexes +FollowSymlinks
IndexOptions FancyIndexing HTMLTable FoldersFirst SuppressRules SuppressDescription SuppressHTMLPreamble Charset=UTF-8
#
# AddIcon* directives tell the server which icon to show for different# files or filename extensions.  These are only displayed for
# FancyIndexed directories.
#
AddIcon /autoindex/icons/application.png .exe .app
AddIcon /autoindex/icons/type_binary.png .bin .hqx .uu
AddIcon /autoindex/icons/type_box.png .tar .tgz .tbz .tbz2 bundle .rar
AddIcon /autoindex/icons/type_code.png .html .htm .htx .htmls .dhtml .phtml .shtml .inc .ssi .c .cc .css .h .rb .js .rb .pl .py .sh .shar .csh .ksh .tcl .as
AddIcon /autoindex/icons/type_database.png .db .sqlite .dat
AddIcon /autoindex/icons/type_disc.png .iso .image
AddIcon /autoindex/icons/type_document.png .ttf
AddIcon /autoindex/icons/type_excel.png .xlsx .xls .xlm .xlt .xla .xlb .xld .xlk .xll .xlv .xlw
AddIcon /autoindex/icons/type_flash.png .flv
AddIcon /autoindex/icons/type_illustrator.png .ai .eps .epsf .epsi
AddIcon /autoindex/icons/type_pdf.png .pdf
AddIcon /autoindex/icons/type_php.png .php .phps .php5 .php3 .php4 .phtm
AddIcon /autoindex/icons/type_photoshop.png .psd
AddIcon /autoindex/icons/monitor.png .ps
AddIcon /autoindex/icons/type_powerpoint.png .ppt .pptx .ppz .pot .pwz .ppa .pps .pow
AddIcon /autoindex/icons/type_swf.png .swf
AddIcon /autoindex/icons/type_text.png .tex .dvi
AddIcon /autoindex/icons/type_vcf.png .vcf .vcard
AddIcon /autoindex/icons/type_word.png .doc .docx
AddIcon /autoindex/icons/type_zip.png .Z .z .tgz .gz .zip
AddIcon /autoindex/icons/globe.png .wrl .wrl.gz .vrm .vrml .iv
AddIcon /autoindex/icons/vector.png .plot

AddIconByType (TXT,/autoindex/icons/type_text.png) text/*
AddIconByType (IMG,/autoindex/icons/type_image.png) image/*
AddIconByType (SND,/autoindex/icons/type_audio.png) audio/*
AddIconByType (VID,/autoindex/icons/type_video.png) video/*
AddIconByEncoding (CMP,/autoindex/icons/type_box.png) x-compress x-gzip
AddIcon /autoindex/icons/back.png ..
AddIcon /autoindex/icons/information.png README INSTALL
AddIcon /autoindex/icons/type_folder.png ^^DIRECTORY^^
AddIcon /autoindex/icons/blank.png ^^BLANKICON^^

#
# DefaultIcon is which icon to show for files which do not have an icon# explicitly set.
#
DefaultIcon /autoindex/icons/type_document.png
#
# Enables PHP to be used in our header file
# 
AddHandler application/x-httpd-php .php
AddType text/html .php .html
#
# ReadmeName is the name of the README file the server will look for by
# default, and append to directory listings.
#
# HeaderName is the name of a file which should be prepended to
# directory indexes.
ReadmeName /autoindex/footer.php
HeaderName /autoindex/header.php
#
# IndexIgnore is a set of filenames which directory indexing should ignore
# and not include in the listing.  Shell-style wildcarding is permitted.
#
IndexIgnore autoindex .??* *~ *# RCS CVS *,v *,t *.dat ..

IndexOptions +NameWidth=42
AddDescription "PNG images" *.png

Warning for PHP 5.3 and higher

I originally had this running with PHP 5.1 and it was working great.  I upgraded to PHP 5.3.3 (latest at the time of writing) and it refused to parse the PHP, despite the PHP working if I called the Header and Footer PHP pages directly.

It turned out to be the directive XHTML in the IndexOptions line.  Remove this and it will parse.  XHTML says:

The XHTML keyword forces mod_autoindex to emit XHTML 1.0 code instead of HTML 3.2.

Whereas the same pages says that a Header/Readme filename “must resolve to a document with a major content type of text/* (e.g., text/html, text/plain, etc.).”

Nov 9 2010

Building Apache 2.2, PHP 5 with GD and MySQLi support from source

1. Download the following
Apache 2.2 from http://httpd.apache.org/download.cgi [2.2.17]
PHP 5.3.3 from http://www.php.net/downloads.php [5.3.3]
Expat from http://sourceforge.net/projects/expat/ [2.0.1]
JPEG from http://www.ijg.org/ [v8b]
PNG from http://sourceforge.net/projects/libpng/files/ [1.4.4]

2. Apache

./configure --enable-so --enable-modules=most --enable-proxy --with-mpm=worker --disable-imap --enable-deflate
make
sudo make install

3. Expat XML Parser

./configure
make
sudo make install

4 JPEG

./configure
make
sudo make install

5. PNG

./configure
make
sudo make install

6. PHP

./configure --disable-cli --enable-embedded-mysqli --with-zlib --enable-shared --with-apxs2=/usr/local/apache2/bin/apxs --with-gd
make
sudo make install
Apr 28 2010

VirtualBox Windows Host, Shared Folders and Linux Guest

VirtualBox comes with a feature to give you access to the hosts’s filesystem through a CIFS server.  This is available after you’ve installed the Guest Additions drivers in the Guest.  More details are available here (specific to Ubuntu but can apply to all versions of Linux): https://help.ubuntu.com/community/VirtualBox

There are more details at that page on how to set up the share.

Frustratingly, the CIFS (Shared Folder) server name you access a hosts file system on a Linux Guest is different to one that you use under Windows so as a quick overview

Windows Guest

net use x: vboxsvrshare

Linux Guest

mount -t vboxsf share mountpoint

Note the change from vboxsvr under Windows to vboxsf under Linux.