movw $0x1f01,0xb8000; hlt;

When booting a Linux based computer from a hibernate image, the process is broken down
into three distinct phases:

1. the bootloader loads the kernel image
2. the (first) loaded kernel reads the hibernate image on disk and jumps into a saved kernel
3. the second kernel, saved in the hibernate image, resets the saved state of the OS and resumes its operations

As an experiment, I wanted to see if it was possible to skip the second step and have the bootloader read and jump into the saved kernel on disk directly.

This has been implemented in resume.c32, a Syslinux COM32 module which can read and load a TuxOnIce image. A small patch for TuxOnIce is needed, see my TuxOnIce repository on github.

Although the experiment has been successful, it is not clear whether this can be used in practice.

First of all, reading the hibernate image is quite I/O expensive and can be done much faster using a real kernel vs using native I/O BIOS routines.

Second, some extra work may be needed in the second phase above for the hardware to be in a safe state before attempting to resume (reset registers, …).
resume.c32 doesn’t do any of that work and although it works in a stock Debian install in Qemu, it is not guaranteed to be stable using real hardware.

To see it in action: http://www.youtube.com/watch?v=TF9nUVkHCrQ.

Code is on Github. All the magic happens in resume_trampoline_asm.S.

I have just released version 1.0.0 of hfind on github.

hfind is a find utility for Hadoop. It implements most of the POSIX specification, so you should be able to use it pretty much as you use find(1) (although most of the superset primaries added by the GNU or BSD versions are not yet implemented).

Here is for instance a Bash snippet I use to cleanup some of my directories on HDFS:

#!/bin/bash -e

LOG=$HOME/log/hadoop-cleanup.$(date "+%Y%m%d")
HADOOP=$HOME/bin/hadoop
HFIND=$HOME/bin/hfind

cleanup_directory() {
    local dir=$1
    local cutoff=$2

    $HFIND $dir -type f -mtime -$cutoff | xargs $HADOOP fs -rm >> $LOG
    $HFIND $dir -mtime -$cutoff -empty | xargs $HADOOP fs -rmr >> $LOG
}

# Keep the last 7 days of test data
cleanup_directory /user/pierre/test 7

# Keep the last 6 months of rollups
cleanup_directory /user/pierre/rollups 186

The following primaries, some originally requested in HDFS-227, have been implemented:

• -type [f|d]
• -atime and -mtime, support both + and – arguments
• -depth n and -d
• -owner/-group/-nouser/-nogroup
• -name, which supports globing and regex
• -size
• -empty

The following ones have been scheduled for the next release:

• -print0 (for piping to xargs -0)
• -perm
• -delete
• parens support

On the last point, -a and -o have been implemented though, i.e. you can do

find / -nouser -o -nogroup -a -name test.dat

but not yet

find / \( -nouser -o -nogroup \) -a -name test.dat

Note that -delete is not implemented yet. I first want to get more feedback and testing. This should be fixed by 1.0.1.

To get started on hfind, read the main page at http://github.com/pierre/hfind.

The direct download link for the 1.0.0 release is http://github.com/downloads/pierre/hfind/metrics.hfind-1.0.0.tar.gz.

Happy finding!

Bash trivia: what’s wrong with the code below?

date='2010-09-01'

year=$((`echo $date | cut -d- -f1`))
month=$((`echo $date | cut -d- -f2`))
day=$((`echo $date | cut -d- -f3`))

echo "Today is $month/$day/$year"

If you run it, you’ll have the following error:

pierre > bash trivia.sh
trivia.sh: line 4: 09: value too great for base (error token is "09")
Today is /1/2010

day is “09″, the preceding 0 makes Bash interpret it as an octal (see the chapter Numerical Constants in the Advance Bash-Scripting guide).

You can change the base by using the base#number notation:

date='2010-09-01'

year=$((10#`echo $date | cut -d- -f1`))
month=$((10#`echo $date | cut -d- -f2`))
day=$((10#`echo $date | cut -d- -f3`))

echo "Today is $month/$day/$year"

pierre > bash trivia.sh
Today is 9/1/2010