Adding a physical disk as RAID 0 using hpacucli in Debian GNU/Linux and HP DL380 G7

This webpage shows hpacucli in Linux.

To install hpacucli in Debian GNU/Linux you may add the HP's repository for the distro in /etc/apt/sources.list:

deb http://downloads.linux.hp.com/SDR/downloads/ProLiantSupportPack/ lucid current/non-free

Yes, I know it means Ubuntu, but it worked for me.

apt-get update

apt-get install hpacucli

Run hpacucli:

# hpacucli

HP Array Configuration Utility CLI 8.60-8.0

Detecting Controllers...Done.

Type "help" for a list of supported commands.

Type "exit" to close the console.

=>

The current situation:

=> ctrl all show config

Smart Array P410i in Slot 0 (Embedded)    (sn: 50014380102A5090)

   array A (SAS, Unused Space: 0 MB)

      logicaldrive 1 (558.9 GB, RAID 0, OK)

      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 600 GB, OK)

   unassigned

      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 600 GB, OK)
      physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS, 600 GB, OK)
      physicaldrive 1I:1:4 (port 1I:box 1:bay 4, SAS, 600 GB, OK)
      physicaldrive 2I:1:5 (port 2I:box 1:bay 5, SAS, 600 GB, OK)
      physicaldrive 2I:1:6 (port 2I:box 1:bay 6, SAS, 600 GB, OK)

   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 250 (WWID: 50014380102A509F)

=>

I need each disk installed individualy, not mixed in arrays.

To add one physical disk as RAID 0:

=> ctrl slot=0 create type=ld drives=1I:1:2 raid=0

Then, the result is what I expected:

=> ctrl all show config                           

Smart Array P410i in Slot 0 (Embedded)    (sn: 50014380102A5090)

   array A (SAS, Unused Space: 0 MB)

      logicaldrive 1 (558.9 GB, RAID 0, OK)

      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 600 GB, OK)

   array B (SAS, Unused Space: 0 MB)


      logicaldrive 2 (558.9 GB, RAID 0, OK)

      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 600 GB, OK)

   unassigned

      physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS, 600 GB, OK)

      physicaldrive 1I:1:4 (port 1I:box 1:bay 4, SAS, 600 GB, OK)

      physicaldrive 2I:1:5 (port 2I:box 1:bay 5, SAS, 600 GB, OK)

      physicaldrive 2I:1:6 (port 2I:box 1:bay 6, SAS, 600 GB, OK)

   SEP (Vendor ID PMCSIERA, Model  SRC 8x6G) 250 (WWID: 50014380102A509F)

=>  

After that, you may setup LVM in minutes as this site suggests.

Nice, what powerful the console is!

Installing Debian GNU/Linux 6 from network using a USB flash drive and HP firmware for HP DL380 G7

This is the easiest way to do the job (of course, it is not a new procedure but I need to register it).

This Debian webpage shows us the propietary firmware available and this webpage shows how to prepare you flash drive to install Debian GNU/Linux 6 in your server!

The procedure

First, download the image from the Debian's repository with propietary firmware.

Second, download the image to format the flash drive from here, in my case amd64 for the HP DL380 G7.

Third, prepare your flash drive in other Linux machine. Let /dev/sdX be the place where your flash drive is mounted.

zcat boot.img.gz > /dev/sdX
mkdir -p /mnt/pendrive
mount /dev/sdX /mnt/pendrive
cp firmware-6.0.2.1-amd64-netinst.iso /mnt/pendrive
umount /mnt/pendrive

Fourth, boot your server with the flash drive inserted to install the system from network. The firmware for the network card will be available (plus some others).

Enjoy!

Folding@Home in Debian GNU/Linux Squeeze amd64 as a service

I think distributed computing is fascinating, these days I have been learning about MPI, clusters and related stuff. The power of a lot of machines working collaboratively to solve a problem... amazing!, I love computing :-)

You know that a mayor part of time your CPU is not fully used because your processes are not all heavyweight all the time, lots of IO locks, dead times, etc. I mean, rarely ours CPUs are 100% used.


Folding@Home is a distributed computing project where protein folding is studied doing complex simulations, so complex and CPU demanding that the scientists turn to ask for help to the whole world. You can share some cycles of your CPU to the Science.

"Protein folding is linked to disease, such as Alzheimer's, ALS, Huntington's, Parkinson's disease, and many Cancers. Moreover, when proteins do not fold correctly (i.e. "misfold"), there can be serious consequences, including many well known diseases, such as Alzheimer's, Mad Cow (BSE), CJD, ALS, Huntington's, Parkinson's disease, and many Cancers and cancer-related syndromes."

I joined to help anonymously some years ago and I hope you can join us, create a team or just add yourself to the default group.

Check the webpage, they have been awarded and several results have been published.

I have installed the client in my dual boot machines (Windows/Linux) and in my home server.

I will show you how to install FAH in Debian GNU/Linux Lenny in a machine without X and in a service mode. Setting FAH as a service will allow it to start automatically every time the system boots up and you still are in charge because you can disable or even uninstall the service.

Log in as root or sudo as you like.

You should create a new user account by security reasons and according with the best practices suggested here (or here), the best place to put our client is /opt/fah.

# groupadd fah
# useradd -g fah fah --shell /bin/bash -d /opt/fah -m

From here you can download the installer, I chose FAH6.34-Linux64.tgz bacause my linux box is:

uname -a

Linux blogger 2.6.32-5-amd64 #1 SMP Tue Jun 14 09:42:28 UTC 2011 x86_64 GNU/Linux

wget http://www.stanford.edu/group/pandegroup/folding/release/FAH6.34-Linux64.tgz

tar xzf FAH6.34-Linux64.tgz

chmod u+x fah6

After that we may configure the client, buy we should do this as the fah user (will create configuration files and data):

su fah 

Then, we run the executable:

$ ./fah6

Note: Please read the license agreement (fah6 -license). Further
use of this software requires that you have read and accepted this agreement.



--- Opening Log file [July 22 00:10:47 UTC]


# Linux Console Edition #######################################################
###############################################################################

                       Folding@Home Client Version 6.34

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /opt/fah
Executable: ./fah6


[00:10:47] Configuring Folding@Home...

User name [Anonymous]? blogger (any username you like)
Team Number [0]? (The default team, create your team or join any one)
Passkey []?
Ask before fetching/sending work (no/yes) [no]? (this should be no as a service)
Use proxy (yes/no) [no]?
Acceptable size of work assignment and work result packets (bigger units
 may have large memory demands) -- 'small' is <5MB, 'normal' is <10MB, and
 'big' is >10MB (small/normal/big) [normal]?
Change advanced options (yes/no) [no]?

[00:12:18] - Ask before connecting: No
[00:12:18] - User name: blogger (Team 0)
[00:12:18] - User ID not found locally
[00:12:18] + Requesting User ID from server
[00:12:19] - Machine ID: 1
[00:12:19]
[00:12:19] Work directory not found. Creating...
[00:12:19] Could not open work queue, generating new queue...
[00:12:19] - Preparing to get new work unit...
[00:12:19] Cleaning up work directory
[00:12:19] + Attempting to get work packet
[00:12:19] - Connecting to assignment server
[00:12:21] - Successful: assigned to (171.67.108.52).
[00:12:21] + News From Folding@Home: Welcome to Folding@Home
[00:12:21] Loaded queue successfully.
[00:12:26] + Closed connections
[00:12:26]
[00:12:26] + Processing work unit
[00:12:26] Core required: FahCore_78.exe
[00:12:26] Core not found.
[00:12:26] - Core is not present or corrupted.
[00:12:26] - Attempting to download new core...
[00:12:26] + Downloading new core: FahCore_78.exe
[00:12:28] + 10240 bytes downloaded
(many bytes downloaded here...)

[00:12:38] + 1134407 bytes downloaded
[00:12:38] Verifying core Core_78.fah...
[00:12:38] Signature is VALID
[00:12:38]
[00:12:38] Trying to unzip core FahCore_78.exe
[00:12:39] Decompressed FahCore_78.exe (3435296 bytes) successfully
[00:12:39] + Core successfully engaged
[00:12:45]
[00:12:45] + Processing work unit
[00:12:45] Core required: FahCore_78.exe
[00:12:45] Core found.
[00:12:45] Working on queue slot 01 [July 22 00:12:45 UTC]
[00:12:45] + Working ...
[00:12:45]
[00:12:45] *------------------------------*
[00:12:45] Folding@Home Gromacs Core
[00:12:45] Version 1.90 (March 8, 2006)
[00:12:45]
[00:12:45] Preparing to commence simulation
[00:12:45] - Looking at optimizations...
[00:12:45] - Created dyn
[00:12:45] - Files status OK
[00:12:46] - Expanded 374844 -> 1805564 (decompressed 481.6 percent)
[00:12:46] - Starting from initial work packet
[00:12:46]
[00:12:46] Project: 6888 (Run 876, Clone 0, Gen 3)
[00:12:46]
[00:12:46] Assembly optimizations on if available.
[00:12:46] Entering M.D.

  Gromacs is Copyright (c) 1991-2003, University of Groningen, The Netherlands
        This inclusion of Gromacs code in the Folding@Home Core is under
        a special license (see http://folding.stanford.edu/gromacs.html)
         specially granted to Stanford by the copyright holders. If you
          are interested in using Gromacs, visit www.gromacs.org where
                you can download a free version of Gromacs under
         the terms of the GNU General Public License (GPL) as published
       by the Free Software Foundation; either version 2 of the License,
                     or (at your option) any later version.

[00:12:52] Protein: ALZHEIMER DISEASE AMYLOID
[00:12:52]
[00:12:52] Writing local files
[00:13:01] Extra SSE boost OK.
[00:13:01] Writing local files
[00:13:01] Completed 0 out of 250000 steps  (0%)

Ok, here we break the download of unit works [CONTROL+C] because we should setup fah as a service. It will continue from here when the binary be executed again anyway.

As root again, let's create /etc/init.d/fah with this content:

#!/bin/sh
### BEGIN INIT INFO
# Provides:          Service script for Folding@Home.
# Required-Start:    $syslog
# Required-Stop:     $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: This manages the service for client of the distributed computing project Folding@Home.
# Description:       Service script for Folding@Home.
#
### END INIT INFO

check_config()
{
        if [ ! -e /opt/fah/client.cfg ]; then
                echo "FAH has not been configured!!!"
                exit 1
        fi
}

case "$1" in
        start)
                check_config
                echo -n "Starting FAH..."
                cd /opt/fah
                start-stop-daemon --start --quiet --chdir /opt/fah -b --chuid fah --exec /opt/fah/fah6
                echo "."
        ;;

        stop)
                echo -n "Stopping FAH..."
                start-stop-daemon --stop --quiet --oknodo --user fah
                echo "."                                                                                     
        ;;                                                                                                   
                                                                                                             
        *)                                                                                                   
                echo "Usage: /etc/init.d/fah {start|stop}"                                                   
                exit 1
esac

exit 0

We make it executable:

chmod u+x /etc/init.d/fah

And we setup the service but this is Squeeze so we use insserv instead of update-rc.d (check this).

insserv /etc/init.d/fah

Now, we start our new service for the first time:

invoke-rc.d fah start

We can see what is happening in the background using tail:

$ cd /op/fah
$ ls -ltr
total 3864
-rwx------ 1 fah fah  314840 Feb 23 20:33 fah6
-rw-r--r-- 1 fah fah  146345 Feb 23 20:33 FAH6.34-Linux64.tgz
-rwxr-x--- 1 fah fah     110 Jul 21 20:12 client.cfg
-rw-r--r-- 1 fah fah    5214 Jul 21 20:12 MyFolding.html
-rw-r--r-- 1 fah fah       8 Jul 21 20:12 machinedependent.dat
-rw-r--r-- 1 fah fah    7168 Jul 21 20:12 queue.dat
-rwxr-x--- 1 fah fah 3435296 Jul 21 20:12 FahCore_78.exe
drwxr-x--- 2 fah fah    4096 Jul 21 20:39 work
-rw-r--r-- 1 fah fah     163 Jul 21 20:39 unitinfo.txt
-rw-r--r-- 1 fah fah    8527 Jul 21 20:39 FAHlog.txt
$ tail -f FAHlog.txt
[00:39:10] Project: 6888 (Run 876, Clone 0, Gen 3)
[00:39:10]
[00:39:10] Assembly optimizations on if available.
[00:39:10] Entering M.D.
[00:39:30] (Starting from checkpoint)
[00:39:30] Protein: ALZHEIMER DISEASE AMYLOID
[00:39:30]
[00:39:30] Writing local files
[00:39:39] Completed 2500 out of 250000 steps  (1%)
[00:39:39] Extra SSE boost OK.


(This continues... [CONTROL+C])

And that is all. Top will show you the process a FahCore using 100% of one of your cores but with the highest nice value, that is to say, with the lowest scheduling priority. So, if you don't use the CPU, FAH will do and you will be helping some scientist to find the cure to some ugly illness.























Take a look at this ranking, these papers (78!), these awards or these videos. Finally meet Vijay Pande!

So, Will you join Folding@Home?

Truecrypt in Debian GNU/Linux Lenny x64

Sometimes you need to encrypt some files like documents, passwords, etc. and puntual encryption maybe just the solution with gpg. However, if you need to encrypt lots of files you should try TrueCrypt for a masive encryption.

So, what about a virtual encrypted partition containing all your files?

The following procedure was done in a remote machine through ssh: 

First, you need to download the installer from here according to your architecture.

uname -a

Linux felix 2.6.32-bpo.5-xen-amd64 #1 SMP Thu Oct 21 13:52:59 UTC 2010 x86_64 GNU/Linux

In my case: truecrypt-7.0a-linux-console-x64.tar.gz

Second, you need some dependencies (?):

apt-get install fuse-utils libdevmapper dmsetup

Third, you must install TrueCrypt:

tar xzf truecrypt-7.0a-linux-console-x64.tar.gz

./truecrypt-7.0a-setup-console-x64

TrueCrypt 7.0a Setup
____________________


Installation options:

 1) Install truecrypt_7.0a_console_amd64.tar.gz
 2) Extract package file truecrypt_7.0a_console_amd64.tar.gz and place it to /tmp

To select, enter 1 or 2: 1

Before you can use, extract, or install TrueCrypt, you must accept the
terms of the TrueCrypt License.

Press Enter to display the license terms...

Press Enter or space bar to see the rest of the license.


TrueCrypt License Version 3.0

Software distributed under this license is distributed on an "AS
IS" BASIS WITHOUT WARRANTIES OF ANY KIND. THE AUTHORS AND
DISTRIBUTORS OF THE SOFTWARE DISCLAIM ANY LIABILITY. ANYONE WHO
USES, COPIES, MODIFIES, OR (RE)DISTRIBUTES ANY PART OF THE
SOFTWARE IS, BY SUCH ACTION(S), ACCEPTING AND AGREEING TO BE
BOUND BY ALL TERMS AND CONDITIONS OF THIS LICENSE. IF YOU DO NOT
ACCEPT THEM, DO NOT USE, COPY, MODIFY, NOR (RE)DISTRIBUTE THE
SOFTWARE, NOR ANY PART(S) THEREOF.


I. Definitions

1. "This Product" means the work (including, but not limited to,
source code, graphics, texts, and accompanying files) made
available under and governed by this version of this license
("License"), as may be indicated by, but is not limited to,
copyright notice(s) attached to or included in the work.

2. "You" means (and "Your" refers to) an individual or a legal
entity (e.g., a non-profit organization, commercial
organization, government agency, etc.) exercising permissions
granted by this License.

3. "Modification" means (and "modify" refers to) any alteration
of This Product, including, but not limited to, addition to or
deletion from the substance or structure of This Product,
translation into another language, repackaging, alteration or
removal of any file included with This Product, and addition of
any new files to This Product.

... ([q] to skip)
..
.

Do you accept and agree to be bound by the license terms? (yes/no): yes

Uninstalling TrueCrypt:
-----------------------

To uninstall TrueCrypt, please run 'truecrypt-uninstall.sh'.

Installing package...
usr/bin/truecrypt
usr/bin/truecrypt-uninstall.sh
usr/share/truecrypt/doc/License.txt
usr/share/truecrypt/doc/TrueCrypt User Guide.pdf

Press Enter to exit...


Four, let's create a virtual disk "Private.tc":


truecrypt Private.tc


Volume type:
 1) Normal
 2) Hidden
Select [1]: 1

Enter volume path: Private.tc

Enter volume size (sizeK/size[M]/sizeG): 1G

Encryption algorithm:
 1) AES
 2) Serpent
 3) Twofish
 4) AES-Twofish
 5) AES-Twofish-Serpent
 6) Serpent-AES
 7) Serpent-Twofish-AES
 8) Twofish-Serpent
Select [1]: 1

Hash algorithm:
 1) RIPEMD-160
 2) SHA-512
 3) Whirlpool
Select [1]: 2

Filesystem:
 1) None
 2) FAT
 3) Linux Ext2
 4) Linux Ext3
 5) Linux Ext4
Select [2]: 4

Enter password: ***********************************
Re-enter password: ***********************************

Enter keyfile path [none]:

Please type at least 320 randomly chosen characters and then press Enter:
Characters remaining: 30 (Press [ENTER] to end or know how much is left)


Done: 100.000%  Speed:   29 MB/s  Left: 0 s

The TrueCrypt volume has been successfully created.


To mount our disk:

truecrypt --mount Private.tc /mnt/truecrypt0

root:/mnt/truecrypt0# ls -ltr
total 16
drwx------ 2 root root 16384 2011-07-19 02:17 lost+found

Now we can move/copy our stuff!


To unmount our disk:

truecrypt -d Private.tc 

References

http://www.howtoforge.com/truecrypt_data_encryption
http://esaucairn-almuric.blogspot.com/2009/03/truecrypt-on-debian.html

Installing Debian GNU/Linux 6 from network using a USB flash drive and HP firmware for HP DL360 G7

UPDATE: A better way!

To install Debian GNU/Linux 6 in a server HP Proliant DL360 G7 using the network installer and a USB flash drive I wrote a shell script. Maybe you find it useful too.

You must know there is some proprietary firmware needed for the server to install (network at least).

After doing a little research, the know-how is in these sites:


http://wiki.debian.org/HP/ProLiant
http://wiki.debian.org/DebianInstaller/NetbootFirmware

In my case, the target architecture was amd64 (x86-64), but you would need some little modifications for any other architecture.

Before to run the shell script you need:

1- the network installer image from http://cdimage.debian.org/debian-cd/6.0.0/amd64/iso-cd/debian-6.0.0-amd64-netinst.iso

2- A USB flash drive (250 MB minimum).

3- A linux machine where you can run the shell script, I used a Debian GNU/Linux  Lenny.

The shell script is interactive, will download some files and write the USB flash drive. Be careful, data will be destroyed (after confirmation) because of formatting!

Remember to grant the needed permissions: chmod u+x script.sh


Use it on your own risk .



#!/bin/bash
function show
{
cat <<End-of-message
--------------------------------------------------
$0 TargetDevice TargetISO

Prepare a bootable pendrive with some HP Firmware!

Use it on your own risk.
-------------------------------------------------

End-of-message
}

##################################################################

show

# Validations

TARGET_DEVICE=$1
TARGET_ISO=$2
myPWD=$(pwd)

if [ $UID -ne 0 ]
then
    echo "You need to be root! (sudo?)"
    exit 3
fi

if [ -z "$TARGET_DEVICE" ]
then
    echo "You must specify the target device!"
    exit 1
fi

if [ -z "$TARGET_ISO" ]
then
        echo "You must specify the target iso!"
        exit 2
fi

echo "The current status of your device, beware any data will be destroyed!"

fdisk -l $TARGET_DEVICE

echo

ls -l $TARGET_ISO

echo

echo "Data will be destroyed!, press [Y] to continue"
read input

if [ "$input" != "Y" ]
then
    echo "Your answered [$input]"
    exit 0
fi

echo
echo "Creating bootable pendrive..."

if [ -n "$(mount | grep $TARGET_DEVICE)" ]
then
    echo "Unmounting $TARGET_DEVICE..."
    umount $TARGET_DEVICE

    if [ $? -ne 0 ]
    then
        echo "Error umounting the device!"
        exit 5
    fi
fi

if [ ! -e boot.img.gz ]
then
    echo
    echo "Downloading... boot.img.gz..."
    wget http://ftp.nl.debian.org/debian/dists/squeeze/main/installer-amd64/current/images/hd-media/boot.img.gz
fi

echo
echo "Formatting..."

zcat boot.img.gz > $TARGET_DEVICE
rm -fr /mnt/pendrive.tmp
mkdir -p /mnt/pendrive.tmp
mount $TARGET_DEVICE /mnt/pendrive.tmp

echo
echo "Copying iso..."

cp $TARGET_ISO /mnt/pendrive.tmp/

ls -l /mnt/pendrive.tmp/*.iso

echo
echo "Updating the initrd.gz with HP firmware..."

# Make a temp dir and transform the firmware tarball into an firmware initramfs addon
FWTMP=$HOME/tmp/d-i_firmware
#rm -rf $FWTMP
mkdir  -p $FWTMP
cd $FWTMP

target=firmware.tar.gz

if [ ! -e $target ]
then
    wget http://cdimage.debian.org/cdimage/unofficial/non-free/firmware/lenny/current/firmware.tar.gz
fi

tar -zxf firmware.tar.gz

if [ -z "$(which pax)" ]
then
    echo
    echo "Installing pax..."
    apt-get install pax
fi

for name in *.deb ; do
    ar -p $name data.tar.gz | tar -zxf -
done

pax -x sv4cpio -s '%lib%/lib%' -w lib | gzip -c >firmware.cpio.gz

# cd to the directory where you have your initrd
cd /mnt/pendrive.tmp

echo
echo "Backing up initrg.gz..."
[ -f initrd.gz.orig ] || cp -p initrd.gz initrd.gz.orig
cat $FWTMP/firmware.cpio.gz >> initrd.gz

ls -l initrd.gz*

cd $myPWD

echo
echo "Unmounting device..."

umount /mnt/pendrive.tmp

echo
echo "Done."


Ooops, I found this a little late :-|