"Unknown Job Id Error (15001)" in Torque (PBS)

Today I spent some time struggling with an little obscure error in work.

A job with dependencies on others jobs was impossible to queue in Torque (running in a virtual cluster created just for development and testing).

The damn job:

----------------------------------------
#!/bin/bash

#PBS -S /bin/bash
#PBS -N metgrid
#PBS -q batch
#PBS -l walltime=02:00:00
#PBS -l nodes=1:ppn=1
#PBS -o metgrid-oe.log
#PBS -j oe
#PBS -W depend=afterok:325.dadm:326.dadm

#bla bla bla
----------------------------------------

This error was reported in :

----------------------------------------
cat /var/spool/torque/server_logs/20140505
...
05/05/2014 17:55:00;0080;PBS_Server.4427;Req;req_reject;Reject reply code=15041(Job rejected by all possible destinations (check syntax, queue resources, ...)), aux=0, type=Commit, from xxx@dhead
05/05/2014 17:55:29;0010;PBS_Server.4427;Job;327.dadm;Exit_status=0 resources_used.cput=00:00:28 resources_used.mem=43356kb resources_used.vmem=96480kb resources_used.walltime=00:00:50
...
----------------------------------------


The dependency jobs running:

----------------------------------------
qstat -n:

dadm:
                                                                                  Req'd    Req'd       Elap
Job ID                  Username    Queue    Jobname          SessID  NDS   TSK   Memory   Time    S   Time
----------------------- ----------- -------- ---------------- ------ ----- ------ ------ --------- - ---------
334.dadm                user        infiniba ungrib            19895     1      1    --   02:00:00 R  00:00:00
   dhead/0
335.dadm                user        infiniba geogrid           19901     1      1    --   02:00:00 R  00:00:00
   dhead/1
----------------------------------------

Then this surprised me:

dhead:~/swrf/test/testMetgrid$ qstat -f 340.dadm
qstat: Unknown Job Id Error 340.dadm.mydomain.cl

So, I was using an incomplete job id that the machine tried to repair adding the domain but even so it failed again...

user@dhead:~/swrf/test/testMetgrid$ qstat -f 340.dadm.mydomain.cl
qstat: Unknown Job Id Error 340.dadm.mydomain.cl

Well, the domain was missing in the configuration of the server...

This explained a solution:

"check that the first name given in your hosts file for your server exactly matches the name given
in your server_name file in your torque configuration."

... and yes, it was about basic configuration :-P

The changes I made:

root@dadm:/var/spool/torque# cat /etc/hosts
127.0.0.1    localhost
192.168.24.38    dadm.mydomain.cl dadm

root@dadm:/var/spool/torque# cat /var/spool/torque/server_name
dadm.mydomain.cl

Then I restarted the server and after queueing my job all went fine.


----------------------------------------
root@dadm:/var/spool/torque# qstat -n

dadm.mydomain.cl:
                                                                                  Req'd    Req'd       Elap
Job ID                  Username    Queue    Jobname          SessID  NDS   TSK   Memory   Time    S   Time
----------------------- ----------- -------- ---------------- ------ ----- ------ ------ --------- - ---------
352.dadm.mydomain.cl     user     infiniba ungrib            20605     1      1    --   02:00:00 R  00:00:04
   dhead/0
353.dadm.mydomain.cl     user     infiniba geogrid           20619     1      1    --   02:00:00 R  00:00:04
   dhead/1
354.dadm.mydomain.cl     user     infiniba metgrid             --      1      1    --   02:00:00 H       --
----------------------------------------

Finally, I just need to repair the shellscripts that build and configure the cluster.

maven-release-plugin + VADDIN

Before releasing a mavenized web project plus Vaadin, several problems popped up because the modification of sources made by Vaadin compilations.


Finally the solution was to exclude the problematic part that always is regenerated if it does not exist anyway:

------------------------------------------------------------------------
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-release-plugin</artifactId>
    <version>2.5</version>
    <configuration>
        <providerImplementations>
            <svn>javasvn</svn>
        </providerImplementations>
        <autoVersionSubmodules>true</autoVersionSubmodules>
        <tagNameFormat>v@{project.version}</tagNameFormat>
        <checkModificationExcludes>
            <checkModificationExclude>${project.build.outputDirectory}/**/VAADIN/**</checkModificationExclude>
        </checkModificationExcludes>
        <configuration>
            <preparationGoals>clean verify</preparationGoals>
        </configuration>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>com.google.code.maven-scm-provider-svnjava</groupId>
            <artifactId>maven-scm-provider-svnjava</artifactId>
            <version>2.1.1</version>
        </dependency>
    </dependencies>
</plugin>
------------------------------------------------------------------------

Enjoy!

Unable to tag SCM with maven-release-plugin

An incomplete layout in your subversion repository may be one of the causes for this error:

mvn release:clean -e
mvn release:prepare --batch-mode -e
mvn release:perform -e


------------------------------------------------
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.4.2:prepare (default-cli) on project geoemissions: Unable to tag SCM
Provider message:
The svn tag command failed.
Command output:
svn: '/svn/sources/GeoEmissions/!svn/bc/91/tags' path not found

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
------------------------------------------------

Similar error with scm:

mvn scm:tag -Dtag="Test"

------------------------------------------------
[ERROR] Provider message:
[ERROR] SVN tag failed.
[ERROR] Command output:
[ERROR] svn: E160013: Path 'http://svn.geoaire.cl/svn/sources/YourProject/tags/v0.01' does not exist in revision 91
------------------------------------------------

My project was up and running in the trunk, what else was I going to need? Maybe tags and branches? :-)

From now and on I rather prepare a shellscript (that I version too) to build up my projects in the very beginning with this three important directories:


------------------------------------------------
WORKSPACE=/srv/svn/repositories/sources/YourProject
mkdir -p $WORKSPACE
svnadmin create --fs-type fsfs $WORKSPACE
chmod 770 -R $WORKSPACE
svn mkdir file:///srv/svn/repositories/sources/YourProject/{trunk,tags,branches}
------------------------------------------------

After the doing the "svn mkdir", the two plugins worked well. This was my configuration:

 ------------------------------------------------
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-scm-plugin</artifactId>
                    <version>1.9</version>
                    <configuration>
                        <providerImplementations>
                            <svn>javasvn</svn>
                        </providerImplementations>
                        <connectionType>developerConnection</connectionType>
                        <username>xxx</username>
                        <password>yyy</password>
                    </configuration>
                    <dependencies>
                        <dependency>
                            <groupId>com.google.code.maven-scm-provider-svnjava</groupId>
                            <artifactId>maven-scm-provider-svnjava</artifactId>
                            <version>2.1.0</version>
                        </dependency>
                    </dependencies>
                </plugin>

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-release-plugin</artifactId>
                    <version>2.4.2</version>
                    <configuration>
                        <providerImplementations>
                            <svn>javasvn</svn>
                        </providerImplementations>                  
                        <autoVersionSubmodules>true</autoVersionSubmodules>
                        <tagNameFormat>v@{project.version}</tagNameFormat>
                    </configuration>
                    <dependencies>
                        <dependency>
                            <groupId>com.google.code.maven-scm-provider-svnjava</groupId>
                            <artifactId>maven-scm-provider-svnjava</artifactId>
                            <version>2.1.0</version>
                        </dependency>
                    </dependencies>
                </plugin>
------------------------------------------------

Enjoy!

Install an Ubuntu server from a pendrive

Did you know that you may install an Ubuntu server image just from a pendrive?

I downloaded the image, inserted a pendrive (be aware, it will be totally erased)...

dmesg

---------------------------------------------------
[25003.567296] usb 2-1.4: new high-speed USB device number 4 using ehci-pci
[25003.676610] usb 2-1.4: New USB device found, idVendor=03f0, idProduct=5307
[25003.676617] usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[25003.676622] usb 2-1.4: Product: v165w
[25003.676627] usb 2-1.4: Manufacturer: HP
[25003.676631] usb 2-1.4: SerialNumber: 00000000000064
[25004.248877] Initializing USB Mass Storage driver...
[25004.249075] scsi4 : usb-storage 2-1.4:1.0
[25004.249243] usbcore: registered new interface driver usb-storage
[25004.249247] USB Mass Storage support registered.
[25005.249142] scsi 4:0:0:0: Direct-Access     hp       v165w            0.00 PQ: 0 ANSI: 2
[25005.249627] sd 4:0:0:0: Attached scsi generic sg3 type 0
[25005.250955] sd 4:0:0:0: [sdc] 7892040 512-byte logical blocks: (4.04 GB/3.76 GiB)
[25005.251674] sd 4:0:0:0: [sdc] Write Protect is off
[25005.251683] sd 4:0:0:0: [sdc] Mode Sense: 00 00 00 00
[25005.252410] sd 4:0:0:0: [sdc] Asking for cache data failed
[25005.252417] sd 4:0:0:0: [sdc] Assuming drive cache: write through
[25005.256280] sd 4:0:0:0: [sdc] Asking for cache data failed
[25005.256287] sd 4:0:0:0: [sdc] Assuming drive cache: write through
[25005.746065]  sdc: sdc1 sdc2
[25005.749229] sd 4:0:0:0: [sdc] Asking for cache data failed
[25005.749236] sd 4:0:0:0: [sdc] Assuming drive cache: write through
[25005.749243] sd 4:0:0:0: [sdc] Attached SCSI removable disk
---------------------------------------------------

... that it was identified as a /dev/sdc, then I saved the image into the pendrive:

---------------------------------------------------
cat ubuntu-12.04.2-server-amd64.iso > /dev/sdc
---------------------------------------------------

Just like that! Boot your server and enjoy!

A DLNA server in Debian GNU/Linux Wheezy for your Samsung Smart TV

Your home server may be configured to be a DLNA server so you can enjoy your own videos, pictures and music in a smart tv.

Of course, it is assumed you have a LAN with a linux server and a smart tv networked.

There are several open source proyects, here I use minidlna that is already available in Debian Wheezy! the new stable version.

Here it is the procedure as described here and here, and it took very little.

apt-get install minidlna

Then, you need to setup the server in /etc/minidlna.conf, my current configuration as example:

root@gatux:~# grep -o "^[a-z].*" /etc/minidlna.conf
media_dir=/srv/shared
log_level=info
port=8200
friendly_name=DLNA Server
serial=12345678
model_number=1
inotify=yes
album_art_names=Cover.jpg/cover.jpg/AlbumArtSmall.jpg/albumartsmall.jpg/AlbumArt.jpg/albumart.jpg/Album.jpg/album.jpg/Folder.jpg/folder.jpg/Thumb.jpg/thumb.jpg
notify_interval=15

(Re)build your database, I will use a cron task every day :-)

/etc/init.d/minidlna stop
minidlna -f /etc/minidlna.conf -R -d
/etc/init.d/minidlna start 

Open the tcp port 8200 in the firewall of your home linux server if it is necessary (sure it is), I use webmin but this may just do the the trick temporarily:

iptables -I INPUT -p tcp --dport 8200 -j ACCEPT

Finally, with your dlna server running, in your smart tv choose the source, a Samsung UN55ES7100 in my case:

Nice, the Debian icon is diplayed!

Descending into the directories I found my collection... well just one video in my case:

Enjoy!