We recently hit a problem when trying to run a fairly simple script through grid engine.  The script used an internal redirection to merge together STDIN and STDOUT before doing a final transformation on the result.

A simplified version of what the script did is something like this:

echo hello 2>&1 | sed s/^/prefix:/

Which should put “prefix:” in front of any output from either STDIN or STDOUT.  Running this from the command line worked, but trying to run it through qsub failed.

$ echo "echo Hello 2>&1 | sed s/^/prefix:/" | qsub -cwd -V

Your job 201149 ("STDIN") has been submitted

$ ls -tr

STDIN.o201149
STDIN.e201149

$ more STDIN.e201149 
Ambiguous output redirect.

After many experiments we found the cause for this, which is actually pretty simple, but since it didn’t turn up in a google search I thought I’d document it.  The reason this type of redirection fails under qsub is that the default shell used by qsub is csh rather than bash, and the normal 2>&1 notation is a bash specific extension.  You can therefore make this work by either using csh redirection, or specifying bash as your preferred shell.  Either of the commands below would work:

echo "echo Hello |& sed s/^/prefix:/" | qsub -cwd -V
echo "echo Hello 2>&1 | sed s/^/prefix:/" | qsub -cwd -V -S /bin/bash