Whenever you work with bash there are three file descriptors that are always in play and that you need to be aware of. These are, STDOUT, STDERR and STDIN which stand for standard output, standard error and standard input. Normally when programs are executed, unless they explicitly write output to a file, they will output to either standard output or standard error, which means the output will go to the terminal and you will see it on the screen (since both STDOUT and STDERR go to the screen by default). STDIN is where your program will get input data from if you prompt for input from the user. The user enters some data on the screen and your program or script will read it (I won’t say too much more about STDIN).
Of course STDOUT and STDERR are concepts, the actual file descriptors themselves are numbers. When you’re using bash, 1 will be the file descriptor that stands for STDOUT and 2 will be the file descriptor that stands for STDERR. So when you want to redirect the output of a program or script to a file rather than going to the screen you need to change where STDOUT or STDERR (or both) are pointing. To do this you can do the following:
ls -al 1>file1.txt 2>file2.txt
You use the > ‘operator’ to redirect where STDOUT and STDERR are going to write their output by giving the > ‘operator’ a file descriptor and the file you want the output to be written to. In this case nothing will appear on the screen but two new files will be created in the current directory, _file1.tx_t will have the contents of STDOUT and file2.txt the contents of STDERR (of course unless an error actually occurred file2.txt will be empty). If you don’t specify any number e.g.:
ls -al > file1.txt
bash assumes you want to redirect STDOUT, so STDOUT will go to the file you supply, while STDERR will still go to the screen.
But what if you want to redirect both standard output and standard error to the same file. This is easily possible and simply builds on the previous concept. You may have seen the following construct before:
ls -al > file1.txt 2>&1
What this actually means is as follows. We first redirect standard output to a particular file, remember when we don’t specify a number, bash assumes we want to redirect STDOUT:
ls -al > file1.txt
The last bit of the command (2>&1) literally means:
- take STDERR (i.e. file descriptor 2)
- and redirect it (i.e. > )
- to the same place where STDOUT is pointing (&1)
The &1 in the last bit is a little tricky. You may think of the & as a sort-of reference and the 1 is of course STDOUT. So, &1 means – ‘the place that STDOUT is currently referencing’. Since we’ve previously redirected STDOUT to a file this will redirect STDERR to that same file. This is actually the old-school way of redirecting both STDOUT and STDERR to the same place. There is a shortcut, you simply need to do the following:
ls -al >& file1.txt
which will achieve the same result.
A couple of interesting points to note here.
- You can easily turn this redirection around and redirect STDERR first and then point STDOUT to the same place, but this looks a little more clunky since bash can’t assume anything and so we need to specify everything explicitly:
ls -al 2> file1.txt 1>&2As you can see we redirect STDERR (2) to a file first and then use the same construct as before but swap the numbers around. This is good to know and demonstrates that we understand what is going on, but we probably wouldn’t do this in the wild. Instead use the shortcut version I described above.
- When you redirect both standard output and standard error to the same file, you may get some unexpected results. This is due to the fact that STDOUT is a buffered stream while STDERR is always unbuffered. This means that every character of STDERR is written as soon as it is available while STDOUT writes stuff in batches. When both STDOUT and STDERR are going to the same file you may see error messages appear sooner than you would have expected them in relation to the actual output of your program or script. It isn’t anything to be alarmed about but is simply a side-effect of buffered vs. unbuffered streams, you just need to keep it in mind.
That’s all there is to know about bash redirection, always good to be comfortable with the basics. If I’ve missed anything, feel free to fill in the blanks (i.e. leave a comment :)).
__
For more tips and opinions on software development, process and people subscribe to skorks.com today.
__Image by Phillie Casablanca