Process Substitution

Input

I use process substitution a lot but most of the time it is for input. You can find a lot examples out that. Here is a few.

Compare two set of output

The example is that you run two commands any they give some output. Now you want to process both output with one command which take files as input.

Lets say you have some auto detected devices on your computer and want to compare the with another computer.

With files we could do like this:

	ls -R /dev > foo
	ssh b.example.org ls -R /dev > bar
	diff foo bar
	rm foo bar
	

Remember to remove the temporary files. Annoying.

With process substitution we can do like this:

	diff <( ls -R /dev ) <( ssh b.example.org ls -R /dev )
	

Output

foo | tee >( bar )

In this example we have the process foo which gives some output we wnat to see. But at the same time we want to process the same output with bar and see that too.

In this example I want to list the file in this directory and count the number of files.

With files we could do like this:

	ls > foo
	cat foo
	cat foo | wc -l
	rm foo
	

With process substitution we can do like this:

	ls | tee >( wc -l )
	

Note that you do not have to remove any files afterwards.

The trick is that tee sends all std-input to std-output, but at the same time saves a copy to the file given as argument. In this case it is a trick file which enclose another process. wc -l counts all the lines, and the output will be on the last line. Try this at home.

foo | tee >( bar ) | quux

In this example we have one output process and want to process it with two other processes.

The job is to take a gzip file, unpack it and make bzip2 file and a xz file.

With files we could do like this:

	zcat foo.gz | bzip2 -c > foo.bzip2
	zcat foo.gz | xz -c > foo.xz
	

In this way we will unpack the gz file twice and if it is 1TB it might take a long time.

With process substitution we can do like this:

	zcat foo.gz | tee >( bzip2 -c > foo.bzip2 ) | xz -c > foo.xz
	

And if you also want a lz as well:

	zcat foo.gz | tee >( bzip2 -c > foo.bzip2 ) | tee >( xz -c > foo.xz ) | lzma -c > foo.lz
	

foo | tee >( bar ) | quux #2

This example is a bit of the same. We have one output and want to process and store the output in two different files.

With files we could do like this:

	ls | grep foo > got-foo
	ls | grep -v foo > no-foo
	

Luckly we do not have any temporary files but we might have a situation where a new file have arrived and we only see that in the second run.

With process substitution we can do like this:

	ls | tee >( grep foo > got-foo ) | grep -v foo > no-foo
	

foo | tee >( sort | tail )

A command takes a lot of time and output to screen. Output should be processed with another time consuming command but we want to see the output of the first command during processing.

	du -m | tee >( sort -n | tail )
	

by Hans Schou <chlor@schou.dk> 2012-09-12