Advertisement
Advertisement


How to pipe list of files returned by find command to cat to view all the files


Question

I am doing a find and then getting a list of files. How do I pipe it to another utility like cat (so that cat displays the contents of all those files) and basically need to grep something from these files.

2013/10/01
1
207
10/1/2013 3:12:01 PM

Accepted Answer

  1. Piping to another process (Although this WON'T accomplish what you said you are trying to do):

    command1 | command2
    

    This will send the output of command1 as the input of command2

  2. -exec on a find (this will do what you are wanting to do -- but is specific to find)

    find . -name '*.foo' -exec cat {} \;
    

    (Everything between find and -exec are the find predicates you were already using. {} will substitute the particular file you found into the command (cat {} in this case); the \; is to end the -exec command.)

  3. send output of one process as command line arguments to another process

    command2 `command1`
    

    for example:

    cat `find . -name '*.foo' -print`
    

    (Note these are BACK-QUOTES not regular quotes (under the tilde ~ on my keyboard).) This will send the output of command1 into command2 as command line arguments. Note that file names containing spaces (newlines, etc) will be broken into separate arguments, though.

2011/11/24
345
11/24/2011 7:33:19 PM

Modern version

POSIX 2008 added the + marker to find which means it now automatically groups as many files as are reasonable into a single command execution, very much like xargs does, but with a number of advantages:

  1. You don't have to worry about odd characters in the file names.
  2. You don't have to worry about the command being invoked with zero file names.

The file name issue is a problem with xargs without the -0 option, and the 'run even with zero file names' issue is a problem with or without the -0 option — but GNU xargs has the -r or --no-run-if-empty option to prevent that happening. Also, this notation cuts down on the number of processes, not that you're likely to measure the difference in performance. Hence, you could sensibly write:

find . -exec grep something {} +

Classic version

find . -print | xargs grep something

If you're on Linux or have the GNU find and xargs commands, then use -print0 with find and -0 with xargs to handle file names containing spaces and other odd-ball characters.

find . -print0 | xargs -0 grep something

Tweaking the results from grep

If you don't want the file names (just the text) then add an appropriate option to grep (usually -h to suppressing 'headings'). To absolutely guarantee the file name is printed by grep (even if only one file is found, or the last invocation of grep is only given 1 file name), then add /dev/null to the xargs command line, so that there will always be at least two file names.

2019/04/04

There are a few ways to pass the list of files returned by the find command to the cat command, though technically not all use piping, and none actually pipe directly to cat.

  1. The simplest is to use backticks (`):

    cat `find [whatever]`
    

    This takes the output of find and effectively places it on the command line of cat. This doesn't work well if find has too much output (more than can fit on a command-line) or if the output has special characters (like spaces).

  2. In some shells, including bash, one can use $() instead of backticks :

    cat $(find [whatever])
    

    This is less portable, but is nestable. Aside from that, it has pretty much the same caveats as backticks.

  3. Because running other commands on what was found is a common use for find, find has an -exec action which executes a command for each file it finds:

    find [whatever] -exec cat {} \;
    

    The {} is a placeholder for the filename, and the \; marks the end of the command (It's possible to have other actions after -exec.)

    This will run cat once for every single file rather than running a single instance of cat passing it multiple filenames which can be inefficient and might not have the behavior you want for some commands (though it's fine for cat). The syntax is also a awkward to type -- you need to escape the semicolon because semicolon is special to the shell!

  4. Some versions of find (most notably the GNU version) let you replace ; with + to use -exec's append mode to run fewer instances of cat:

    find [whatever] -exec cat {} +
    

    This will pass multiple filenames to each invocation of cat, which can be more efficient.

    Note that this is not guaranteed to use a single invocation, however. If the command line would be too long then the arguments are spread across multiple invocations of cat. For cat this is probably not a big deal, but for some other commands this may change the behavior in undesirable ways. On Linux systems, the command line length limit is quite large, so splitting into multiple invocations is quite rare compared to some other OSes.

  5. The classic/portable approach is to use xargs:

    find [whatever] | xargs cat
    

    xargs runs the command specified (cat, in this case), and adds arguments based on what it reads from stdin. Just like -exec with +, this will break up the command-line if necessary. That is, if find produces too much output, it'll run cat multiple times. As mentioned in the section about -exec earlier, there are some commands where this splitting may result in different behavior. Note that using xargs like this has issues with spaces in filenames, as xargs just uses whitespace as a delimiter.

  6. The most robust, portable, and efficient method also uses xargs:

    find [whatever] -print0 | xargs -0 cat
    

    The -print0 flag tells find to use \0 (null character) delimiters between filenames, and the -0 flag tells xargs to expect these \0 delimiters. This has pretty much identical behavior to the -exec...+ approach, though is more portable (but unfortunately more verbose).

2018/05/24

To achieve this (using bash) I would do as follows:

cat $(find . -name '*.foo')

This is known as the the "command substitution" and it strips line feed by default which is really convinient !

more infos here

2012/09/27

Sounds like a job for a shell script to me:

for file in 'find -name *.xml'
do
   grep 'hello' file
done

or something like that

2009/05/14

Here's my way to find file names that contain some content that I'm interested in, just a single bash line that nicely handles spaces in filenames too:

find . -name \*.xml | while read i; do grep '<?xml' "$i" >/dev/null; [ $? == 0 ] && echo $i; done
2013/05/18

Source: https://stackoverflow.com/questions/864316
Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Email: [email protected]