Find content of one file from another set of gz files in UNIX-CodePudding

I have a list of 500 ids in search_these.txt file; and 300 gzipped data files numbered like {}_of_300_search_in.gz. I want to extract data for 500 ids from each of 300 data files into one output file.

I have tried the following code but it throws an error Usage: grep [OPTION]... PATTERN [FILE].... I'm not sure what am I doing wrong here.

cat search_these.txt | while read line
do
    seq 1 300 | xargs -I '{}' bash -c 'zcat {}_of_300_search_in.gz | grep -w $line' >> output.txt
done

CodePudding user response：

Try grep -w -- "$line" to avoid $line being interpreted as options and arguments. The -- ends the option list, which is necessary when $line starts with a leading -.

CodePudding user response：

As far as I see, you can reduce your entire code by the following single command:

zgrep -hf search_these.txt {1..300}_of_300_search_in.gz

The command zgrep is a simple wrapper, similar to zcat, that gunzip's the file and passes it and all accompanying arguments to grep. So the arguments -h and -f can be read from man grep to understand what they do.