Here's a simplified version of the command I'm trying to run:

xtrabackup --backup --stream=xbstream | xbcloud put --storage=s3

Essentially what this does is generate a binary stream of a database backup and pipe it to another command which uploads it to AWS S3.

However, while this command is running, both xtrabackup and xbcloud echo output to the terminal. I want all of this output to go into a file instead.

I've tried adding > file.txt to the end but that only captures the output of the xbcloud command and not the xtrabackup command. I've also done some searching on SO and found suggestions that I should put | tee file.txt at the end but that just seems to result in an empty file.

CodePudding user response：

You may use the following syntax, to:

• Collect all errors from the first command xtrabackup into a file named xtrabackup-stderr.log
• Collect all standard and error outputs of the second command xbcloud to a file named xbcloud-all-output.log

xtrabackup --backup --stream=xbstream 2> xtrabackup-stderr.log | xbcloud put --storage=s3 &> xbcloud-all-output.log

Does this satisfy your requirements?

Some details

In Bash:

• 1> or > (considered as the same 1>) redirects the stdout Stream Channel (known as Standard Output) to the defined file path.
• 2> redirects the sterr Stream Channel (known as Standard Error Output) to the defined file path.
• &> redirects both stdout and stderr to the defined file path.
• | Known as Pipeline, passes the previous command's stdout to the next command/process.

Note: You won't like to redirect the first command's stdout to a file since you are already passing its stdout to the next command to be processed via | pipeline, otherwise the second command xbcloud won't get any data to process/upload.

Update #1

You may also separate the second command stdout and stderr to different files by:

xtrabackup --backup --stream=xbstream 2> xtrabackup-stderr.log | xbcloud put --storage=s3 > xbcloud-stdout.log 2> xbcloud-stderr.log