Skip to main content

Streams and Pipelines in UNIX environments

Linux shells use three standard I/O streams, each of which is associated with a well-known file descriptor: 
  1. stdout is the standard output stream, which displays output from commands. It has file descriptor 1.
  2. stderr is the standard error stream, which displays error output from commands. It has file descriptor 2.
  3. stdin is the standard input stream, which provides input to commands. It has file descriptor 0.

Redirecting output 

ls x* z* >stdout.txt 2>stderr.txt
Standard output in the first file and the errors in the second file.
Output redirection using n> usually overwrites existing files. You can control this with the noclobber option of the set builtin.

ls x* z* 2>&1 >output.txt

This will redirect all errors to standard output and standard output is redirected to the file. The errors will be displayed in the terminal and the output is written to a file.
  
ls x* z* >output.txt 2>&1 

This next example first redirects all standard output to a file and then redirects errors to the standard output. This causes all content to be written to the file.

ls x* z* &>output.txt
 
Redirects both outputs to the file

ls x* z* 2>/dev/null
 
Ignores errors by writing to a null file.

tr a b > text1

Replaces letter a in the entries of the redirected input file with letter b.


Creating Pipelines

You use the | (pipe) operator between two commands to direct the stdout of the first to the stdin of the second.

ls y* x* z* u* q*  2>&1 |sort

Redirected errors and output lines will be passed to be sorted.

cat file3 | tr c d > file4

Contents of file 3 will be passed to the tr command which is redirected to a new file called file4.

bunzip2 -c somefile.tar.bz2 | tar -xvf -

The hyphen is used in place of file name to be redirected from the standard input. This will wait until the standard input becomes available.


xargs Command


xargs < text1 echo "args>"

args > 1 apple 2 pear 3 banana

xargs --max-args 3 text1 echo "args>"
args > 1 apple 2 args > pear 3 banana 

teeCommand
ls tee | file.txt

Displays results in the screen while writing to a file

Comments

Post a Comment

Popular posts from this blog

Detaching a process from terminal - exec(), system(), setsid() and nohup

Linux processes are created by fork() and exec(). The very first process of POSIX systems is init and subsequent processes are derived from the init as parent. These subsequent processes are child processes. During forking the parent process copies itself - with all I/O, address space, stack, everything. The only thing that is different is the process ID. The parent and child will have 2 different process IDs. The system() library function uses fork(2) to create a child process that executes the shell command specified in command using execl(3) as follows: execl("/bin/sh", "sh", "-c", command, (char *) 0); system() returns after the command has been completed. system() executes a command specified in command by calling /bin/sh -c command , and returns after the command has been completed. During execution of the command, SIGCHLD will be blocked, and SIGINT and SIGQUIT will be ignored.  system() calls are often made within programs to execut...
Post a Comment
Read more

Vaadin vs GWT

From Chapter 1 of book of Vaadin I quote the following. Vaadin Framework is a Java web application development framework that is designed to make creation and maintenance of high quality web-based user interfaces easy. Vaadin supports two different programming models: server-side and client-side . The server-driven programming model is the more powerful one . It lets you forget the web and program user interfaces much like you would program a desktop application with conventional Java toolkits such as AWT, Swing, or SWT. But easier. While traditional web programming is a fun way to spend your time learning new web technologies, you probably want to be productive and concentrate on the application logic. The server-side Vaadin framework takes care of managing the user interface in the browser and the AJAX communications between the browser and the server . With the Vaadin approach, you do not need to learn and deal directly with browser technologies, such as HTML or JavaScript. ...
3 comments
Read more

C++ Callbacks using function pointers vs boost bind +boost function

In C, the most common uses of callbacks are as parameters to library functions like qsort , and as callbacks for Windows functions, etc. For example you might have a library that provides some sorting functions but you want to allow the library user to provide his own sorting function. Since the arguments and the return values do not change depending on the sorting algorithm, this can be facilitated in a convenient manner using function callbacks. Callbacks are also used as event listeners. onMouseClick(), onTerminalInput(), onData(), onConnectionStatus(), onRead() are probably some examples you've already seen in different libraries. The libraries have these callback functions and the users of the library are supposed to implement them. The library has a function pointer to these functions and calls them on their event loop which will invoke the code of the inherited classes of the library user. The implementation of function pointers is simple: they are just "code p...
Post a Comment
Read more