Difference between revisions of "Tutorial5: Redirection"

From CDOT Wiki
Jump to: navigation, search
(INVESTIGATION 2: REDIRECTION USING PIPES)
(Piping (Using Pipes))
 
(192 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
===Main Objectives of this Practice Tutorial===
 
===Main Objectives of this Practice Tutorial===
  
:* Define the terms '''Standard Input''' ('''stdin'''), '''Standard Output''' ('''stdout'''), and '''Standard Error''' ('''stderr''')
+
:* Understand and use the '''cut''', '''tr''', and '''wc''' Linux commands
  
:* Understand the purposes of redirection symbols '''>''', '''>>''', '''2>''', '''2>>''', '''|''' and define their purpose in terms of redirection
+
:* Define the terms '''Standard Input''' (''stdin''), '''Standard Output''' (''stdout''), and '''Standard Error''' (''stderr'')
  
:* Understand the purpose of the additional file manipulation commands: '''cut''', '''tr''', and '''wc'''
+
:* Understand and use the '''>''', '''>>''', '''2>''', '''2>>''' symbols with Linux commands
  
:* Define the term '''pipeline command''' and explain how a pipeline command works in terms of redirection
+
:* Understand the purpose of the the '''/dev/null''' file and the '''Here Document'''
  
:* Define the term '''filter''' and how it relates to redirection using pipeline commands
+
:* Define the term '''pipeline command''' and explain how a pipeline command functions
  
:* Understand how to use the '''semicolon''' symbol "''';'''" to issue multiple Unix / Linux commands
+
:* Define the term '''filter''' and how it relates to pipeline commands
  
:* Understand how to use '''grouping''' using the symbols "'''( )'''" and how it is used to issue multiple Unix / Linux commands
+
:* Use the '''semicolon''' ";" and '''grouping''' "( )" symbols to issue multiple Unix / Linux commands on a single line
 +
 
 +
:* Use the '''backslash''' "\" symbol to spread-out long Unix/Linux commands over multiple lines
 
<br>
 
<br>
  
Line 39: Line 41:
 
* [http://www.linfo.org/standard_output.html Standard Output (stdout)]
 
* [http://www.linfo.org/standard_output.html Standard Output (stdout)]
 
* [http://www.linfo.org/standard_error.html Standard Error (stderr)]
 
* [http://www.linfo.org/standard_error.html Standard Error (stderr)]
* [http://www.linfo.org/pipe.html Piping (pipes)]<br><br>
+
* [http://www.linfo.org/pipe.html Pipeline Commands]<br><br>
 
Multiple Commands
 
Multiple Commands
 
* [https://www.javatpoint.com/linux-semicolon Semicolon]<br>
 
* [https://www.javatpoint.com/linux-semicolon Semicolon]<br>
Line 58: Line 60:
  
 
= KEY CONCEPTS =
 
= KEY CONCEPTS =
 +
 +
===Additional File Manipulation Commands===
 +
 +
Before proceeding, let's look at some additional commands used to manipulate content of text files.
 +
 +
Refer to the table below regarding these text file manipulation commands:
 +
<br><br>
 +
[[Image:manipulation-commands.png|left|700px|]]
 +
<br><br><br><br><br><br><br><br><br>
  
 
===Redirection (Standard Input, Standard Output, Standard Error)===
 
===Redirection (Standard Input, Standard Output, Standard Error)===
  
''... '''standard streams''' are preconnected input and output communication channels between a computer program and its environment when it begins execution. The three input/output (I/O) connections are called '''standard input''' ('''stdin'''), '''standard output''' ('''stdout''') and '''standard error''' ('''stderr'''). Originally I/O happened via a physically connected system console (input via keyboard, output via monitor), but standard streams '''abstract''' this. When a command is executed via an interactive shell, the streams are typically connected to the text terminal on which the shell is running, but can be changed with '''redirection''' or a '''pipeline'''. ''
+
<i>'''Redirection''' can be defined as changing the way from where commands read input to where commands sends output.<br>You can redirect input and output of a command.</i>
  
Reference: https://en.wikipedia.org/wiki/Standard_streams
+
Reference: https://www.javatpoint.com/linux-input-output-redirection
  
 
[[Image:stdin-symbol.png|thumb|right|250px|The '''standard input''' ('''stdin''') symbol that describes where a Unix/Linux command receives '''input''']]
 
[[Image:stdin-symbol.png|thumb|right|250px|The '''standard input''' ('''stdin''') symbol that describes where a Unix/Linux command receives '''input''']]
'''Standard input''' ('''stdin''') is a term which describes from where a command receives input.<br>
+
'''Standard input''' ('''stdin''') is a term which describes from where a command receives '''input'''.<br>
This would apply only to Unix/Linux commands that accept stdin input (like ''cat'', ''more'', ''less'', ''sort'', ''grep'', ''head'', ''tail'', etc.).<br>
+
This would apply only to Unix/Linux commands that accept stdin input<br>(like ''cat'', ''more'', ''less'', ''sort'', ''grep'', ''head'', ''tail'', ''tr'', ''cut'', ''wc'', etc.).<br>
With the examples below, standard input is being sent into the command from the text file.
 
  
 
''Examples:''
 
''Examples:''
Line 75: Line 85:
  
  
<table align="right"><tr><td>[[Image:stdout-symbol-1.png|thumb|right|250px|The '''standard input''' ('''stdin''') symbol with one greater than sign overwrites existing file content with command output]]</td><td>[[Image:stdout-symbol-2.png|thumb|right|250px|The '''standard input''' ('''stdin''') symbol  with two greater than signs add command's output to bottom of existing file's contents.]]</td></tr></table>
+
<table align="right"><tr><td>[[Image:stdout-symbol-1.png|thumb|right|250px|The '''standard input''' ('''stdin''') symbol with one greater than sign '''overwrites''' existing file content with command output]]</td><td>[[Image:stdout-symbol-2.png|thumb|right|250px|The '''standard input''' ('''stdin''') symbol  with two greater than signs '''add''' command's output to '''bottom''' of existing file's contents.]]</td></tr></table>
'''Standard output''' ('''stdout''') describes where a command sends it's output.<br>In the examples below, output from a command is sent to the '''monitor''', unless it is sent to a '''regular file'''.<br>We will discuss another redirection technique called '''pipelines''' later.
+
'''Standard output''' ('''stdout''') describes where a command sends its '''output'''.<br>In the examples below, output from a command is sent to the '''monitor''', unless it is sent to a '''text file'''.
 +
 
  
 
''Examples:''
 
''Examples:''
Line 82: Line 93:
 
<span style="font-family:courier;font-weight:bold">ls -l<br>ls -l > detailed-listing.txt<br>ls /bin >> output.txt</span>
 
<span style="font-family:courier;font-weight:bold">ls -l<br>ls -l > detailed-listing.txt<br>ls /bin >> output.txt</span>
  
<table align="right"><tr><td>[[Image:stderr-symbol-1.png|thumb|right|250px|The '''standard error''' ('''sterr''') symbol with one greater than sign overwrites existing file content with command's error message.]]</td><td>[[Image:stderr-symbol-2.png|thumb|right|250px|The '''standard error''' ('''stderr''') symbol  with two greater than signs add command's error message to bottom of existing file's contents.]]</td></tr></table>
 
'''Standard Error''' ('''stderr''') describes where a command sends it's error messages.<br>In the examples below, error messages from a command are sent to a file or are redirected to a file that acts as a "garbage can".
 
  
''Examples (remember, Unix/Linux is case sensitive and issuing PWD in caps will generate an error message):''
+
<table align="right"><tr><td>[[Image:stderr-symbol-1.png|thumb|right|250px|The '''standard error''' ('''sterr''') symbol with one greater than sign '''overwrites''' existing file content with command's '''error message'''.]]</td><td>[[Image:stderr-symbol-2.png|thumb|right|250px|The '''standard error''' ('''stderr''') symbol  with two greater than signs '''add''' command's error message to '''bottom''' of existing file's contents.]]</td></tr></table>
 +
'''Standard Error''' ('''stderr''') describes where a command sends it's error messages. In the examples below we issue the pwd in capitals on purpose to generate an error message, which can be redirected to a '''text file'''.
 +
 
 +
 
 +
''Examples:''
  
 
<span style="font-family:courier;font-weight:bold">PWD<br>PWD 2> error-message.txt<br>PWD 2 >> error-messages.txt<br>PWD 2> /dev/null</span>
 
<span style="font-family:courier;font-weight:bold">PWD<br>PWD 2> error-message.txt<br>PWD 2 >> error-messages.txt<br>PWD 2> /dev/null</span>
 +
<br><br>
 +
 +
====The /dev/null File====
  
  
===Additional File Manipulation Commands===
+
The '''/dev/null''' file (sometimes called the '''bit bucket''' or  '''black hole''') is a special system file<br>that '''discard''' all  data written into it. This is useful to discard unwanted command output.
 +
 
  
There are some additional regular file manipulation commands that you can use with redirection<br>(in addition to the other regular file manipulation commands introduced in week 2).
+
''Examples:''
  
These commands are displayed in the table below:
+
<span style="font-family:courier;font-weight:bold;">LS 2> /dev/null<br>ls > /dev/null<br>find / -name "tempfile" 2> /dev/null</span>
 +
<br><br>
  
<table cellpadding="5"><tr><th style="border-bottom: 1px solid black;text-align:left;">Linux Command</th><th style="border-bottom: 1px solid black;text-align:left;">Purpose</th></tr><tr valign="top"><td>'''cut'''</td><td>'''Used to extract fields and characters from records. The option '''-c''' option is used to cut by a character or a range of characters. The '''-f''' option indicates the field number or field range to display (this may require using the '''-d''' option to indicate the field separator (delimiter).'''<br><br>''Examples:''<br><span style="font-family:courier;font-weight:bold;">cut -f2 filename</span> - extract 2nd field from all records in file, using tab as delimiter (default)<br><span style="font-family:courier;font-weight:bold;">cut -d' ' -f2,5 filename</span> - extract 2nd and 5th field, using space as delimiter<br><span style="font-family:courier;font-weight:bold;">cut -d' ' -f1-3,5 filename</span> - extract 1st through 3rd and 5th fields, using space as delimiter<br><span style="font-family:courier;font-weight:bold;">cut -c3-5 filename</span> - extract 3rd to 5th characters</td></tr><tr valign="top"><td>'''tr'''</td><td>'''Used to translate characters to different characters.'''<br><br>''Examples:''<br><span style="font-family:courier;font-weight:bold;">tr a A < filename</span> - translate all characters "a" to "A"<br><span style="font-family:courier;font-weight:bold;">tr "[a-z]" "[A-Z]" < filename</span> - translate lowercase "a" through "z" to uppercase<br><span style="font-family:courier;font-weight:bold;">tr "a-z" "A-Z" < filename</span> - translate lowercase "a" through "z" to uppercase, different syntax (non-System V)<br><span style="font-family:courier;font-weight:bold;">tr ':' ' ' < filename</span> - translate all colons to spaces<br><span style="font-family:courier;font-weight:bold;">tr ' ' '\n' < filename</span>  - translate all spaces to newline characters<br><span style="font-family:courier;font-weight:bold;">tr 'abc' 'A' < filename</span> - translate 'a', 'b', and 'c' to 'A', the last character in the "to" string repeats<br></td></tr><tr valign="top"><td><span style="font-family:courier;font-weight:bold;">wc</span></td><td>'''Displays various counts of the contents of a file.'''<br><br>''Examples:''<br><span style="font-family:courier;font-weight:bold;">wc -l filename</span> - displays number of lines in file<br><span style="font-family:courier;font-weight:bold;">wc -c filename</span> - displays number of characters in file<br><span style="font-family:courier;font-weight:bold;">wc -w filename</span> - displays number of words in fil<br></td></tr></table>
+
====The Here Document====
  
  
The '''/dev/null''' file (sometimes called the '''bit bucket''' or  '''black hole''') is a special system file<br>that discards all  data written into it. This is useful to discard unwanted command output.
+
[[Image:Here Document.png|thumb|right|175px|The '''Here Document''' allows a user to redirect stdin from <u>within</u> the command itself.]]
 +
In Linux, the '''Here Document''' allows a user to redirect stdin from within the command itself.
  
 
''Example:''
 
''Example:''
  
<span style="font-family:courier;font-weight:bold;">find / -name "tempfile" 2> /dev/null</span>
+
<span style="font-family:courier;font-weight:bold">cat <<+<br>Line 1<br>Line 2<br>Line 3<br>+<br><br>
<br><br>
 
  
===Piping (Using Pipes)===
+
===Pipeline Commands===
  
 
[[Image:pipe-diagram-1.png|thumb|right|450px|A '''pipeline command''' sends a command's '''standard output''' directly to '''standard input''' of other command(s) without having to create temporary files.]]
 
[[Image:pipe-diagram-1.png|thumb|right|450px|A '''pipeline command''' sends a command's '''standard output''' directly to '''standard input''' of other command(s) without having to create temporary files.]]
'''Pipeline Command:''' Having commands send their s'''tandard output''' <u>directly</u> to '''standard input''' of other  commands without having to use temporary files.
+
'''Pipeline Command:''' Having commands send their '''standard output''' <u>directly</u><br>to '''standard input''' of other  commands WITHOUT having to use '''temporary''' files.
 +
 
 +
Pipes that are used in a '''pipeline command''' are represented by the '''pipe''' "|" symbol.<br>
  
A few simple commands can be combined to form a more powerful command line.<br>
+
A few simple commands can be '''combined''' to form a more <u>powerful</u> command line.<br>
  
  
Pipes that are used in a pipeline command are represented by the | symbol.
+
Commands to the '''right''' of the pipe symbol are referred to as '''filters'''. They are referred to as ''filters'' since those commands are used to '''modify''' the stdout of the <u>previous command</u>. Many commands can be "piped" together, but these commands (filters) must be chained in a specific order, depending on what you wish to accomplish
  
Commands to the right of the pipe symbol are referred to as '''filters'''. They are referred to as ''filters'' since those commands are used to modify the stdin that was sent from the previous command. Many commands can be "piped" together, but these commands (filters) must be chained in a specific order, depending on what you wish to accomplish
 
  
 
''Examples:''<br>
 
''Examples:''<br>
 
<span style="font-family:courier;font-weight:bold">ls -al | more<br>ls | sort -r<br>ls | sort | more<br>ls -l | cut -d" " -f2 | tr 'a-z' 'A-z"<br>ls | grep Linux | head -5<br>head -7 filename | tail -2</span>  
 
<span style="font-family:courier;font-weight:bold">ls -al | more<br>ls | sort -r<br>ls | sort | more<br>ls -l | cut -d" " -f2 | tr 'a-z' 'A-z"<br>ls | grep Linux | head -5<br>head -7 filename | tail -2</span>  
  
 +
====The tee Command====
  
[[Image:tee-diagram-1.png|thumb|right|250px|The '''tee''' utility can be used to split the flow of information. For example to save in a file as well as display on a screen. <br>(Image licensed under [https://creativecommons.org/licenses/by-sa/3.0/ cc])]]
+
 
 +
[[Image:tee-diagram-1.png|thumb|right|250px|The '''tee''' utility can be used to '''split''' the flow of information. For example to save in a file as well as display on a screen. <br>(Image licensed under [https://creativecommons.org/licenses/by-sa/3.0/ cc])]]
 
The '''tee''' utility can be used to split the flow of information. The tee option '''-a''' can be used to add content to the bottom of an existing file as opposed to overwriting the file's previous contents.
 
The '''tee''' utility can be used to split the flow of information. The tee option '''-a''' can be used to add content to the bottom of an existing file as opposed to overwriting the file's previous contents.
  
The reason for the name "tee" is that the splitting of the flow of information resembles a capital T.
+
The reason for the name "'''tee'''" is that the splitting of the flow of information resembles a capital T.
 +
 
  
 
''Examples:''
 
''Examples:''
Line 138: Line 160:
 
''Example:''
 
''Example:''
  
<span style="font-family:courier;font-weight:bold">sleep 5; ls</span>  (''Note: each command will be executed when the previous command has terminated'')
+
<span style="font-family:courier;font-weight:bold">sleep 5; ls</span>   
  
  
Line 146: Line 168:
 
''Example:''
 
''Example:''
  
<span style="font-family:courier;font-weight:bold">(echo "Who is on:"; w) > whoson</span>
+
<span style="font-family:courier;font-weight:bold">(echo "Who is on:"; w) > whoson</span><br>('''''Note:''' <u>all</u> command output is sent to a file'')
  
  
Commands may also be '''split over multiple lines''', making it easier (for humans) to interpret a long command.<br>
+
Commands may also be '''spread-out over multiple lines''', making it easier (for humans) to interpret a long command.<br>
You can add a '''quote''' or '''"escape" the newline character''' at the end of a line, to get rid of the special meaning<br>of newline (to end a command line)
+
You can add a '''backslash''' symbol "\" at the end of a line, to get rid of the special meaning<br>of newline (to end a command line)
  
  
Line 161: Line 183:
  
 
<br>
 
<br>
In this section, you will learn how to redirect standard input, standard output and standard error when issuing Unix / Linux commands.
+
In this section, you will learn how to redirect '''standard input''', '''standard output''' and '''standard error''' when issuing Unix / Linux commands.
  
  
 
'''Perform the Following Steps:'''
 
'''Perform the Following Steps:'''
  
# '''Login''' your matrix account and issue a command to '''confirm''' you are located in your '''home''' directory.<br><br>
+
# '''Login''' to your matrix account and issue a command to '''confirm''' you are located in your '''home''' directory.<br><br>
# Issue the following Linux command to create the directory: <span style="color:blue;font-weight:bold;font-family:courier;">mkdir ~/redirect</span><br><br>
+
# Issue the following Linux command to create the following directory: <span style="color:blue;font-weight:bold;font-family:courier;">mkdir ~/redirect</span><br><br>
 
# Change to the '''~/redirect''' directory and confirm that you changed to that directory.<br><br>
 
# Change to the '''~/redirect''' directory and confirm that you changed to that directory.<br><br>
 
# Use a text editor to create a file in your current directory called '''data.txt''' and enter the following text displayed below:<br><br><span style="font-family:courier;font-weight:bold">This is line 1<br>This is line 2<br>This is line 3</span><br><br>
 
# Use a text editor to create a file in your current directory called '''data.txt''' and enter the following text displayed below:<br><br><span style="font-family:courier;font-weight:bold">This is line 1<br>This is line 2<br>This is line 3</span><br><br>
# Save editing changes and exit the text editor.<br><br>
+
# '''Save''' editing changes and '''exit''' the text editor.<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' < data.txt</span><br><br>What this command do?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' < data.txt</span><br><br>What does this command do?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tail -2 data.txt > output.txt</span><br><br>What does this command do? Check the contents of the '''output.txt''' file to confirm.<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' < data.txt > output.txt</span><br><br>What does this command do? What are the contents of the file ''output.txt''?<br><br>
 +
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' > output.txt < data.txt</span><br><br>What does this command do? Is there any difference in terms of this command and the previous command issued?<br><br>
 +
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' >> output.txt < data.txt</span><br><br>What happens to the content of the '''output.txt''' file? Why?<br><br>
 +
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tail -2 < data.txt > output.txt</span><br><br>What does this command do? Check the contents of the '''output.txt''' file to confirm.<br><br>
 
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tail -2 > output2.txt < data.txt </span><br><br>Why does this command render the same results as the previous command?<br>Try explaining how the command works in terms of '''stdin''' and then '''stdout'''.<br><br>
 
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tail -2 > output2.txt < data.txt </span><br><br>Why does this command render the same results as the previous command?<br>Try explaining how the command works in terms of '''stdin''' and then '''stdout'''.<br><br>
 
# Issue the following Linux command to create a file: <span style="color:blue;font-weight:bold;font-family:courier;">cat > output3.txt </span><br><br>
 
# Issue the following Linux command to create a file: <span style="color:blue;font-weight:bold;font-family:courier;">cat > output3.txt </span><br><br>
Line 180: Line 205:
 
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cp ~murray.saul/uli101/cars .</span><br><br>
 
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cp ~murray.saul/uli101/cars .</span><br><br>
 
# Issue the '''cat''' command to view the contents of the '''cars''' file.<br><br>
 
# Issue the '''cat''' command to view the contents of the '''cars''' file.<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -c1-10 uli101/cars</span><br><br>What did this command do?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -c1-10 cars</span><br><br>What did this command do?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f5 uli101/cars > field5.txt</span><br><br>What did this command do?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f5 cars > field5.txt</span><br><br>What did this command do?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f1-3 uli101/cars > field123.txt</span><br><br>What did this command do?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f1-3 cars > field123.txt</span><br><br>What did this command do?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f1,5 uli101/cars > field15.txt</span><br><br>What did this command do?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f1,5 cars > field15.txt</span><br><br>What did this command do?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc cars > count1.txt</span><br><br>What information does the count1.txt file contain?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc cars > count.txt</span><br><br>What information does the '''count.txt''' file contain?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc cars > count2.txt</span><br><br>What information does the count2.txt file contain?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc =l cars > count1.txt</span><br><br>What information does the '''count1.txt''' file contain?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">ls -l > listing.txt</span><br><br>What information does the count2.txt file contain?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc -w cars > count2.txt</span><br><br>What information does the '''count2.txt''' file contain?<br><br>
 +
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">ls -l > listing.txt</span><br><br>What information does the '''count2.txt''' file contain?<br><br>
 
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">pwd > listing.txt</span><br><br>What happenned to the original contents of the file called '''listing.txt'''?<br><br>
 
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">pwd > listing.txt</span><br><br>What happenned to the original contents of the file called '''listing.txt'''?<br><br>
 
# Issue the following Linux command (use 2 greater-than signs): <span style="color:blue;font-weight:bold;font-family:courier;">date >> listing.txt</span><br><br>What information does the '''listing.txt''' file contain?<br><br>
 
# Issue the following Linux command (use 2 greater-than signs): <span style="color:blue;font-weight:bold;font-family:courier;">date >> listing.txt</span><br><br>What information does the '''listing.txt''' file contain?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray 2> result.txt</span><br><br>What information does the '''result.txt''' file contain?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars > combined.txt</span><br><br>What information does the '''combined.txt''' file contain?<br><br>'''NOTE''': The '''cat''' command stands for "'''concatenate'''" which means to '''combine''' contents of multiple files into a single file.<br>This is why the command is called "''cat''".<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray > myoutput.txt 2> result.txt</span><br><br>What is displayed on the monitor? what do those files contain?<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray 2> result.txt</span><br><br>What is displayed on the monitor? What information does the '''result.txt''' file contain?<br><br>
# Make certain that your current directory is '''your home directory'''.<br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray > myoutput.txt 2> /dev/null</span><br><br>What is displayed on the monitor? What happened to the error message?<br><br>
# Issue the following Linux command run checking script:<br><span style="color:blue;font-weight:bold;font-family:courier;">bash /home/murray.saul/scripts/week5-check-1</span><br><br>
+
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray > myoutput.txt 2> result.txt</span><br><br>What is displayed on the monitor? what do those files contain?<br><br>The '''Here Document''' allows you to redirect stdin from with the Linux command itself. Let's get some practice using the Here Document.<br><br>
# If you encounter errors and re-run the checking script until you receive a congratulations message, and proceed to the next section.<br><br>
+
# Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">cat <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice?<br><br>
 +
# Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">grep 2 <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice? How does this differ from the previous command? Why?<br><br>
 +
# Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">grep 2 > line2.txt <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice? What is contained in the file '''line2.txt'''? Why?<br><br>'''NOTE:''' You will now run a shell script to confirm that you properly issued Linux commands using redirection.<br><br>
 +
# Issue the following Linux command to run a checking script:<br><span style="color:blue;font-weight:bold;font-family:courier;">bash /home/murray.saul/scripts/week5-check-1</span><br><br>
 +
# If you encounter errors, make corrections and '''re-run''' the checking script until you receive a congratulations message, then you can proceed.<br><br>
 +
# Issue the '''ls''' command to see all of the '''temporary files''' that were created as a result of redirection.<br><br>The problem with using these redirection symbols is that you create '''temporary text files''' that take up '''space''' on your file system.<br><br>
 +
# Issue a Linux command to '''remove''' those temporary text files in the current directory.<br><br>
 +
# Issue the following Linux command to check that you removed ALL of those temporary text files:<br><span style="color:blue;font-weight:bold;font-family:courier;">bash /home/murray.saul/scripts/week5-check-2</span><br><br>
 +
# If you encounter errors, make corrections and '''re-run''' the checking script until you receive a congratulations message, then you can proceed.<br><br>
  
The problem with using redirection to create files, you have these files taking up space, which requires you remove them. In the next investigation, you will be learning how to issue pipeline commands which can provide information by issuing several Linux commands without creating temporary files.<br><br>
+
:In the next investigation, you will be learning how to issue '''pipeline Linux commands''' which can accomplish tasks <u>without</u> creating temporary files.<br><br>
  
=INVESTIGATION 2: REDIRECTION USING PIPES =
+
=INVESTIGATION 2: REDIRECTION USING PIPELINE COMMANDS =
  
In this section, you will learn to issue pipeline commands and learn how to perform tasks<br>using Linux commands with or without the generation of temporary files.
+
In this section, you will learn to issue '''pipeline commands''' to to accomplish tasks <u>without</u> having to generate temporary files.
  
  
Line 205: Line 239:
 
'''Perform the Following Steps:'''
 
'''Perform the Following Steps:'''
  
# x<br><br>
+
# Confirm that you are still located in the '''~/redirect''' directory.<br><br>
 +
# Issue the '''ls''' command to view the contents of your '''~/redirect''' directory.<br><br>The '''problem''' with creating temporary files, is that they take up space on your server,<br>and should be removed.<br><br>
 +
# Issue the following Linux command to remove all temporary files in your ''redirect'' directory: <span style="color:blue;font-weight:bold;font-family:courier;">rm  -r ~/redirect</span><br>and confirm that you have removed this directory and its contents.<br><br>'''NOTE:''' You will be issuing a '''pipeline command''' which will use the pipe symbol "|"<br>that will send the stdout from a command as stdin into another command without having to create temporary files.<br><br>
 +
# Issue the follow Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin | more</span><br><br>What happened?<br><br>
 +
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin | who</span><br><br>What happened? Why is the result different than antipated?[[Image:pipe-diagram-1.png|thumb|right|350px|]]<br><br>'''NOTE:''' When issuing pipeline commands, commands to the right of the pipe symbol must be designed to <u>accept</u> '''standard input'''. Since the ''who'' command does not, you did NOT see the contents of the '''/bin''' directory but only information relating to the ''who'' command. Therefore, the '''order''' of which you build your pipeline command and the '''type of command''' that is used as a ''filter'' is extremely important!<br><br>
 +
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? > listing.txt</span><br><br>
 +
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">sort -r listing.txt</span><br><br>
 +
# Issue the following Linux command to remove the listing file: <span style="color:blue;font-weight:bold;font-family:courier;">rm listing.txt</span><br><br>
 +
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort -r</span><br><br>You should notice that the output from this pipeline command is the same output<br>from the command you issued in '''step #7'''<br><br>
 +
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort -r | more</span><br><br>What is different with this pipeline command as opposed to the <u>previous</u> pipeline command?<br><br>
 +
# Issue the '''ls''' command.<br><br>You should notice that '''no files have been created'''.<br>Let's get practice issuing more pipeline commands using commands<br>(previously learned or new) to be used as filters.<br><br>
 +
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort -r | head -5</span><br><br>What did you notice?<br><br>
 +
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | sort -r | grep r | tail -2</span><br><br>What did you notice?<br><br>
 +
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | sort -r | grep r | cut -c1-3</span><br><br>
 +
# Issue the following Linux '''pipeline command''':<br><span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | tee unsorted.txt | sort -r | tee sorted.txt | tee unmatched.txt | grep r | tail -2</span><br><br>What did you notice?<br><br>
 +
# Check the files that were created to see how the '''tee''' command was used in the previous pipeline command.<br><br>
 +
# Issue the '''rm''' command to <u>only</u> remove the temporary files called: '''unsorted.txt''' , '''sorted.txt''' , and '''unmatched.txt'''<br><br>
  
In the next investigation, you will ...
+
:In the next investigation, you will learn various techniques to issue multiple Linux commands on the same line,<br>or long Linux commands over multiple lines.
 +
<br><br>
  
 
=INVESTIGATION 3: ISSUING MULTIPLE UNIX/LINUX COMMANDS=
 
=INVESTIGATION 3: ISSUING MULTIPLE UNIX/LINUX COMMANDS=
  
In this section, you will learn how to ...
+
In this section, you will learn how to issue multiple Unix / Linux commands in a single line or over multiple lines.
 
 
  
  
 
'''Perform the Following Steps:'''
 
'''Perform the Following Steps:'''
  
# x<br>
+
# Issue a Linux command to change to your home directory.<br><br>
 +
# Confirm you are located in your '''home''' directory.<br><br>
 +
# Issue the following Linux commands (using semicolon to separate each command): <span style="color:blue;font-weight:bold;font-family:courier;">cal;pwd;date</span><br><br>Note the from the output the order of how each of those commands were processed.<br><br>
 +
# Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">(cal;pwd;date)</span><br><br>Was there any difference in the output of this command as opposed to the previous command?<br><br>Let's see how grouping affects working with redirection.<br><br>
 +
# Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">cal;pwd;date > output.txt</span><br><br>
 +
# Issue a Linux command to view the contents of the file called '''output.txt'''<br><br>What do you notice?<br><br>Let's use grouping to make modification to the previous command<br><br>
 +
# Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">(cal;pwd;date) > output.txt</span><br><br>
 +
# Issue a Linux command to view the contents of the file called '''output.txt'''<br><br>What do you notice? What did grouping the three Linux commands do?<br><br>
 +
# Issue the following Linux pipeline command (using \ at the end of most lines):<br><span style="color:blue;font-family:courier;font-weight:bold">echo "This will be split over multiple \<br>lines.  Note that the shell will realize \<br>that a pipe requires another command, so \<br>it will automatically go to the next line" |tr '[a-z]' '[A-Z]'</span><br><br>Did the command work? What does this command do?<br><br>
 +
# After you complete the Review Questions sections to get additional practice, then work on your '''online assignment 2'''<br>and complete '''section3''' labelled: '''Redirection and Pipes'''.
 +
<br><br>
  
 
= LINUX PRACTICE QUESTIONS =
 
= LINUX PRACTICE QUESTIONS =
Line 230: Line 290:
 
Your instructor may take-up these questions during class. It is up to the student to attend classes in order to obtain the answers to the following questions. Your instructor will NOT provide these answers in any other form (eg. e-mail, etc).
 
Your instructor may take-up these questions during class. It is up to the student to attend classes in order to obtain the answers to the following questions. Your instructor will NOT provide these answers in any other form (eg. e-mail, etc).
  
When answering Linux command questions, refer to the following Inverted Tree diagram. The linux directory is contained in your home directory. Assume that you just logged into your Matrix account. Directories are <u>underlined</u>.
+
When answering Linux command questions, refer to the following Inverted Tree Diagram. The linux directory is contained in your home directory. Assume that you just logged into your Matrix account. Directories are <u>underlined</u>.
  
 
[[Image:week5-dir.png|thumb|left|300px|]]
 
[[Image:week5-dir.png|thumb|left|300px|]]
Line 239: Line 299:
 
'''Review Questions:'''
 
'''Review Questions:'''
  
# Write a single Linux command to provide a detailed listing of all files in the /bin directory, sending the output to a file called listing.txt in the “projects” directory (append output to existing file and use a relative pathname)
+
# Write a single Linux command to provide a detailed listing of all files in the '''/bin''' directory, sending the output to a file called listing.txt in the “'''projects'''” directory (append output to existing file and use a relative pathname)
# Write a single Linux command to redirect the stderr from the command:<br>cat a.txt b.txt c.txt to a file called error.txt contained in the “assignments” directory. (overwrite previous file’s contents and use only relative pathnames)  
+
# Write a single Linux command to redirect the stderr from the command:<br>'''cat a.txt b.txt c.txt''' to a file called '''error.txt''' contained in the “'''assignments'''” directory. (overwrite previous file’s contents and use only relative pathnames)  
# Write a single Linux command: cat ~/a.txt ~/b.txt ~/c.txt and redirect stdout to a file called “good.txt” to the “tests” directory and stderr to a file called “bad.txt” to the “tests” directory. (overwrite previous contents for both files and use only relative-to-home pathnames).
+
# Write a single Linux command: '''cat ~/a.txt ~/b.txt ~/c.txt''' and redirect stdout to a file called “good.txt” to the “tests” directory and stderr to a file called “'''bad.txt'''” to the “'''tests'''” directory. (overwrite previous contents for both files and use only relative-to-home pathnames).
# Write a single Linux command to redirect the stdout from the command:<br>cat a.txt b.txt c.txt to a file called wrong.txt contained in the “projects” directory and throw-out any standard error messages so they don’t appear on the screen (append output to existing file and use only relative pathnames).<br><br>
+
# Write a single Linux command to redirect the stdout from the command:<br>'''cat a.txt b.txt c.txt''' to a file called wrong.txt contained in the “'''projects'''” directory and throw-out any standard error messages so they don’t appear on the screen (append output to existing file and use only relative pathnames).<br><br>
# Write a single Linux pipeline command to display a detailed listing of the “projects “directory but pause one screen at a time to view and navigate through all of the directory contents. Use a relative-to-home pathname.
+
# Write a single Linux '''pipeline command''' to display a detailed listing of the '''projects''' directory but pause one screen at a time to view and navigate through all of the directory contents. Use a relative-to-home pathname.
# Write a single Linux pipeline command to display the sorted contents (in reverse alphabetical order) of the “linux” directory. Use a relative pathname.
+
# Write a single Linux '''pipeline command''' to display the sorted contents (in reverse alphabetical order) of the “'''linux'''” directory. Use a relative pathname.
# Assume that the text file called “.answers.txt” contains 10 lines. Write a single Linux pipeline command to only displays lines 5 through 8 for this file. Use only relative pathnames.
+
# Assume that the text file called “'''.answers.txt'''” contains 10 lines. Write a single Linux pipeline command to only displays lines 5 through 8 for this file. Use only relative pathnames.
# Write a single Linux pipeline command to only display the contents of the “assignments” directory whose filenames match the pattern “murray” (both upper or lowercase). Use an absolute pathname.
+
# Write a single Linux '''pipeline command''' to only display the contents of the “'''assignments'''” directory whose filenames match the pattern “'''murray'''” (both upper or lowercase). Use an absolute pathname.
# Write a single Linux pipeline command to display the number of characters contained in the file called “.answers.txt”. Use a relative-to-home pathname.
+
# Write a single Linux '''pipeline command''' to display the number of characters contained in the file called “'''.answers.txt'''”. Use a relative-to-home pathname.
# Write a single Linux pipeline command to display the number of lines contained in the file called “questions.txt”. Use a relative pathname.
+
# Write a single Linux '''pipeline command''' to display the number of lines contained in the file called “'''questions.txt'''”. Use a relative pathname.
# Write a single Linux pipeline command to display only the first 10 characters of each filename contained in your current directory. Also, there is will be a lot of output, so also pause at each screenful so you can navigate throughout the display contents. Use a relative pathname.
+
# Write a single Linux '''pipeline command''' to display only the first 10 characters of each filename contained in your current directory. Also, there is will be a lot of output, so also pause at each screenful so you can navigate throughout the display contents. Use a relative pathname.
 
# Create a '''table''' listing each Linux command, useful options that were mentioned in this tutorial for the following Linux commands: '''cut''' , '''tr''' , '''wc''' , and '''tee'''.
 
# Create a '''table''' listing each Linux command, useful options that were mentioned in this tutorial for the following Linux commands: '''cut''' , '''tr''' , '''wc''' , and '''tee'''.
  
  
 
[[Category:ULI101]]
 
[[Category:ULI101]]

Latest revision as of 09:02, 20 January 2021

REDIRECTION: STANDARD INPUT / STANDARD OUTPUT / STANDARD ERROR


Main Objectives of this Practice Tutorial

  • Understand and use the cut, tr, and wc Linux commands
  • Define the terms Standard Input (stdin), Standard Output (stdout), and Standard Error (stderr)
  • Understand and use the >, >>, 2>, 2>> symbols with Linux commands
  • Understand the purpose of the the /dev/null file and the Here Document
  • Define the term pipeline command and explain how a pipeline command functions
  • Define the term filter and how it relates to pipeline commands
  • Use the semicolon ";" and grouping "( )" symbols to issue multiple Unix / Linux commands on a single line
  • Use the backslash "\" symbol to spread-out long Unix/Linux commands over multiple lines


Tutorial Reference Material

Course Notes
Linux Command / Shortcut Reference
YouTube Videos
Course Notes:


Redirection

Multiple Commands

Redirection Filters Brauer Instructional Videos:

KEY CONCEPTS

Additional File Manipulation Commands

Before proceeding, let's look at some additional commands used to manipulate content of text files.

Refer to the table below regarding these text file manipulation commands:

Manipulation-commands.png










Redirection (Standard Input, Standard Output, Standard Error)

Redirection can be defined as changing the way from where commands read input to where commands sends output.
You can redirect input and output of a command.

Reference: https://www.javatpoint.com/linux-input-output-redirection

The standard input (stdin) symbol that describes where a Unix/Linux command receives input

Standard input (stdin) is a term which describes from where a command receives input.
This would apply only to Unix/Linux commands that accept stdin input
(like cat, more, less, sort, grep, head, tail, tr, cut, wc, etc.).

Examples:

tr 'a-z' 'A-Z' < words.txt
cat < abc.txt
sort < xyz.txt


The standard input (stdin) symbol with one greater than sign overwrites existing file content with command output
The standard input (stdin) symbol with two greater than signs add command's output to bottom of existing file's contents.

Standard output (stdout) describes where a command sends its output.
In the examples below, output from a command is sent to the monitor, unless it is sent to a text file.


Examples:

ls -l
ls -l > detailed-listing.txt
ls /bin >> output.txt


The standard error (sterr) symbol with one greater than sign overwrites existing file content with command's error message.
The standard error (stderr) symbol with two greater than signs add command's error message to bottom of existing file's contents.

Standard Error (stderr) describes where a command sends it's error messages. In the examples below we issue the pwd in capitals on purpose to generate an error message, which can be redirected to a text file.


Examples:

PWD
PWD 2> error-message.txt
PWD 2 >> error-messages.txt
PWD 2> /dev/null


The /dev/null File

The /dev/null file (sometimes called the bit bucket or black hole) is a special system file
that discard all data written into it. This is useful to discard unwanted command output.


Examples:

LS 2> /dev/null
ls > /dev/null
find / -name "tempfile" 2> /dev/null


The Here Document

The Here Document allows a user to redirect stdin from within the command itself.

In Linux, the Here Document allows a user to redirect stdin from within the command itself.

Example:

cat <<+
Line 1
Line 2
Line 3
+

Pipeline Commands

A pipeline command sends a command's standard output directly to standard input of other command(s) without having to create temporary files.

Pipeline Command: Having commands send their standard output directly
to standard input of other commands WITHOUT having to use temporary files.

Pipes that are used in a pipeline command are represented by the pipe "|" symbol.

A few simple commands can be combined to form a more powerful command line.


Commands to the right of the pipe symbol are referred to as filters. They are referred to as filters since those commands are used to modify the stdout of the previous command. Many commands can be "piped" together, but these commands (filters) must be chained in a specific order, depending on what you wish to accomplish


Examples:
ls -al | more
ls | sort -r
ls | sort | more
ls -l | cut -d" " -f2 | tr 'a-z' 'A-z"
ls | grep Linux | head -5
head -7 filename | tail -2

The tee Command

The tee utility can be used to split the flow of information. For example to save in a file as well as display on a screen.
(Image licensed under cc)

The tee utility can be used to split the flow of information. The tee option -a can be used to add content to the bottom of an existing file as opposed to overwriting the file's previous contents.

The reason for the name "tee" is that the splitting of the flow of information resembles a capital T.


Examples: ls | tee unsorted.txt | sort
ls | grep Linux | tee matched.txt | more
ls | head -5 | tee -a listing.txt


Multiple Commands Using Semicolon, Grouping, and Backquotes

Besides piping, there are other ways that multiple commands may be placed in one line:
commands may be separated by semi-colons.


Example:

sleep 5; ls


Multiple commands can also be grouped by using parentheses.


Example:

(echo "Who is on:"; w) > whoson
(Note: all command output is sent to a file)


Commands may also be spread-out over multiple lines, making it easier (for humans) to interpret a long command.
You can add a backslash symbol "\" at the end of a line, to get rid of the special meaning
of newline (to end a command line)


Example:

echo "This will be split over multiple \
lines. Note that the shell will realize \
that a pipe requires another command, so \
it will automatically go to the next line" |tr '[a-z]' '[A-Z]'


INVESTIGATION 1: BASICS OF REDIRECTION


In this section, you will learn how to redirect standard input, standard output and standard error when issuing Unix / Linux commands.


Perform the Following Steps:

  1. Login to your matrix account and issue a command to confirm you are located in your home directory.

  2. Issue the following Linux command to create the following directory: mkdir ~/redirect

  3. Change to the ~/redirect directory and confirm that you changed to that directory.

  4. Use a text editor to create a file in your current directory called data.txt and enter the following text displayed below:

    This is line 1
    This is line 2
    This is line 3


  5. Save editing changes and exit the text editor.

  6. Issue the following Linux command: tr 'a-z' 'A-Z' < data.txt

    What does this command do?

  7. Issue the following Linux command: tr 'a-z' 'A-Z' < data.txt > output.txt

    What does this command do? What are the contents of the file output.txt?

  8. Issue the following Linux command: tr 'a-z' 'A-Z' > output.txt < data.txt

    What does this command do? Is there any difference in terms of this command and the previous command issued?

  9. Issue the following Linux command: tr 'a-z' 'A-Z' >> output.txt < data.txt

    What happens to the content of the output.txt file? Why?

  10. Issue the following Linux command: tail -2 < data.txt > output.txt

    What does this command do? Check the contents of the output.txt file to confirm.

  11. Issue the following Linux command: tail -2 > output2.txt < data.txt

    Why does this command render the same results as the previous command?
    Try explaining how the command works in terms of stdin and then stdout.

  12. Issue the following Linux command to create a file: cat > output3.txt

  13. Enter the follow text displayed below:

    This is the file output3.txt

  14. Press ctrl-d to exit the command.

  15. Issue the cat command to view the contents of the file: output3.txt

  16. Issue the following Linux command: cp ~murray.saul/uli101/cars .

  17. Issue the cat command to view the contents of the cars file.

  18. Issue the following Linux command: cut -c1-10 cars

    What did this command do?

  19. Issue the following Linux command: cut -f5 cars > field5.txt

    What did this command do?

  20. Issue the following Linux command: cut -f1-3 cars > field123.txt

    What did this command do?

  21. Issue the following Linux command: cut -f1,5 cars > field15.txt

    What did this command do?

  22. Issue the following Linux command: wc cars > count.txt

    What information does the count.txt file contain?

  23. Issue the following Linux command: wc =l cars > count1.txt

    What information does the count1.txt file contain?

  24. Issue the following Linux command: wc -w cars > count2.txt

    What information does the count2.txt file contain?

  25. Issue the following Linux command: ls -l > listing.txt

    What information does the count2.txt file contain?

  26. Issue the following Linux command: pwd > listing.txt

    What happenned to the original contents of the file called listing.txt?

  27. Issue the following Linux command (use 2 greater-than signs): date >> listing.txt

    What information does the listing.txt file contain?

  28. Issue the following Linux command: cat listing.txt cars > combined.txt

    What information does the combined.txt file contain?

    NOTE: The cat command stands for "concatenate" which means to combine contents of multiple files into a single file.
    This is why the command is called "cat".

  29. Issue the following Linux command: cat listing.txt cars murray 2> result.txt

    What is displayed on the monitor? What information does the result.txt file contain?

  30. Issue the following Linux command: cat listing.txt cars murray > myoutput.txt 2> /dev/null

    What is displayed on the monitor? What happened to the error message?

  31. Issue the following Linux command: cat listing.txt cars murray > myoutput.txt 2> result.txt

    What is displayed on the monitor? what do those files contain?

    The Here Document allows you to redirect stdin from with the Linux command itself. Let's get some practice using the Here Document.

  32. Issue the following Linux command:
    cat <<+
    line 1
    line 2
    line 3
    +


    What do you notice?

  33. Issue the following Linux command:
    grep 2 <<+
    line 1
    line 2
    line 3
    +


    What do you notice? How does this differ from the previous command? Why?

  34. Issue the following Linux command:
    grep 2 > line2.txt <<+
    line 1
    line 2
    line 3
    +


    What do you notice? What is contained in the file line2.txt? Why?

    NOTE: You will now run a shell script to confirm that you properly issued Linux commands using redirection.

  35. Issue the following Linux command to run a checking script:
    bash /home/murray.saul/scripts/week5-check-1

  36. If you encounter errors, make corrections and re-run the checking script until you receive a congratulations message, then you can proceed.

  37. Issue the ls command to see all of the temporary files that were created as a result of redirection.

    The problem with using these redirection symbols is that you create temporary text files that take up space on your file system.

  38. Issue a Linux command to remove those temporary text files in the current directory.

  39. Issue the following Linux command to check that you removed ALL of those temporary text files:
    bash /home/murray.saul/scripts/week5-check-2

  40. If you encounter errors, make corrections and re-run the checking script until you receive a congratulations message, then you can proceed.

In the next investigation, you will be learning how to issue pipeline Linux commands which can accomplish tasks without creating temporary files.

INVESTIGATION 2: REDIRECTION USING PIPELINE COMMANDS

In this section, you will learn to issue pipeline commands to to accomplish tasks without having to generate temporary files.


Perform the Following Steps:

  1. Confirm that you are still located in the ~/redirect directory.

  2. Issue the ls command to view the contents of your ~/redirect directory.

    The problem with creating temporary files, is that they take up space on your server,
    and should be removed.

  3. Issue the following Linux command to remove all temporary files in your redirect directory: rm -r ~/redirect
    and confirm that you have removed this directory and its contents.

    NOTE: You will be issuing a pipeline command which will use the pipe symbol "|"
    that will send the stdout from a command as stdin into another command without having to create temporary files.

  4. Issue the follow Linux pipeline command: ls /bin | more

    What happened?

  5. Issue the following Linux pipeline command: ls /bin | who

    What happened? Why is the result different than antipated?
    Pipe-diagram-1.png


    NOTE: When issuing pipeline commands, commands to the right of the pipe symbol must be designed to accept standard input. Since the who command does not, you did NOT see the contents of the /bin directory but only information relating to the who command. Therefore, the order of which you build your pipeline command and the type of command that is used as a filter is extremely important!

  6. Issue the following Linux command: ls /bin/?? > listing.txt

  7. Issue the following Linux command: sort -r listing.txt

  8. Issue the following Linux command to remove the listing file: rm listing.txt

  9. Issue the following Linux pipeline command: ls /bin/?? | sort -r

    You should notice that the output from this pipeline command is the same output
    from the command you issued in step #7

  10. Issue the following Linux pipeline command: ls /bin/?? | sort -r | more

    What is different with this pipeline command as opposed to the previous pipeline command?

  11. Issue the ls command.

    You should notice that no files have been created.
    Let's get practice issuing more pipeline commands using commands
    (previously learned or new) to be used as filters.

  12. Issue the following Linux pipeline command: ls /bin/?? | sort -r | head -5

    What did you notice?

  13. Issue the following Linux pipeline command: ls /bin/???? | sort -r | grep r | tail -2

    What did you notice?

  14. Issue the following Linux pipeline command: ls /bin/???? | sort -r | grep r | cut -c1-3

  15. Issue the following Linux pipeline command:
    ls /bin/???? | tee unsorted.txt | sort -r | tee sorted.txt | tee unmatched.txt | grep r | tail -2

    What did you notice?

  16. Check the files that were created to see how the tee command was used in the previous pipeline command.

  17. Issue the rm command to only remove the temporary files called: unsorted.txt , sorted.txt , and unmatched.txt

In the next investigation, you will learn various techniques to issue multiple Linux commands on the same line,
or long Linux commands over multiple lines.



INVESTIGATION 3: ISSUING MULTIPLE UNIX/LINUX COMMANDS

In this section, you will learn how to issue multiple Unix / Linux commands in a single line or over multiple lines.


Perform the Following Steps:

  1. Issue a Linux command to change to your home directory.

  2. Confirm you are located in your home directory.

  3. Issue the following Linux commands (using semicolon to separate each command): cal;pwd;date

    Note the from the output the order of how each of those commands were processed.

  4. Issue the following Linux commands: (cal;pwd;date)

    Was there any difference in the output of this command as opposed to the previous command?

    Let's see how grouping affects working with redirection.

  5. Issue the following Linux commands: cal;pwd;date > output.txt

  6. Issue a Linux command to view the contents of the file called output.txt

    What do you notice?

    Let's use grouping to make modification to the previous command

  7. Issue the following Linux commands: (cal;pwd;date) > output.txt

  8. Issue a Linux command to view the contents of the file called output.txt

    What do you notice? What did grouping the three Linux commands do?

  9. Issue the following Linux pipeline command (using \ at the end of most lines):
    echo "This will be split over multiple \
    lines. Note that the shell will realize \
    that a pipe requires another command, so \
    it will automatically go to the next line" |tr '[a-z]' '[A-Z]'


    Did the command work? What does this command do?

  10. After you complete the Review Questions sections to get additional practice, then work on your online assignment 2
    and complete section3 labelled: Redirection and Pipes.



LINUX PRACTICE QUESTIONS

The purpose of this section is to obtain extra practice to help with quizzes, your midterm, and your final exam.

Here is a link to the MS Word Document of ALL of the questions displayed below but with extra room to answer on the document to simulate a quiz:

https://ict.senecacollege.ca/~murray.saul/uli101/uli101_week5_practice.docx

Your instructor may take-up these questions during class. It is up to the student to attend classes in order to obtain the answers to the following questions. Your instructor will NOT provide these answers in any other form (eg. e-mail, etc).

When answering Linux command questions, refer to the following Inverted Tree Diagram. The linux directory is contained in your home directory. Assume that you just logged into your Matrix account. Directories are underlined.

Week5-dir.png











Review Questions:

  1. Write a single Linux command to provide a detailed listing of all files in the /bin directory, sending the output to a file called listing.txt in the “projects” directory (append output to existing file and use a relative pathname)
  2. Write a single Linux command to redirect the stderr from the command:
    cat a.txt b.txt c.txt to a file called error.txt contained in the “assignments” directory. (overwrite previous file’s contents and use only relative pathnames)
  3. Write a single Linux command: cat ~/a.txt ~/b.txt ~/c.txt and redirect stdout to a file called “good.txt” to the “tests” directory and stderr to a file called “bad.txt” to the “tests” directory. (overwrite previous contents for both files and use only relative-to-home pathnames).
  4. Write a single Linux command to redirect the stdout from the command:
    cat a.txt b.txt c.txt to a file called wrong.txt contained in the “projects” directory and throw-out any standard error messages so they don’t appear on the screen (append output to existing file and use only relative pathnames).

  5. Write a single Linux pipeline command to display a detailed listing of the projects directory but pause one screen at a time to view and navigate through all of the directory contents. Use a relative-to-home pathname.
  6. Write a single Linux pipeline command to display the sorted contents (in reverse alphabetical order) of the “linux” directory. Use a relative pathname.
  7. Assume that the text file called “.answers.txt” contains 10 lines. Write a single Linux pipeline command to only displays lines 5 through 8 for this file. Use only relative pathnames.
  8. Write a single Linux pipeline command to only display the contents of the “assignments” directory whose filenames match the pattern “murray” (both upper or lowercase). Use an absolute pathname.
  9. Write a single Linux pipeline command to display the number of characters contained in the file called “.answers.txt”. Use a relative-to-home pathname.
  10. Write a single Linux pipeline command to display the number of lines contained in the file called “questions.txt”. Use a relative pathname.
  11. Write a single Linux pipeline command to display only the first 10 characters of each filename contained in your current directory. Also, there is will be a lot of output, so also pause at each screenful so you can navigate throughout the display contents. Use a relative pathname.
  12. Create a table listing each Linux command, useful options that were mentioned in this tutorial for the following Linux commands: cut , tr , wc , and tee.