Tutorial2: Unix / Linux File Management

From CDOT Wiki
Revision as of 10:03, 11 January 2020 by Msaul (talk | contribs) (Part 3: Manage / Manipulate Text File Content)
Jump to: navigation, search

UNIX / LINUX FILE MANAGEMENT CONCEPTS


Main Objectives of this Practice Tutorial

  • Understand the Purpose of the Unix / Linux Directory Structure.
  • List Common Directories that are Contained in a Typical Unix / Linux Filesystem
  • Use Common Unix / Linux Commands to Perform Directory Management Tasks.
  • Use Text Editors to Create and Modify Text Files
  • Use Common Unix / Linux Commands to Manipulate text filss.



Tutorial Reference Material

Course Notes
Linux Command/Shortcut Reference
YouTube Videos
Course Notes:Tutorials:


File Management Text Editors /
File Content
Brauer Instructional Videos:

Unix / Linux Directories

In Unix / Linux (as opposed to MS Windows), there are no drive letters (such as C:, or D:).

All files and directories appear under a single ancestor directory called "the root directory".

A path points to a file system location by following the directory tree hierarchy expressed in a string of characters in which path components, separated by a delimiting character, represent each directory. The delimiting character is most commonly the slash ("/").

The Unix/Linux file system is hierarchical, similar to other operating systems such as
Windows, Mac OSX, etc. Files are organized in directories. Directories may contain sub-directories.

In Unix / Linux (as opposed to MS Windows), there are no drive letters (such as C:, or D:). All files and directories appear under a single ancestor directory called the "root directory".

Learning how to issue Linux commands for navigating the Linux filesystem and manipulating directory and files are essential skills for Linux users and administrators.

In the Linux (Unix) OS, the "root directory" / is the starting directory, and other "child directories", "grandchild directories", etc. are created The hierarchical structure resembles an "upside- down tree". There is actually a command called tree that can display a "tree diagram"!

Directory Pathnames

A path, the general form of the name of a file or directory, specifies a unique location in a file system.
A path points to a file system location by following the directory tree hierarchy expressed in a string of characters in which path components, separated by a delimiting character, represent each directory. The delimiting character is most commonly the slash ("/").

Reference: https://en.wikipedia.org/wiki/Path_(computing)


The following table displays and defines commonly used directories
(listed by directory pathname) for for ALL Unix / Linux Filesystems:

Please take a few moments to review these pathnames.

Common directory pathnames
for for ALL Unix / Linux Filesystems.
Directory PathnamePurpose
/Root directory (ancestor to all directories)
/homeUsed to store users’ home directories
/home/usernameA specific User's Home Directory
/binCommon system binaries (commands)
/usr/binCommon utilities (commands) for users.
/usr/sbinCommon utilities for system administration
/etcSystem administration files (eg. passwd)
/varDynamic files (log and mail files)
/tmp , /var/tmpTemporary files for programs
/devDevice driver files (terminals, printers, etc.)


When you log into your Matrix account, you are directed to your home directory. This directory is where the user can store files, and create subdirectories to organize their files.

We will now learn to create and manage subdirectories within your home directory.

INVESTIGATION 1: MANAGING DIRECTORIES


In this investigation, you will learn to create, navigate, listing contents and remove directories in your Matrix account.


Directory File Naming Rules

Before you learn how to create directories, it is important to understand what represents an appropriate directory filename.

Listed below are some common file-naming rules:

  • Unix/Linux characters are case sensitive. It is recommended to be consistent (e.g. use all lowercase letters)
  • Adopt a consistent directory naming scheme (this will help you to navigate within your directory structure)
  • Make your directory names meaningful
  • Avoid non-alphanumeric characters, as they may have a special meaning to the system that will make your work more difficult when changing to directories, etc.
  • Avoid using spaces for directory names (consider periods, hyphens, and underscores instead)


Part 1: Creating Directories

Tree Diagram of Directory Structure to Create in your Home Directory (displayed in blue text).

Creating subdirectories within your home directory makes it more efficient to save and access files on your Linux server.

A comparison would be rooms in a house. If there were no rooms, just one large room in a 3,000 square foot house, it would be "messy" and difficult to locate items. Each room in a house is used to for a specific purpose to be more productive to perform a task such as a kitchen, bedroom, bathroom, etc.


You would like to create a directory structure within your home directory as displayed in the diagram on the right aide.


As previously mentioned, the root directory is the "starting point" in the Matrix file system, that the home directory is used to store all Matrix user accounts by their userid, and that your userid contained within the home directory represents YOUR home directory where you can create files (both directory files, text files, etc).


Perform the Following Steps:

  1. Login your matrix account.

  2. Issue a command to confirm you are located in your home directory. You should know how to do this from the previous tutorial.

  3. Issue the following Linux command: mkdir uli101

    NOTE: You should always confirm that you have created a directory.
    This can be done by issuing the ls command.

    Creating and Confirming the Creation of a Directory.
  4. Issue the following Linux command: ls uli101

    There are no contents that are contained in this newly-created directory; therefore, no contents appear. A useful option -d can be used to confirm that the actual directory has been created as opposed to viewing the contents of the directory.

  5. Issue the following Linux command: ls -d uli101

    You should now see just the directory listed. You can also combine the -d and -l options to provide more detail regarding the newly-created directory.

  6. Issue the following Linux command:ls -d -l uli101

    How can you confirm from the output of this command that the file uli101 is a directory?

  7. Issue the following Linux command:ls -ld uli101

    Is the output from this command the same was the output from the previous command?
    If so, what does this say about how to use multiple options for Linux commands?

  8. Issue the following Linux command to create the acp100 and xyz100 directories: mkdir acp100 xyz100

    NOTE: You can create multiple directories by issuing the mkdir with multiple arguments.

  9. Issue the following Linux command to confirm that those directories have been created: ls -ld acp100 xyz100

  10. We will now create the subdirectories that are contained in the uli101 directory.
    Issue the following Linux command to move to the uli101 directory: cd uli101

  11. Issue a command to confirm that your current location is in the uli101 directory.
    You should know how to issue this command from a previous tutorial.

  12. Issue the following Linux command to create the directories called notes, tutorials and examples: mkdir notes tutorials examples

  13. Issue a Linux command to confirm that those directories have been created. You should know how to do this...

  14. This technique is considered to be inefficient. There are options and ways to create the same directory structure without using the cd command and only issue a single Linux command.

    Using this inefficient method tends to show that you are a novice or "newbie", and you may lose marks if you issue multiple Linux commands to performed tasks that can be performed using only a single Linux command.

    Although we will teach you how to remove directories and their contents in a future section, let's change back to your home directory and issue a command to remove the directory structure that you just created so you can learn a more efficient method of creating the same directory structure.

  15. Issue Linux commands to move to your home directory and confirm that your current directory is your home directory. You should know how to do this...

  16. Issue the following Linux command to remove all of the directories that you have created:
    rm -r uli101 acp100 xyz100

    NOTE: You will learn how to safely remove directories and their contents in another section.

  17. Issue the following single Linux command to create the entire directory structure:
    mkdir -p uli101/notes uli101/tutorials uli101/examples acp100 xyz100

    NOTE: The -p option allows "parent" directories to be automatically created first to then create their subdirectories.

  18. Issue the following Linux command to confirm that all of the directories have been created:
    ls -ld uli101 uli101/notes uli101/tutorials uli101/examples acp100 xyz100

Downloading & Running a Shell Script to Check your Work

Although you are being asked to create the directory structure, you might have made some mistakes:
For Example:

  • Forgetting to create a directory.
  • Making syntax errors (eg. spelling or mixing up character case)
  • Not creating subdirectories within specified directories.


If all all check pass, then user performed task correctly and can continue.
If there is a warning, then feedback is provided to user to correct and re-run checking script.

To check for mistakes, a shell script has been created to check your work. If the checking shell script detects an error, then it will provide feedback and offer constructive feedback on how to fix that problem so you can re-run the checking shell scripts until your work is correct.


Perform the Following Steps:

  1. Make certain that your current directory is your home directory.

  2. Issue the following Linux command to download a shell script to your home directory that will check your work:
    wget https://matrix.senecacollege.ca/~murray.saul/uli101/week1-check-1

  3. Run the checking script to check your work by issuing the following:
    bash week1-check-1

  4. If you encounter errors, then view the feedback to make corrections, and then re-run the checking script. If you receive a congratulation message that there are no errors, then proceed to the next section.


Part 2: Viewing Directory Contents / Copying & Moving Directories

Now that you have learned how to efficiently create your directory structure, you will now learn how to issue Linux commands to view contents contained in directories, as well as copy and move directories.


Perform the Following Steps:

Output of the tree command to display directory structure.
Output of the ls -lR command to display directory structure.
  1. Issue the following Linux command: tree

    NOTE: You should see the directory structure that you created in the previous section. You can also issue the tree command using a directory pathname to display the directory structure for a specific scope.

    You can also use the -R option for the ls command to display all directories and subdirectories for a specified directory path (referred to as a recursive directory listing).

  2. Issue the following Linux command: ls -lR

    What directories do you see?

  3. Issue the following Linux command: ls -lR uli101

    Note the differences between both of these commands.

    You can copy and move entire directories (and their contents) to other directories.

    Output of the tree command to confirm copy of uli101 directory (and contents) to the xyz directory.
    Output of the tree command to confirm movement of acp100 directory directory (and contents) to the xyz diredtory.
  4. Issue the following Linux command:
    cp -R uli101 xyz100

  5. Issue the following Linux command to display the directory structure of your home directory to confirm you copied the uli101 directory:
    tree

  6. Issue the following Linux command:
    mv acp100 xyz100/uli101/tutorials

  7. Issue the following Linux command to display the directory structure of your home directory to confirm you copied the uli101 directory:
    tree

    Let's download and run a checking script to see if you moved and copied the directories correctly.

  8. Make certain that your current directory is your home directory.

  9. Issue the following Linux command to download a shell script to your home directory that will check your work:
    wget https://matrix.senecacollege.ca/~murray.saul/uli101/week1-check-2

  10. Run the checking script to check your work by issuing the following:
    bash week1-check-2

    If you encounter errors and re-run the checking script until you receive a congratulations message, and proceed to the next section.

Part 3: Removing Directories

In this section, we will learn how to safely remove directories and their contents.


Perform the Following Steps:

  1. Confirm that you are located in your home directory.

    The rmdir command is used to remove empty directories.

  2. Issue the following Linux command to remove the empty directory called uli101/tutorials: rmdir uli101/tutorials

  3. Issue the following Linux command to remove the empty directory called uli101: rmdir uli101

    NOTE: You should get an error message, since the uli101 directory is NOT empty.

    To remove non-empty directories, you can use the rm -r command.
    The -r option stands for recursive, which can travel down the directory paths and their contents.

  4. Issue the following Linux command to remove the uli101 directory and its contents: rm -r uli101

  5. Issue the tree command to confirm that the uli101 directory (contained in your home directory) and its contents have been removed.

    NOTE: To safely remove non-empty directories, you can add the -i option which will prompt the user if they wish to remove contents as it your travel recursively down a directory to remove directories and their contents.

  6. Issue the following Linux command (entering "y" when prompted): rm -ri xyz100

    NOTE: You should have removed all directories that you have created.

    Let's download and run a checking script to see if you have removed all of the directories that you earlier created.

  7. Make certain that your current directory is your home directory.

  8. Issue the following Linux command to download a shell script to your home directory that will check your work:
    wget https://matrix.senecacollege.ca/~murray.saul/uli101/week1-check-3

  9. Run the checking script to check your work by issuing the following:
    bash week1-check-3

INVESTIGATION 2: MANAGING TEXT FILES

The Nano Text Editor is Easy to Operate for Novice Users.
The Vi Text Editor, although taking longer to master, has outstanding features and allow the user to be more productive.


Since IT students will be working in the Unix / Linux command line environment, it is useful for them to learn a least a couple of command-line text editors.

Although programming students can use graphical IDE's to code and compile programs, they can create source code using a text editor in the Linux system, and compile their source code to generate executable programs (without having to transfer them for compilation or execution).

Networking and Tech Support students would find a text editor useful to edit configuration files. These students in upper grades will become familiar with the process of installing, configuring, and running network services and text editors are an important tools
to help setup but also "tweak" or make periodic changes in networking services configuration.

A general rule is for a student to expose themselves to a number of different text editors and then use one that they feel most comfortable working with. The two most readily-available command line text editors in Linux are Nano and Vi. The Nano text editor would seem like an easier-to-use text editor, but Vi (although taking longer to learn) has outstanding features and allow the user to be more productive.

Part 1: Creating Text Files Using The Nano Text Editor

Directory-structure-10.png
Enter the following text that appears in this diagram.

You will now learn basic editing skills using the Nano text editor including creating, editing, and saving text files.

Perform the Following Steps:

  1. Create the following directory structure (displayed on the right side) by issuing a single Linux command:
    You should know how to perform this task.

  2. Issue the following Linux command to edit a text file called mytext.txt in the linux/practice directory:
    nano uli101/practice/mytext.txt

    NOTE: When using the Nano text editor, you are placed in INPUT mode, so you can enter text immediately.

  3. Enter the lines shown in the other diagram of the nano text editor on the right side.

    NOTE: The ^ symbol represents press the <ctrl> followed by a character.

  4. To save your editing session, press: <ctrl><x>

  5. You will be prompted to modify your file: type the letter y for yes.

  6. The name of the file will be displayed: press ENTER.

    NOTE: This prompt for file name allows you to change the name of the file if you wish.
    By pressing ENTER, it will accept the default filename.

  7. Refer to the table below for a list of the most common Nano editing commands:

    Keyboard CombinationPurpose
    <ctrl><k>Cut line
    <alt><M><^>Copy Line
    <ctrl><u>Paste Cut/Copied Text
    <ctrl><g>Display help screen (ctrl-x to exit help screen)

    Part 2: Creating Text Files Using The Vi Text Editor

    The online vi-tutorial provides users "hands-on" experience of using the vi text editor.

    An online tutorial has been created to give you "hands-on" experience on how to use Vi text editor. It is recommended that you run this online tutorial in your Matrix account to learn how to create and edit text files with the Vi text editor.


    Perform the Following Steps:

    1. Issue the following to run the Vi online tutorial:
      /home/murray.saul/vi-tutorial

    2. In the tutorial menu, select the first menu item labelled "USING THE VI TEXT EDITOR"

    3. Read and follow the instructions in the tutorial. Eventually, it will display a simulated Vi environment and will receive step-by-step instructions on how to use this text editor.

    4. When you have completed that section, you will be returned to the main menu. If you want to get extra practice, you can select the menu item labelled "REVIEW EXERCISE".

    5. When you want to exit the tutorial, select the menu option to exit the tutorial.

    Part 3: Manage / Manipulate Text File Content

    We finish this tutorial by learning to use commands that manage the content contained instead text files whether it relate to to viewing file content or manipulating file content.
    Perform the Following Steps:

    1. View the following table of Text File Management commands Your instructor should have demonstrated in class how to use these commands):
      Linux CommandPurpose
      touchCreate empty file(s) / Updates Existing File's Date/Time Stamp
      catDisplay text file's contents without editing (small files)
      more , lessDisplay / Navigate within large text files without editing
      cpCopy text file(s)
      mvMove / Rename text files
      rmRemove text file(s)
      diff file1 file2Displays differences between 2 files
      fileGives info about the contents of the file (e.g. file with no extention)
      findTo find files matching specified characteristics: valign="top">
      find . -name "file*"List pathname of any filenames beginning with "file",
      from the current directory and any subdirectories
      find . -size +50kList pathname of any files larger than 50 kb, from the current directory and any subdirectories
      find . -mmin -5 List files modified less than 5 minutes ago
      find -P .Lists file pathnames in the current directory


    2. Make certain that you are located in your home directory.

    3. Use the touch command to create the empty files called a.txt, b.txt, and c.txt

    4. Use the nano text editor to edit the empty file called a.txt.

    5. Type the number "1" and press ENTER. On the second line, type the number "2" and press ENTER. Continue entering increasing number values until you reach the number 40 on line 40.

    6. Save your editing session.

    7. Issue the following Linux command: cat a.txt

      Can you see all of the contents?

    8. Issue the following Linux command: more a.txt

      Can you view or at least navigate to see all of the contents? What is the advantage of using the more command?

    9. Type the letter "q" to exit the more command.

    10. Issue the following Linux command: less a.txt

      Is there any difference between the more and less commands?
      (again press q to quit)

    11. Issue the following Linux command: cp a.txt b.txt

    12. Issue one of the commands you learned to display the contents of the file called b.txt without editing.

      What happened to this file?

    13. Issue the following Linux command: mv a.txt a.txt.bk

    14. Issue a Linux command to view the directory contents.

      What happened to the file called a.txt? Why?

    15. Issue the following Linux command: file b.txt

      What sort of information did it provide?

    16. Issue the following Linux command: diff a.txt.bk b.txt

      What was the output? Why do you think caused the result of this output?

    17. Issue the following Linux command: diff a.txt.bk c.txt

      What reason would this type of output occur?

    18. Issue the following Linux command: find -P .

      What is the output of this command?

    19. Issue the following Linux command: rm b.txt a.txt.bk c.txt

    20. Issue the ls command to verify that these files have been removed.

    21. After you complete the Review Questions sections to get additional practice, then work on your online assignment, section 2 labelled "Basic Unix Commands" (parts 4 to 6) in this section which are respectively labelled: Managing Files , Accessing Files and Review Exercise.

    LINUX PRACTICE QUESTIONS

    The purpose of this section is to obtain extra practice to help with quizzes, your midterm, and your final exam.

    Here is a link to the MS Word Document of ALL of the questions displayed below but with extra room to answer on the document to simulate a quiz:

    https://ict.senecacollege.ca/~murray.saul/uli101/uli101_week2_practice.docx

    Your instructor may take-up these questions during class. It is up to the student to attend classes in order to obtain the answers to the following questions. Your instructor will NOT provide these answers in any other form (eg. e-mail, etc).


    Review Questions:

    Directory-structure-11.png
    1. Write a single Linux command to create the directory structure starting from your home directory from the diagram displayed on the right.
    2. Write a Linux command to display a detailed listing of history directory.
      How would this command differ if you wanted to also view hidden files as well?
    3. Write a Linux command to change to the project directory.
      What command would you issue to return to your home directory?
    4. Write a Linux command to copy the project directory and its contents to the history directory.
    5. Write a Linux command to move the directory called directories to the history directory.
    6. Write a Linux command to remove both directories named 1 and 2.
    7. Write a Linux command to remove the concepts directory and its contents.
    8. Write a Linux command to remove the concepts directory and prompt the user if they want to remove this directory’s contents.
    9. Write a single Linux command to create the following empty files in the concepts directory:
      myfile.txt
      yourfile.txt
      thefile.txt

    10. Write a Linux command to view the contents of the myfile.txt text file to prove it is empty.
      What is the difference between the commands: cat, more and less?
    11. Write a Linux command to sort the contents of a file called uli101/customers.txt
    12. Write a Linux command to display the first 4 lines of a file called uli101/customers.txt
    13. Write a Linux command to display the last line of a file called uli101/customers.txt
    14. Write a Linux command to match a line containing the pattern Linux in a file called uli101/customers.txt
    15. Write a Linux command to display unique occurrences of consecutive lines in a file called uli101/customers.txt
    16. Create a table listing each Linux command, useful options that are displayed near the top of this tutorial labelled: Tutorial Reference Material