= LAB OBJECTIVES =
:0. Review SSH setup and remote shell execution
:1. Explore the Fabric Python library and its command line tool "fab".
:2. Create Fabric scripts utilizing Fabric's API to define tasks for the '''fab''' command.
:3. Use the '''fab''' command to execute fabric script to perform pre-defined tasks on remote Linux machines.
= LAB OBJECTIVES =Overview ==:Completing this course will give you Fabric is a Python library and command-line tool for streamlining the prerequisites use of SSH for getting into the DevOps fieldapplication deployment or system administration tasks. A DevOps professional is It has two major components::# a command-line interface program called "fab" that lets you execute arbitrary Python functions :# a system/network administrator with programming skillsset of Python APIs that you can use and call in your Python functions to make executing shell commands over SSH much easier. As an introduction to that field, we will look at : We are going use the Fabric API and its '''Fabricfab''' command to define and execute Python functions (or tasks), to automate interactions with remote Linux machines in this lab. Using Fabric you can automate deploying software, monitoring, and updating many systems at the same time.
== REFERENCE ==
:1. These links are helpful for learning more about Fabric's features:
{| class="wikitable" | style="margin-left:20px; border: 2px solid black;"
| style="border: 2px solid black;" | Category
| style="border: 2px solid black;" | Resource Link
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official '''Fabric''' website
| style="border: 2px solid black;" valign="top"|
:[http://www.fabfile.org/]
|- style="background-color:white;border:none;"
:[https://www.digitalocean.com/community/tutorials/how-to-use-fabric-to-automate-administration-tasks-and-deployments]
|- style="background-color:white;border:none;"
| style="border: 2px solid black;" valign="top"|
:Official '''Fabric''' website
| style="border: 2px solid black;" valign="top"|
:[http://www.fabfile.org/]
|}
: <font color='blue'>Please note that the version of Fabric installed on matrix.senecacollege.ca for this lab is 1.14 and it supports only Python version 2.</font> The Fabric script files we are going to create in this lab have to meet Python version 2.x requirements. (e.g. print is a keyword, not a built-in function in Python 2.x)
:2. You should have some experience on the following topics in OPS235 and or OPS335. Please review them to prepare for the activities in this lab:
:* create and configure a regular user on a Linux system.
:* configure and manage sudo privilege for a regular user
:* Configure sudoers using the visudo command
:* using the yum command to install, remove, and update rpm packages
:* Retrieve current firewall setting using the iptables -L -n -v command
= INVESTIGATION 1: Extra VM Setup The Fabric Environment =
In order to experience : The Fabric environment consists of the following components: :# Controller workstation - the machine that has the Fabric package installed and runs the "fab" command:## the Fabric Python Library - the fabric package (already installed on matrix):## the Fabric API - fabric.api:## the Fabriccommand - '''fab'''s features in a realistic way: run Fabric script, we're going name of the script is default to set up several virtual machinesfabfile. To begin py in the current working directory unless specified otherwise with they are all going the '-f' option.:## Fabric script: contains fabric environment object value and Python functions (or tasks) to have be executed by the same configuration'''fab''' command. Please make sure that each VM has direct network connect with other VMs you wish to control and configure:# Remote machine: the target machine on which one or more Fabric tasks will be executed.:## running the ssh server daemon:## use public key (or password based) authentication for ssh connection
== PART 1 - Set up Configure and test your controller workstation ==: In this lab you will use your login account on matrix.senecacollege.ca as your Fabric controller workstation.
In this lab you will use your existing vm : The Fabric package version 1.14.0 has already been installed on matrix.senecacollege.ca. You should have access to the '''centos7fab''' as a workstation command on matrix. Login to control other VMs which wematrix.senecacollege.ca and run the following command to confirm the version of the fabric package:<source lang='bash'>fab --version</source>: Type the following command to get the command line options of the fab command:<source lang='bash'>fab --help</source>:You should get something similar to the following:<source lang="bash">Usage: fab [options] <command>[:arg1,arg2=val2,host=foo,hosts='h1;h2'll call workers,...] ...
Install Options: -h, --help show this help message and exit -d NAME, --display=NAME print detailed info about command NAME -F FORMAT, --list-format=FORMAT formats --list, choices: short, normal, nested -I, --initial-password-prompt Force password prompt up-front --initial-sudo-password-prompt Force sudo password prompt up-front -l, --list print list of possible commands and exit --set=KEY=VALUE,... comma separated KEY=VALUE pairs to set Fab env vars --shortlist alias for -F short --list -V, --version show program's version number and exit -a, --no_agent don't use the running SSH agent -A, --forward-agent forward local agent to remote end --abort-on-prompts abort instead of prompting (for password, host, etc) -c PATH, --config=PATH specify location of config file to use --colorize-errors Color error output -D, --disable-known-hosts do not load user known_hosts file -e, --eagerly-disconnect disconnect from hosts as soon as possible -f PATH, --fabfile=PATH python module file to import, e.g. 'fabric../other.py' -g HOST, --gateway=HOST gateway host to connect through --gss-auth Use GSS-API authentication --gss-deleg Delegate GSS-API client credentials or not --gss-kex Perform GSS-API Key Exchange and user authentication --hide=LEVELS comma-separated list of output levels to hide -H HOSTS, --hosts=HOSTS comma-separated list of hosts to operate on -i PATH path to SSH private key file. May be repeated. -k, --no-keys don'' using yumt load private key files from ~/. Once it'ssh/ --keepalive=N enables a keepalive every N seconds --linewise print line-by-line instead of byte-by-byte -n M, --connection-attempts=M make M attempts to connect before giving up --no-pty do not use pseudo-terminal in run/sudo -p PASSWORD, --password=PASSWORD password for use with authentication and/or sudo -P, --parallel default to parallel execution method --port=PORT SSH connection port -r, --reject-unknown-hosts reject unknown hosts --sudo-password=SUDO_PASSWORD password for use with sudo only --system-known-hosts=SYSTEM_KNOWN_HOSTS load system known_hosts file before reading user known_hosts -R ROLES, --roles=ROLES comma-separated list of roles to operate on -s installed you should have SHELL, --shell=SHELL specify a new shell, defaults to '/bin/bash -l -c' --show=LEVELS comma-separated list of output levels to show --skip-bad-hosts skip over hosts that can'fab't be reached --skip-unknown-tasks skip over unknown tasks --ssh-config-path=PATH Path to SSH config file -t N, --timeout=N set connection timeout to N seconds -T N, --command-timeout=N set remote command timeout to N seconds -u USER, --user=USER username to use when connecting to remote hosts -w, --warn-only warn, instead of abort, when commands fail -x HOSTS, --exclude-hosts=HOSTS comma-separated list of hosts to exclude -z INT, --pool-size=INT number of concurrent processes to use in parallel mode</source><font color='green' ><b>Please note and study the following command available.-line options as they will be used in some of the activities in this lab::# -H, :# -f,:# -i, :# -l, :# --port:# --user:# --initial-sudo-password-prompt </b></font>
== PART 2 : Connect to VM in myvmlab.senecacollege.ca ==: You should have received an email from ITS containing the following information::* account name: (usually 'student'):* password: (let's assume it is 'P@ssw0rd' for the following instruction in this lab):* port number for SSH access via myvmlab.senecacollege.ca (e.g. 7200): This VM will be used as the remote Linux machine in our Fabric environment. Login to matrix and try the following SSH command to test the connectivity between matrix and your assignment VM:<pre> [raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca student@myvmlab.senecacollege.ca's password: Last login: Fri Jul 3 11:06:24 2020 from mtrx-node05pd.dcm.senecacollege.ca</pre>: Once you are on your VM, try the following commands: hostname, id, and df, and record the results for later comparison with the results of other commands:<source lang='bash'>[student@centos7 ~]$ hostnamecentos7[student@centos7 ~]$ iduid=1002(student) gid=1002(student) groups=1002(student),10(wheel)[student@centos7 ~]$ dfFilesystem 1K-blocks Used Available Use% Mounted ondevtmpfs 878260 0 878260 0% /devtmpfs 889792 0 889792 0% /dev/shmtmpfs 889792 9492 880300 2% /runtmpfs 889792 0 889792 0% /sys/fs/cgroup/dev/mapper/centos- Create master Worker image root 38680112 1745524 36934588 5% //dev/sda2 1038336 331228 707108 32% /boot/dev/sda1 204580 11296 193284 6% /boot/efi/dev/mapper/centos-home 18880512 33160 18847352 1% /hometmpfs 177960 0 177960 0% /run/user/1002</source>:Logout from your VM and get back to matrix.:The previous SSH command when executed successfully, created a login shell on the remote machine. If the previous SSH command is followed by a specific bash command, it will be executed on the remote host instead of creating a login shell. Consider the following:<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'hostname;id;df'student@myvmlab.senecacollege.ca's password:centos7uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)Filesystem 1K-blocks Used Available Use% Mounted ondevtmpfs 878260 0 878260 0% /devtmpfs 889792 0 889792 0% /dev/shmtmpfs 889792 9492 880300 2% /runtmpfs 889792 0 889792 0% /sys/fs/cgroup/dev/mapper/centos-root 38680112 1745608 36934504 5% //dev/sda2 1038336 331228 707108 32% /boot/dev/sda1 204580 11296 193284 6% /boot/efi/dev/mapper/centos-home 18880512 33160 18847352 1% /hometmpfs 177960 0 177960 0% /run/user/1002</source>:The three shell commands: hostname, id, and df were executed sequentially. Compare the outputs above with the previous results when executing the corresponding commands in the login shell.:Please note that you were asked to provide the user's password for every SSH connection.
Create a new virtual machine==PART 3: Set up SSH login with public key authentication ==: In order for your controller workstation to automate tasks execution on your VM, you need to configure your VM to SSH public key authentication instead of password authentication. You've done this in both OPS235 and OPS335, and allocate for here is a summary on how to do it 1GB of RAM between your account on matrix and 8GB of disk space. Install a Basic Web Server configuration of CentOS in that your VM using the same CentOS .iso file you used for your first machine in this course.:
Make sure that:Create a new SSH key pair (one private, and one public) under your account on matrix.senecacollege.ca. * The hostname of : Once you have both keys, you can use the system is '''worker1ssh-copy-id'''command to copy your public key to the student account on your VM, replace the port number with the correct value for your VM:<source lang='bash'>ssh-copy-id -i ~/.ssh/id_rsa.pub -p 7200 student@myvmlab.senecacollege.ca* It has a static IP address appropriate for </source>: The above command should add the contents of your virtual networkpub key to ~/.* Create a regular user using ssh/authorized_keys under your student account on your Seneca email name as the user nameVM.* After installation ensure : Verify and confirm that you your account on matrix can access SSH to your VM as 'student'without prompting for a password:<source lang='worker1''bash' >[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.caLast login: Fri Jul 3 12:46:19 2020 from your main vm using the static IP address you've assigned mtrx-node05pd.dcm.senecacollege.ca[student@centos7 ~]$ exitlogoutConnection to itmyvmlab.senecacollege.ca closed.
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca 'date;hostname;id'Fri Jul 3 12:55:22 EDT 2020centos7uid=1002(student) gid=1002(student) groups= Set up SSH 1002(student),10(wheel)[raymond.chan@mtrx-node05pd lab8]$</source>: If you got similar result as above, you have successfully configure your controller workstation and your VM to use public key login ===authentication.
In order for an automated system =INVESTIGATION 2 - Running the fab command in ad-hoc mode =: The fab command relies on SSH to be able make the connection to connect the remote machine before executing the intended commands. The fab command can run in ad-hoc mode:<source lang='bash'>fab [options] -- [shell commands]</source>: When running the fab command in ad-hoc mode, it is very similar to running the SSH with commands attached at the end. == PART 1: running non-privileged shell commands on remote machines ==: In the following example, we use the '''fab''' to execute the "date", "hostname", and "id" command remotely on our VM. Try the following ad-hoc fab commands and record their results for later use, replace the port number with the correct value for your VM and administer it :<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- you will need to be able to connect to it using SSH keys'date;hostname;id'[myvmlab.senecacollege. Youca] Executing task 've done this in both OPS235 and OPS335<remainder>'[myvmlab.senecacollege.ca] run: date;hostname;id[myvmlab.senecacollege.ca] out: Fri Jul 3 13:05:39 EDT 2020[myvmlab.senecacollege.ca] out: centos7[myvmlab.senecacollege.ca] out: uid=1002(student) gid=1002(student) groups=1002(student),10(wheel)[myvmlab.senecacollege.ca] out:
Create a new SSH key on your main VM with your regular user. Please do not use root. Then set things up so that your regular user on your '''controller VM''' can SSH to the worker VM as root without putting in a password.
== PART Done.Disconnecting from myvmlab.senecacollege.ca:7200... done.[raymond.chan@mtrx-node05pd lab8]$</source>: Note that there is no password prompting if you complete part 3 successfully, otherwise, the SSH server daemon on your VM will prompt you for a password. The output from the fab's ad-hoc mode is not much different from the SSH command with shell command attached at the end, however, please note that the additional information on the output from the fab command can be very useful for record keeping purpose - Clone what has been done and whether the Workers ==commands had been carried out successfully or not.
== PART 2: running privileged commands on remote machines ==: We're only simulating say that running an ad-hoc fab command is very similar to the SSH command with shell commands attached at the real world where youend. Let'd have hundreds of VMs in one or more cloudss try both with privileged commands, but you can just imagine that like the VMs you're creating on your computer are actually being created on an Amazon or Microsoft server"yum" command.
=== Run the "yum" command on remote machine with SSH ===: By default, your VM doesn't have the "tree" rpm package installed. You can verify this with the following SSH command (remember to replace the port number with the correct value for your VM):<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca "yum list tree"Loaded plugins: fastestmirrorLoading mirror speeds from cached hostfile *base: centos.mirror.colo-serv.net * Optional *extras: centos.mirror.colo-serv.net *updates: centos.mirror.ca.planethoster.netAvailable Packagestree.x86_64 1.6.0-10.el7 base[raymond.chan@mtrx-node05pd lab8]$</source>: Please note that the tree package is "Available", but not yet installed.: Let't try to install the "tree" package with the shell command "yum install tree -y":<source lang='bash' Make four clones of the master worker image you've just created>[raymond.chan@mtrx-node05pd lab8]$ ssh -p student@myvmlab.senecacollege. Then make sure that each of them has a unique IP addressca "yum install tree -y"Loaded plugins: fastestmirrorYou need to be root to perform this command. That</source>: Using the "yum" command to query rpm package doesn's all t need special privilege, however, it does when you're required try to change manuallyinstall or remove rpm packages. All the other configuration : Your "student" account on your VM was configured to allow you to run the workers (inlcuding "sudo" command to perform software management using the hostnames) will be set by Fabric"yum" command. Normally you would have some kind of automation doing all this cloning Let's login to your VM and try the following "sudo" command to install and IP address assignment as well, but we donthen remove the "tree" rpm package:<source lang='bash't have time >[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.caLast login: Fri Jul 3 16:51:07 2020 from mtrx-node05pd.dcm.senecacollege.ca[student@centos7 ~]$ sudo yum install tree -y[sudo] password for that this semesterstudent:Loaded plugins: fastestmirrorLoading mirror speeds from cached hostfile * base: less.cogeco.net * extras: centos.mirror.colo-serv.net * updates: mirror.calgah.comResolving DependenciesMake snapshots of all your workers so that you can easily restore them to the original state after you modify them--> Running transaction check---> Package tree.x86_64 0:1.6.0-10.el7 will be installed--> Finished Dependency Resolution
= INVESTIGATION 2: Fabric practice =Dependencies Resolved
We will start with some basics======================================================================================================================== Package Arch Version Repository Size========================================================================================================================Installing: tree x86_64 1. Fabric runs python programs on the controller and the workers6. You create an "instruction" file on your controller, and execute it on the controller using the '''fab''' program. When you do that 0- you specify which workers you want your instructions to be executed on10.el7 base 46 k
The instructions are stored in a python file. Let's start with a simple one named '''fabfile.py''' (the default filename for fab):Transaction Summary========================================================================================================================Install 1 Package
== PART Total download size: 46 kInstalled size: 87 kDownloading packages:tree-1.6.0-10.el7.x86_64.rpm | 46 kB 00:00:00Running transaction checkRunning transaction testTransaction test succeededRunning transaction Installing : tree-1.6.0-10.el7.x86_64 1/1 Verifying : Simplest example ==tree-1.6.0-10.el7.x86_64 1/1
<source lang="python">from fabricInstalled: tree.api import *x86_64 0:1.6.0-10.el7
# Will get the hostname of this worker:Complete!def getHostname(): name = run("hostname") print(name)[student@centos7 ~]$
</source>
: Please note that when you run the "sudo" command the first time, it asks you for the user's password (i.e. user student's password). Let's now remote the "tree" package:<source lang='bash'>
[student@centos7 ~]$ yum remove tree -y
Loaded plugins: fastestmirror
You need to be root to perform this command.
[student@centos7 ~]$ sudo yum remove tree -y
Loaded plugins: fastestmirror
Resolving Dependencies
--> Running transaction check
---> Package tree.x86_64 0:1.6.0-10.el7 will be erased
--> Finished Dependency Resolution
All this will do is get the hostname of the worker and print it (on the controller). We run it on the controller like this:Dependencies Resolved
<source lang="bash">fab --fabfile=fabfile====================================================================================================================== Package Arch Version Repository Size========================================================================================================================Removing: tree x86_64 1.py 6.0-H 192.168.5610.11 getHostname</source>el7 @base 87 k
In the command above we're using the fab program to read the file fabfile.py and execute the getHostname function on the worker 192.168.56.11. Note that the IP address of your first worker will likely be different.Transaction Summary========================================================================================================================Remove 1 Package
If you did all the setup right and you try to execute the command above Installed size: 87 kDownloading packages:Running transaction checkRunning transaction testTransaction test succeededRunning transaction Erasing : tree-1.6.0- you will get a password prompt10. That shouldn't happen, the point is for everything to be automated, and that's why you've set up SSH keysel7. To fix it, add the following to your fab filex86_64 1/1 Verifying :tree-1.6.0-10.el7.x86_64 1/1
<source lang="python">envRemoved: tree.user = 'root'</source>x86_64 0:1.6.0-10.el7
That should have workedComplete![student@centos7 ~]$</source>: The above tests confirm that the student user is allowed to run the yum command to install and remove rpm package. Now let's logout from the VM and go back to matrix. On matrix, try to run the sudo command using SSH:<source lang='bash'>[student@centos7 ~]$ exitlogoutConnection to myvmlab.senecacollege.ca closed.[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7211 student@myvmlab.senecacollege.ca "sudo yum install tree -y"sudo: no tty present and no askpass program specified[raymond.chan@mtrx-node05pd lab8]$</source>: The above error indicated that you need a tty for the SSH session to prompt you'd get output like this:for the sudo password. Please look up the ssh man page to find out the option which turn on a tty for the SSH session.
=== Run the privileged yum command on remote machine using ad-hoc fab command ===: Let's try the corresponding ad-hoc fab command on your VM:<presource lang='bash'>$ fab --fabfilehost=fabfilemyvmlab.py senecacollege.ca -H 192-port=7200 --user=student -- 'sudo yum install tree -y'</source>: Type in your user student's password when prompted for "sudo password", the yum install command to install the tree rpm package should be executed successfully.168.56.11 getHostnameIf the tree rpm package is already installed, you can remove it with the following ad-hoc fab command: <source lang='bash'>[192fab --host=myvmlab.168senecacollege.56.11] Executing task ca --port=7200 --user=student -- 'getHostnamesudo yum remove tree -y'[192.168.56.11] run: hostname</source>[192.168.56.11] out: www[192.168.56Try remove the "tree" rpm package with the appropriate ad-hoc fab command.11] out:
worker1= INVESTIGATION 3: Running the fab command in script mode =: From investigation 2, we can see that running '''fab''' in ad-hoc mode is quick, straight forward, and easy. However, the rich output generated can not be easily captured and processed. If you have a need to capture and process the output generated by the commands executed on the remote machines, the solution is to run the '''fab''' command in script mode.: The first step in running the '''fab''' command in script mode is to create a fabric script file.: Let's start with a simple fabric script file to demonstrate some basic concepts that use the API from the Fabric python library.: On matrix, cd to your lab8 directory and create a simple fabric script file named '''fabfile.py''' (this is the default filename used by the fab command when you invoke it without the '-f' optino):
Done== PART 1: Non-privileged task example =====Create non-privileged tasks: Getting the hostname of remote machines===: Add the following contents to the default fabric script called "fabfile.py" in your lab8 directory:<source lang="python">Disconnecting from 192fabric.168.56.11... done.</pre>api import *
In # set the above you have:* Lines with an IP address telling you which worker name of the output is for/from.* Messages from user login to the controller (e.g. "Executing task...", and "run: ...").remote host* Output from the worker ("out: env...")* Output on the controller from your fab file ("worker1" which came from the "print()" call)user = 'student'
You should # Define the task to get used to the above. It's a lot hostname of output but it's important to understand where every part remote machines:def getHostname(): name = run("hostname") print("The host name is coming from:", so you are able to debug problems when they happen.name)</source>
: To check for syntax error in the fabric script, run the following command in the lab8 directory where it contains the fabric script named "fabfile.py":<source lang="bash">fab -l</source>: you should get a list of tasks defined in your fabfile.py:<source lang= Part 2"bash">[rchan@centos7 lab8]$ fab -lAvailable commands: Set up web server ==
Let's pretend that we needed getHostname</source>: To perform the task of getHostname on your VM (replace with the actual port # for connecting to deploy a web server your VM), run the fab command on several machinesmatrix:<source lang="bash">[raymond.chan@mtrx-node05pd lab8]$ fab --hosts=myvmlab.senecacollege.ca --port=7200 getHostname[myvmlab.senecacollege. Weca] Executing task 'getHostname'll set up a simple example of such a deployment here[myvmlab.senecacollege.ca] run: hostname[myvmlab.senecacollege.ca] out: centos7[myvmlab.senecacollege.ca] out:
=== Install Apache ===The host name is: centos7
Add a setupWebServer() function to your python file: <source lang="python"># Will set up a working web server with a pre-built websiteDone.def setupWebServer()Disconnecting from myvmlab.senecacollege.ca:7200... done. run("hostnamectl set[raymond.chan@mtrx-hostname www") run("yum install httpd") run("systemctl enable httpd") run("systemctl start httpd")node05pd lab8]$
</source>
: Notice that there is no need to specify the user name at the '''fab''' command line since we defined it in the fabric script file (env.user = 'student'). Also notice that we can capture the host name returned from the "hostname" command and print it out together with an descriptive text in a line.
Note that each call to :In the above executed '''fab''' command, the fab program imports the fabric script named "run()fabfile.py" will run a command and execute the getHostname function on the workerVM connect at port 7200 on myvmlab.senecacollege.ca. In this function we set Note that the hostname port number for your first will likely be of the machine to "www", install Apache, enable the Apache service, and start that service now. Pretty simple commandsa different value.
: If you try to run did all the setup right and you got a password prompt when execute the above command, read the prompt carefully and see who's password it was prompting you for. If it is not for the user student, verify that you have the same way following line in your fabfile.py and you can ssh to your VM as beforethe user student without password:
:<presource lang="python">$ fab --fabfileenv.user =fabfile.py -H 192.168.56.11 setupWebServer'student'</presource>
You: In the above output from the 'll find that yum prompts ''fab''' command, you to answer questions, which have::* Lines with the FQDN of the remote machine you don't want to do in an automated environmentare working on.:* Messages from the controller workstation (e. And also yum prints too much output, which also isn't helpful in an automated environmentg. We'll fix it by adding two switches to yum: "-yExecuting task..." , and "-d1run: ...").:* Output from the remote machine ("out: ..."):* Output generated on the controller workstation from your fab file (the print statement)
Notice also that all of the four commands can be run as many times as you want, :You should get used to the result will be above messages from the same'''fab''' command. This It's a lot of output but it's important to understand where every part is not always coming from, so easyyou are able to debug problems when they happen.
At this point if you log in == PART 2: Privileged Tasks Examples =====Creat privileged tasks: install and remove rpm package on remote machines===: Add the following two new functions to worker1 - you should see a new hostname, and httpd installed and running (try with the end of the fabric script "systemctl statusfabfile.py").in your lab8 directory:<source lang='bash'>
def installPackage(pkg='dummy'): cmd ='yum install ' + pkg + ' -y' status = Deploy a website ===sudo(cmd) print(status)
Now def removePackage(pkg): if pkg == '': cmd = 'yum remove dummy -y' else: cmd = 'yum remove ' + pkg + ' -y' status = sudo(cmd) print(status)</source>: Note that we have a web server runningboth functions take one function argument in different ways. However, if no function argument is passed when calling the function, we also want both will default to put a website string value of "dummy". Both functions call the sudo() from the fabric.api to execute the command contained in the "cmd" object on itthe remote machine via sudo. The website can be of : To check for any complexitysyntax error in your updated fabric script, but to keep this demonstration simple we'll have a single HTML filerun the following command in the same directory as the fabfile. You can pretend that itpy:<source lang='s as complex as you like. Create an bash'>fab -l</source>: You should get a list of tasks defined similar to the following:<source lang='bash'index>[raymond.html''' file like thischan@mtrx-node05pd lab8]$ fab -lAvailable commands:
getHostname installPackage removePackage[raymond.chan@mtrx-node05pd lab8]$</source>: If you only need to connect to the same remote machine, you can specify the host and port number in the fabfile.py to save some typing when executing the fab command. Add the following two lines after the env.user line in your fabfile.py:<source lang="html"'bash'>env.port = '7200' # <h1>My fancy web server-- please replace with the actual value of your VM's port numberenv.hosts =['myvmlab.senecacollege.ca'] </h1source>: You can also store the user's password in this file so that it will respond to the "sudo password" prompt for sudo() call. It is not safe to do so as you can configure the sudo module on the remote machine not to ask for sudo password.: Now you can run the fab command without the "--host" and "--port" option.: Run the following two fab commands, note the results and compare their difference:</sourcelang='bash'>fab installPackage
And since we're pretending that it's a large website with many files and directories, we'll compress it into an archive named '''webcontents.tar.bz2''' using a tar command. You've done this since OPS235.fab installPackage:tree</source>Once you have your archive, make sure it's in : Run the same directory as your following two fab file. Then add commands, note the following to your setupWebServer() functionresults and compare their difference:<source lang='bash'>fab removePackage
<source lang="python"> with cd("/var/www/html/")fab removePackage: put("webcontents.tar.bz2", ".") run("tar xvf webcontents.tar.bz2") run("rm webcontents.tar.bz2")tree
</source>
There is something weird in the code above that you haven't seen before but it's required == Part 2: Create remote task for some uses of Fabricupdating rpm packages ==: the '''with''' statement. The problem is that separate '''run''' commands each execute in Add a brand new session, each with its own shell. They are not like separate lines in a single shell script even though they look like they should be. That means if you run a cd command and then a tar command separately - the tar command will not run in the directory where you think it will. In order function called "updatePackage" to fix this you have to nest commands inside a '''with''' - it's like a '''run''' but with persistant resultsyour fabfile. The code we added py according to the following requirements::* Accept optional function will cd to argument as the rpm package name:* If no function argument was given when called, default web site directory on the worker, upload your web contents tarball from your controller to that directory on all the worker, extract it, and delete the tarball.packages installed After it's done - you should have a working web server and simple website on your worker1. Except you won't be able to access it because : The output of the firewall. We'll deal with that in the next section.updatePackage when executed, should produce similar output as shown below: == Part 2: Set up the firewall == Recall that in our OPS courses we've been using iptables instead of firewalld, which is installed by default in CentOS. Let's make sure that our workers have that set up as well. In the same '''fabfile1.py''' you've been using all along, add Update a new function like thissingle package: <source lang="python"'bash'># Will uninstall firewalld and replace it with iptablesdef setupFirewall()fab updatePackage: run("yum -y -d1 remove firewalld") run("yum -y -d1 install iptables-services") run("systemctl enable iptables") run("systemctl start iptables")tree
</source>
: Sample output:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab updatePackage:tree
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update tree -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
[myvmlab.senecacollege.ca] out: No packages marked for update
[myvmlab.senecacollege.ca] out:
That should by now look prett obviousLoaded plugins: fastestmirrorLoading mirror speeds from cached hostfile * base: less. On the worker you're going to uninstall firewalld, install iptables, and make sure that the iptables service is runningcogeco.net * extras: centos.mirror.ca.planethoster.net * updates: less.cogeco.netNo packages marked for update
Execute the function for worker1 and double-check that it workedDone. === Allow access to Apache through the firewall === The default setup of iptables also doesn't allow access to our web serverDisconnecting from myvmlab. We'll need to add some more to our function to allow itsenecacollege. This would probably make more sense in setupWebServer() but for now let's put it into setupFirewall()ca:7200... done. <source lang="python"> run("iptables -I INPUT -p tcp --dport 80 -j ACCEPT") run("iptables[raymond.chan@mtrx-save > /etc/sysconfig/iptables")node05pd lab8]$
</source>
Easy enough, but there's on problem - if we run this more than once, we're going to end up with duplicate iptables rules for port 80 (check with iptables -L):2. In order to avoid that - we have to first check whether the rule exists before we add it. We can do that like thisUpdate all installed package: <source lang="'bash">iptables -C INPUT -p tcp --dport 80 -j ACCEPT"</source'> Unfortunately that command answers "yes" or "no" by succeeding or failing depending on whether that rule exists. In Fabric when a command fails - the entire fab file execution stops, assuming that it's an unrecoverable error. We need to prevent that with another with statement: <source lang="python"> with settings(warn_only=True)updatePackage: firewallAlreadySetUp = run("iptables -C INPUT -p tcp --dport 80 -j ACCEPT") if firewallAlreadySetUp.return_code == 1: ... move your iptables rules setup here ...
</source>
: The following output had been trimmed, only showing the first few lines:<source lang='bash'>
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
...
Test your new setupFirewall function on worker1, and make sure it opens access to Apache but does not create duplicate rules every time it's run Verifying : systemd-219-73.el7_8.5.x86_64 53/54 Verifying : systemd-libs-219-73.el7_8.5.x86_64 54/54
= INVESTIGATION Removed: kernel.x86_64 0:3: Multiplying your work =.10.0-862.el7
After completing all the previous parts of the lab Installed: kernel.x86_64 0:3.10.0- you should have a working fabfile1127.py with two working functions: setupFirewall() and setupWebServer()13.1.el7
'''** Optional **'''You were asked to test them on worker1Updated: bind-export-libs.x86_64 32:9.11.4-16.P2.el7_8.6 binutils.x86_64 0:2.27-43.base.el7_8.1 ca-certificates.noarch 0:2020.2.41-70.0.el7_8 device-mapper.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 kernel-tools.x86_64 0:3.10.0-1127.13.1.el7 kernel-tools-libs.x86_64 0:3.10.0-1127.13.1.el7 lvm2.x86_64 7:2.02.186-7.el7_8.2 lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 microcode_ctl.x86_64 2:2.1-61.10.el7_8 net-snmp.x86_64 1:5.7.2-48.el7_8.1 net-snmp-agent-libs.x86_64 1:5.7.2-48.el7_8.1 net-snmp-libs.x86_64 1:5.7.2-48.el7_8.1 net-snmp-utils.x86_64 1:5.7.2-48.el7_8.1 ntp.x86_64 0:4.2.6p5-29.el7.centos.2 ntpdate.x86_64 0:4.2.6p5-29.el7.centos.2 python-perf.x86_64 0:3.10.0-1127.13.1.el7 rsyslog.x86_64 0:8.24.0-52.el7_8.2 selinux-policy.noarch 0:3.13.1-266.el7_8.1 selinux-policy-targeted.noarch 0:3.13.1-266.el7_8.1 systemd.x86_64 0:219-73. Now let's run these two functions on all your workers at the same timeel7_8. The command is almost the same, except for the list of IP addresses8 systemd-libs.x86_64 0:219-73.el7_8.8 systemd-sysv.x86_64 0:219-73.el7_8.8 yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8
<source lang="bash">fab --fabfile=fabfile.py -H 192.168.56.11,192.168.56.12,192.168.56.13,192.168.56.14,192.168.56.15 setupWebServer</source>Complete!
Again - your IP addresses will be different but the command will be the sameDone. You can also reconfigure the firewall on all the workers at the same time, using a command like this on your controller: <source lang="bash">fab --fabfile=fabfileDisconnecting from myvmlab.py -H 192senecacollege.168ca:7200.56.11,192.168done.56[raymond.12,192.168.56.13,192.168.56.14,192.168.56.15 setupFirewallchan@mtrx-node05pd lab8]$</source>
And imagine that you might have 10= Lab Exercise: Create a Fabric task called makeUser() =: Study the Fabric API run(), 50sudo(), 100 servers and put() and utilize them to do this create a new task called makeUser(): The makeUser() function should perform the following::* create a new user called "ops435p" with home directory "/home/ops435p".:* add it to the sudo group called "wheel". :* ask your professor for a ssh public key and add it to the file named "authorized_keys" in the ~ops435p/.ssh directory. Make sure that you set the proper permissions on - could you do it without both the directory ~ops435p/.ssh and the file "~ops435p/.ssh/authorized_keys.:Add the makeUser() to your final version of fabfile.py. :Run the automation?new task makeUser() on your VM.:Verify and confirm that your new makeUser() task is working correctly.
= Final Task - Apply your instructions in fabfile.py to your VM on myvmlab =
: Since your account on your vm on myvmlab is a regular user with sudo privilege. You need to make the following changes to your fabfile.py before applying it to your vm on myvmlab:
:* Change env.user from 'root' to your account on your vm in myvmlab.
:* Change all the commands that need super user privilege from calling the run() function to instead calling the sudo() function.
: Test your updated fabfile.py until you get the same result as when you apply it to your own worker VM.
= LAB 8 SIGN-OFF (SHOW INSTRUCTOR) =
:'''Have Ready to Show Your Instructor:'''* Complete all the parts of the lab and show upload the version of your fabfile.py which works on your vm on myvmlabto Blackboard by the due date. = LAB REVIEW =
[[Category:OPS435-Python]][[Category:rchan]]