1,760
edits
Changes
no edit summary
= LAB OBJECTIVES =
:0. Review SSH setup and remote shell execution
:1. Explore the Fabric Python library and its command line tool "fab".
:2. Create Fabric scripts utilize utilizing Fabric's API to define tasks that can be executed by for the '''fab''' programcommand.:3. Use the '''fab''' command to execute fabric script to perform regular/administrative pre-defined tasks on remote Linux machines.
== Overview ==
:[http://www.fabfile.org/]
|}
: <font color='blue'>Please note that the version of Fabric we are going to use installed on matrix.senecacollege.ca for this lab is 1.14 and it supports only Python version 2.</font>The Fabric script files we are going to create in this lab have to meet Python version 2.x requirements. (e.g. print is a keyword, not a built-in function in Python 2.x)
:2. You should have some experience on the following topics in OPS235 and or OPS335. Please review them to prepare for the activities in this lab:
:* create and configure a regular user on a Linux system.
: Create a new SSH key pair (one private, and one public) under your account on matrix.senecacollege.ca.
: Once you have both keys, you can use the '''ssh-copy-id''' command to copy your public key to the student account on your VM, replace the port number with the correct value for your VM:<source lang='bash'>
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 7200 student@myvmlab.senecacollege.ca
</source>
: When running the fab command in ad-hoc mode, it is very similar to running the SSH with commands attached at the end.
== PART 1: running non-privileged shell commands on remote machines ==
: In the following example, we use the '''fab''' to execute the "date", "hostname", and "id" command remotely on our VM. Try the following ad-hoc fab commands and record their results for later use, replace the port number with the correct value for your VM:<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'date;hostname;id'
[myvmlab.senecacollege.ca] Executing task '<remainder>'
=== Run the "yum" command on remote machine with SSH ===
: By default, your VM doesn't have the "tree" rpm package installed. You can verify this with the following SSH command(remember to replace the port number with the correct value for your VM):<source lang='bash'>
[raymond.chan@mtrx-node05pd lab8]$ ssh -p 7200 student@myvmlab.senecacollege.ca "yum list tree"
Loaded plugins: fastestmirror
fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'sudo yum install tree -y'
</source>
: Type in your user student's password when prompted for "sudo password", the yum install command to install the tree rpm package should be executed successfully.If the tree rpm package is already installed, you can remove it with the following ad-hoc fab command: <source lang='bash'>fab --host=myvmlab.senecacollege.ca --port=7200 --user=student -- 'sudo yum remove tree -y'</source>
: Try remove the "tree" rpm package with the appropriate ad-hoc fab command.
= INVESTIGATION 3: Running the fab command in script mode =
: From investigation 2, we can see that running '''fab''' in ad-hoc mode is quick, straight forward, and easy. However, the rich output generated can not be easily captured and processed. If you have a need to capture and process the output generated by the commands executed on the remote machines, the solution is to run the '''fab''' command in script mode.
: The first step in running the '''fab''' command in script mode is to create a fabric script file.
: On matrix, cd to your lab8 directory and create a simple fabric script file named '''fabfile.py''' (this is the default filename used by the fab command when you invoke it without the '-f' optino):
== PART 1: Simplest Non-privileged task example =====Create non-privileged tasks: Getting the hostname of the remote machinemachines===: Add the following contents to the file default fabric script called "fabfile.py" in your lab8 directory:<source lang="python">
from fabric.api import *
env.user = 'student'
# Define the task to get the hostname of the remote machinemachines:
def getHostname():
name = run("hostname")
:You should get used to the above messages from the '''fab''' command. It's a lot of output but it's important to understand where every part is coming from, so you are able to debug problems when they happen.
== Part PART 2: Set up more administrative Privileged Tasks Examples =====Creat privileged tasks: install and remove rpm package on remote machines===: Add the following two new functions to the end of the fabric script "fabfile.py" in your lab8 directory:<source lang='bash'>
def installPackage(pkg='dummy'):Let cmd = 'yum install ' + pkg + 's pretend that we need collect the disk usage on several machines so that we can plan for storage maintenance. We-y'll set up a simple example of such a deployment here. status = sudo(cmd) print(status)
def removePackage(pkg): if pkg =='': cmd = Getting 'yum remove dummy -y' else: cmd = 'yum remove ' + pkg + ' -y' status = sudo(cmd) print(status)</source>: Note that both functions take one function argument in different ways. However, if no function argument is passed when calling the function, both will default to a string value of "dummy". Both functions call the sudo() from the fabric.api to execute the command contained in the disk usage "cmd" object on the remote worker =machine via sudo.: To check for any syntax error in your updated fabric script, run the following command in the same directory as the fabfile.py:<source lang='bash'>fab -l</source>: You should get a list of tasks defined similar to the following:<source lang='bash'>[raymond.chan@mtrx-node05pd lab8]$ fab -lAvailable commands:
</source>
:Note that each call If you only need to connect to "run()" will run a command on the worker. In this function we get the date/time of the same remote workmachine, you can specify the host and then get port number in the disk usagefabfile. The print() function print out both py to save some typing when executing the values returnedfab command. :If you try to run it Add the following two lines after the same way as before: <pre>$ fab --fabfile=env.user line in your fabfile.py -H 192.168.122.169 getDiskUsage</pre> :You should get the following output:<source lang="'bash"'>[rchan@centos7 lab8]$ fab --fabfileenv.port =fabfile.py -H 192.168.122.169 getDiskUsage[192.168.122.169] Executing task 'getDiskUsage7200'[192.168.122.169] run: date[192.168.122.169] out: Sun Nov 10 13:17:16 EST 2019[192.168.122.169] out: [192.168.122.169] run: df # <-H[192.168.122.169] out: Filesystem Size Used Avail Use% Mounted on[192.168.122.169] out: devtmpfs 947M 0 947M 0% /dev[192.168.122.169] out: tmpfs 964M 0 964M 0% /dev/shm[192.168.122.169] out: tmpfs 964M 9.7M 954M 2% /run[192.168.122.169] out: tmpfs 964M 0 964M 0% /sys/fs/cgroup[192.168.122.169] out: /dev/mapper/centos-root 7.7G 5.6G 2.1G 73% /please replace with the actual value of your VM's port number[192env.168.122.169] out: /dev/vda1 1.1G 298M 766M 29% /boothosts =[192'myvmlab.168senecacollege.122.169] out: tmpfs 193M 17k 193M 1% /run/user/42[192.168.122.169ca'] out: tmpfs 193M 0 193M 0% /run/user/1000[192.168.122.169] out: Current Disk Usage at Sun Nov 10 13:17:16 EST 2019Filesystem Size Used Avail Use% Mounted ondevtmpfs 947M 0 947M 0% /devtmpfs 964M 0 964M 0% /dev/shmtmpfs 964M 9.7M 954M 2% /runtmpfs 964M 0 964M 0% /sys/fs/cgroup/dev/mapper/centos-root 7.7G 5.6G 2.1G 73% //dev/vda1 1.1G 298M 766M 29% /boottmpfs 193M 17k 193M 1% /run/user/42tmpfs 193M 0 193M 0% /run/user/1000 Done.Disconnecting from 192.168.122.169... done.
</source>
: You can also store the user's password in this file so that it will respond to the "sudo password" prompt for sudo() call. It is not safe to do so as you can configure the sudo module on the remote machine not to ask for sudo password.
: Now you can run the fab command without the "--host" and "--port" option.
: Run the following two fab commands, note the results and compare their difference:<source lang='bash'>
fab installPackage
</source>
: Do a syntax check with Run the "following two fab -l" command.: When you try to run it commands, note the same way as before, you encounter some issue as shown belowresults and compare their difference:<source lang="'bash"'>[rchan@centos7 lab8]$ fab --fabfile=fabfile.py -H 192.168.122.169 performSoftwareUpdate[192.168.122.169] Executing task 'performSoftwareUpdate'[192.168.122.169] run: yum update -y[192.168.122.169] out: Loaded plugins: fastestmirror, langpacks[192.168.122.169] out: You need to be root to perform this command.[192.168.122.169] out: Fatal error: run() received nonzero return code 1 while executing!removePackage
</source>
== Part 2: Create remote task for updating rpm packages ==
: Add a new function called "updatePackage" to your fabfile.py according to the following requirements:
:* Accept optional function argument as the rpm package name
:* If no function argument was given when called, default to all the packages installed
: The output of the updatePackage when executed, should produce similar output as shown below:
:1. Update a single package:<source lang='bash'>
fab updatePackage:tree
</source>
: Save the fabfileSample output:<source lang='bash'>[raymond.py with the change and run it againchan@mtrx-node05pd lab8]$ fab updatePackage:tree[myvmlab.senecacollege.ca] Executing task 'updatePackage'[myvmlab.senecacollege.ca] sudo: yum update tree -y[myvmlab.senecacollege.ca] out: If you see the sudo password prompt again, make sure that you can ssh :[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror[myvmlab.senecacollege.ca] out: Loading mirror speeds from your controller as a regular user to your worker vm as root without passwordcached hostfile[myvmlab.senecacollege.ca] out: * base: less.cogeco.net[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net[myvmlab.senecacollege.ca] out: The other way is to replace all the run() function calls No packages marked for commands that need superuser privilege by the sudo() function calls in your fabfileupdate[myvmlab.py. You are asked to investigate this in the final investigation of this labsenecacollege.ca] out:
Done.Disconnecting from myvmlab.senecacollege.ca: Recall that in our OPS courses we've been using iptables instead of firewalld, which is installed by default in CentOS7200.. Let's make sure that our workers have that set up as well. In the same '''fabfiledone.py''' you've been using all along, add a new function like this: : <source lang="python"># Will uninstall firewalld and replace it with iptablesdef setupFirewall(): run("yum -y -d1 remove firewalld") run("yum [raymond.chan@mtrx-y -d1 install iptables-services") run("systemctl enable iptables") run("systemctl start iptables")node05pd lab8]$
</source>
: That should by now look pretty obvious. On the worker you're going to uninstall firewalld, install iptables, and make sure that the iptables service is running2. : Execute the function for worker1 and double-check that it worked.Update all installed package: <font colorsource lang='red'>'''**Warning**bash''' </font>Do not do this on your vm on myvmlab. If you do, you may lock yourself out for good. === Check firewall configuration === fab updatePackage: To check your firewall configuration your remote worker, you can retrieve its current configuration by creating another Fabric task called "getFirewallConfigure(). Let's put the following code to your fabfile.py:<source lang="python">def getFirewallConfig(): fw_config = run("iptables -L -n -v") print(fw_config)
</source>
: The following output had been trimmed, only showing the first few lines:<source lang='bash'>
[myvmlab.senecacollege.ca] Executing task 'updatePackage'
[myvmlab.senecacollege.ca] sudo: yum update -y
[myvmlab.senecacollege.ca] out: sudo password:
[myvmlab.senecacollege.ca] out: Loaded plugins: fastestmirror
[myvmlab.senecacollege.ca] out: Loading mirror speeds from cached hostfile
[myvmlab.senecacollege.ca] out: * base: less.cogeco.net
[myvmlab.senecacollege.ca] out: * extras: centos.mirror.ca.planethoster.net
[myvmlab.senecacollege.ca] out: * updates: less.cogeco.net
...
Verifying : Try to run the getFirewallConfig() task the same way as beforesystemd-219-73.el7_8.5.x86_64 53/54 Verifying : Troubleshoot if you encounter any issuesystemd-libs-219-73.el7_8.5.x86_64 54/54
Installed: After completing all the previous parts of the lab kernel.x86_64 0:3.10.0- you should have a working fabfile1127.py with three working functions: getDiskUsage(), performSoftwareUpdate() and getFirewallConfig()13.1.el7
</source>
= Lab Exercise: Test your updated fabfile.py until you get the same result as when you apply it to your own worker VM. == Create a Fabric task called makeUser() ==: Study the Fabric API run(), sudo(), and put() and utilize them to create a new task called makeUser()
: The makeUser() function should perform the following:
:Add the makeUser() to your final version of fabfile.py.
:Test Run the new task makeUser() on your local VM first, and deploy to your vm on myvmlab.:After the successful deployment of the Verify and confirm that your new makeUser() task on your vm on myvmlab, ask your professor to verify and confirm that the new user account "ops435p" on myvmlab has been created is working correctly.
= LAB 8 SIGN-OFF (SHOW INSTRUCTOR) =
:'''Have Ready to Show Your Instructor:'''* Complete all the parts of the lab and upload the version of your fabfile.py which works on your vm on myvmlab to Blackboardby the due date.
[[Category:OPS435-Python]][[Category:rchan]]
