1
edit
Changes
no edit summary
[[Category:Fedora]][[Category:SOP]][[Category:Fedora ARM Secondary Architecture]]{{Draft}}
=Seneca CDOT ARM Project: Management Server Backup Strategy and Recovery Planplan=Contact info:
* Phone: X33463
* Email: fedora-arm at senecacollege dot ca
As for the management servers, there are five major ones that are critical to this operation. For a complete list of servers and the services those offer please consult the necessary documents. These four servers are Australia (Koji builders temporary work storage), Chile (The complete ARM repository), Hongkong (Koji hub and web interface), Iraq (Central backup management and storage) and Ireland (Koji database).
==Description==
The backup system has been broken into two custom scripts. First one is to create a local copy of the configurations and to back up the repository to removable drives, it is called blaze. The second script is called wildfire and it is to synchronize those local copies to different locations over the network. The servers get the copies of those scripts from the central backup management and storage server (iraq) via scp and runs blaze first and then wildfire.
===Backup Script Source===
The scripts are located at (any server with backup enabled):
/usr/local/bin/
===Backup Directory Structure===
The backups (both local copies and networked synced copies) are stored inside /archive/blaze-backup (/archive/repo-backup for the repository sync) in each servers respective directory. Example:
[root@australia ~]# tree /archive
/archive
├── blaze-backup
│ ├── australia
│ │ ├── australia-cron-Feb-27-2012.tar.bz2
│ │ ├── australia-etc-Feb-27-2012.tar.bz2
│ │ └── australia-userlocalbin-Feb-27-2012.tar.bz2
│ ├── chile
│ │ ├── chile-cron-Feb-27-2012.tar.bz2
│ │ ├── chile-etc-Feb-27-2012.tar.bz2
│ │ └── chile-userlocalbin-Feb-27-2012.tar.bz2
│ └── ireland
│ ├── ireland-cron-Feb-27-2012.tar.bz2
│ ├── ireland-etc-Feb-27-2012.tar.bz2
│ ├── ireland-postgresql-Mar-27-2012.sql.bz2
│ └── ireland-userlocalbin-Feb-27-2012.tar.bz2
└── lost+found
===Space Consideration and Allocation===
For backing up the configurations and keeping copies of those for the previous 30 days and also allocating room for possible synced copies from other servers, minimum space requirements is 10GB. To accommodate for the synced copy of the repository that could go up to 2TB
===Data Retention===
* Removable drives will keep data up to 3 days
* Network synced drives/servers will keep data up to 30 days
===Initial Setup===
This applies to any new server that will be backed up (this process could become a script eventually)
* Create a user named "backup"
* Give that user certificate access through ssh (to and from) the other servers (in the backup group). Tutorials available [http://zenit.senecac.on.ca/wiki/index.php/OPS235_Lab_7 here]
* Also give cert access to the root of each server so that those can remote access to the "backup" user of other servers.
* Add a new lvm called archive that will be mounted as /archive on startup (add an entry to /etc/fstab). LVM tutorial available [http://zenit.senecac.on.ca/wiki/index.php/OPS235_Lab_4 here]
* Create the following directories and give "backup" write permission to those directories
/archive/blaze-backup
/archive/repo-backup
/var/log/blaze-backup
* Create the following file and give "backup" write permission to that file and also add the hostname for the admin server (iraq) in that file (please see offered services section for other options)
/etc/sysconfig/blaze
* In the admin host, add the hostname in the buddyhost section in blaze and wildfire with a random syncpeer. (optional, please see offered services section for more options)
* Add the following entry to /etc/aliases (use proper email format):
blaze-wildfire: fedora-arm at senecacollege dot ca
* Schedule backup as root (crontab entry). It will look something similar to this:
20 2 * * * ADMHOST=$(cat /etc/sysconfig/blaze); BKU=backup; export ADMHOST BKU; scp $BKU@$ADMHOST:/usr/local/bin/blaze \
/usr/local/bin/; chown -R $BKU:$BKU /usr/local/bin/blaze; chmod 755 /usr/local/bin/blaze; /usr/local/bin/blaze
* If the server is going to be a reposync host then follow the first three steps to create a user account called apache and give it access to and from the repohost
==Backup script explanation=Offered/Backedup Servers/sourceServices===* Please add the service type in the appropriate row in the "Define the names of the remote machines and services" section in each script** PostgreSQL Database Server (DBHOST)(Possibly one - Ireland)** Repository (REPOHOST) (Possibly one - Chile)** Server that has backup to removable drive capabilities (RBDHOST) (Chile or Ireland or Scotland)** Central Sync Server (SYNCHOST) (Iraq, Scotland and Romania)** Repository Backup (COPYREPOHOST) (Scotland Romania)** Sync Peer (BUDDYHOST) (All)** Custom Log and Mail Server (AGGREGHOST) (Iraq)** Administrator Host (ADMHOST) (Iraq, for now)** All related servers under blaze wildfire system (ALLSYS) (Australia Chile Hongkong Iraq Ireland Scotland Romania)
== Conclusion==ARM is still a developing technology and expectations are that The Fedora ARM Project at Seneca CDOT will be highly active for the next few years. Depending on the level of activity the backup strategy in place may become out of date in a matter of months or it may need to be supplemented with other solutions.==Appendix=====Admin Configuration===This file specifies which server is the admin host. It is available in each host at /etc/sysconfig/blaze===Backup layout exampleLayout=== {| class="wikitablemediawiki" border="1"cellspacing="2" width="100%"|:'''Source Server'''|:'''Backup Source'''|+ :'''Type of Data'''|:'''LAN Targets'''|:'''Backup Location'''|-|:Hongkong|:/etc/:/var/spool/cron/:/usr/local/bin/:/var/lib/bcfg2/|:Configurations:Cronjobs:Scripts:BCFG2 DB|:Iraq and Scotland|:/archive/blaze-backup layout example/hongkong/! source !! files or folder !! type of data !! destination !! location|-|:Ireland|:/etc/:/var/spool/cron/:pg_dump:/usr/local/bin/|:Configurations:Cronjobs:PostgreSQL DB:Scripts|:Iraq and Chile|:/archive/blaze-backup/ireland/|-|:Australia|:/etc/:/var/spool/cron/:/usr/local/bin/|:Configurations:Cronjobs:Scripts|:Iraq and Hongkong|:/archive/blaze-backup/australia/|-|:Chile|:/etc/:/var/spool/cron/:/mnt/koji/:/usr/local/bin/|:Configurations:Cronjobs:Repo:Scripts|:Iraq and Australia:(Repo Sync in Scotland and Romania only)|:/archive/blaze-backup/chile/:/archive/repo-backup/|-|:Iraq|:/etc/:/var/spool/cron/:/usr/local/bin/|:Configurations:Cronjobs:Scripts|:Chile|:/archive/blaze-backup/iraq/
|-
| hongkong |:Scotland| :/etc /:/var/spool/cron/:/usr/local/bin/:/var/lib/etherpad-lite|:Configurations:Cronjobs:Scripts:Etherpad Lite DB| configurations || backup_server |:Iraq and Ireland| :/vararchive/blaze-backup/scotland/
|-
| chile |:Romania| :/etc/:/var/libspool/db || database |cron/:/usr/local/bin/| backup_server :Configurations:Cronjobs:Scripts|:Australia, Iraq and Scotland| :/vararchive/blaze-backup/romania/
|}
===Backup Location Capacity===
hongkong
Filesystem Type Size Mounted on
/dev/mapper/hk-archive ext4 5.0G /archive
iraq
Filesystem Type Size Mounted on
/dev/mapper/iraq-archive ext4 50.0G /archive
ireland
Filesystem Type Size Mounted on
/dev/mapper/ireland-archive ext4 1.2T /archive
australia
Filesystem Type Size Mounted on
/dev/mapper/australia-archive ext4 9.9G /archive
chile
Filesystem Type Size Mounted on
/dev/mapper/chile-archive ext4 9.9G /archive
scotland
Filesystem Type Size Mounted on
/dev/mapper/scotland-archive ext4 2.2T /archive
===Log Files===
Available on all servers running blaze
/var/log/blaze-backup.log
/var/log/wildfire-sync.log
Available on central/aggregator/admin server
/var/log/blaze-wildfire.log