|
|
| Line 29: |
Line 29: |
| | | | |
| | = Fedora-ARM Projects = | | = Fedora-ARM Projects = |
| − |
| |
| − | == System Administration Tools for the ARM Build Farm ==
| |
| − |
| |
| − | The [[Fedora ARM Koji Buildsystem]] or "farm" consists of 23 ARM builder systems which will grow to 38 by the end of this semester. As it grows, the need for efficient system management tools increases. The previous semesters' students started the work of setting up [http://www.nagios.org/ nagios] (monitoring) and [[https://fedorahosted.org/func/ func]] (group control) tools.
| |
| − |
| |
| − | This project involves configuring these tools to work with all of the systems in the ARM build farm, as well as setting up and configuring the [http://www.puppetlabs.com/ puppet] (configuration management) tool.
| |
| − |
| |
| − | * Maximum number of students: 2
| |
| − | * Skills required: Linux system administration, problem solving, documentation writing
| |
| − | * Resources: wiki notes from previous semesters (e.g., [[How to Setup and configure Nagios]]), [[Fedora ARM Koji Buildsystem]], [[CDOT Development Systems]]
| |
| − | * Expected result: nagios, func, and puppet working across the entire ARM build farm; documentation on how to use these tools on the farm and how to add additional devices
| |
| − | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]]
| |
| − |
| |
| − | == Koji Hub on ARM ==
| |
| − |
| |
| − | The [[Fedora ARM Koji Buildsystem]] system uses [[CDOT_Development_Systems#Machine_names.2C_IPs.2C_and_Status|HongKong]], an x86_64 system, as the [[:fedora:Koji|Koji]] hub, along with a group of [[Fedora ARM Secondary Architecture/ARM hardware|ARM builders]].
| |
| − |
| |
| − | Ideally, it would be nice to prove the ability of the Fedora-ARM project to be entirely self-hosting by using an ARM system as the Koji hub (this is sometimes called "Eating your own dogfood" in the industry). This project involves configuring the [http://www.globalscaletechnologies.com/t-openrdcdetails.aspx OpenRD-Client] system as Koji hub and determining if this is a viable configuration.
| |
| − |
| |
| − | * Maximum number of students: 1
| |
| − | * Skills required: Linux system administration, problem solving, documentation writing
| |
| − | * Resources: wiki notes from previous 2 semesters, access to the OpenRD and a GuruPlug or BeagleBoard
| |
| − | * Expected result: koji-hub running on the OpenRD; a recommendation on whether the OpenRD is suitable for use as a hub for the Fedora-ARM project
| |
| − | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]]
| |
| − |
| |
| − | == Device Support and Testing: PandaBoard ==
| |
| − |
| |
| − | Various ARM devices need different driver sets and/or kernels. This project will test the Fedora-ARM system on the [http://pandaboard.org/ PandaBoard], creating a kernel that works well with it, and figuring out how to use as many of the built-in peripherals as possible.
| |
| − |
| |
| − | * Maximum number of students: 1
| |
| − | * Skills required: Linux system administration, kernel building, research, documentation writing
| |
| − | * Resources: a [http://pandaboard.org/ PandaBoard], notes from other [http://www.omappedia.com/wiki/PandaBoard PanadaBoard support projects],
| |
| − | * Expected result: a kernel (or kernels) for use with the PanadaBoard and the Fedora 12 or Fedora 13 root filesystems; user documentation on how to set up the PandaBoard with Fedora; wiki notes on setting up other ARM devices
| |
| − | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]]
| |
| − |
| |
| − | == Add PandaBoards to the Fedora-ARM Build System ==
| |
| − |
| |
| − | We have 15 [http://pandaboard.org/ PandaBoards] on order for the [[Fedora ARM Koji Buildsystem|Fedora-ARM build farm]]. These machines need to be configured and added into the farm, and then optimized to build packages as quickly as possible.
| |
| − |
| |
| − | * Maximum number of students: 2
| |
| − | * Skills required: system administration, network administration, troubleshooting, benchmarking
| |
| − | * Resources: PandaBoard systems
| |
| − | * Expected result: a filesystem image and documented standard operating procedure for adding PanadaBoards to the build farm; new PandaBoards actively building
| |
| − | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]]
| |
| | | | |
| | == iSCSI/AoE Support * == | | == iSCSI/AoE Support * == |
| Line 87: |
Line 43: |
| | * Expected result: iSCSI on ARM fixed and tested, and changes pushed upstream; AoE tested on ARM; report comparing iSCSI and AoE performance on ARM; ARM buildsystem configured to use a high-performing iSCSI or AoE storage solution in place of the existing NFS system | | * Expected result: iSCSI on ARM fixed and tested, and changes pushed upstream; AoE tested on ARM; report comparing iSCSI and AoE performance on ARM; ARM buildsystem configured to use a high-performing iSCSI or AoE storage solution in place of the existing NFS system |
| | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]] | | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]] |
| − |
| |
| − | == CreateRepo Performance Improvements * ==
| |
| − |
| |
| − | The [[Fedora ARM Koji Buildsystem|Fedora-ARM build farm]] consists of the koji-hub/koji-web system (on [[CDOT_Development_Systems#Machine_names.2C_IPs.2C_and_Status|HongKong]], an x86_64 machine) and a group of ARM builders. HongKong also handles the [http://createrepo.baseurl.org/ CreateRepo] tasks, which heavily load that machine. It might make sense to move those tasks to another machine, but doing so makes the CreateRepo jobs take a lot longer (15 minutes on HongKong vs. 55 minutes on another x86_64 server). This is presumably due to the overhead of sharing files between HongKong and the other server over NFS on the 100 Mbps Seneca network.
| |
| − |
| |
| − | This project involves figuring out how to run the CreateRepo jobs more quickly. Possible solutions include a 1 Gbps LAN, a redistribution of the file storage or a change to a different file storage technology, or optimizing the CreateRepo tasks on HongKong for best speed.
| |
| − |
| |
| − | * Maximum number of students: 1
| |
| − | * Skills required: system administration, benchmarking
| |
| − | * Resources: CDOT server systems (HongKong, Ireland, Scotland)
| |
| − | * Expected result: significant reduction in CreateRepo times, especially when multiple CreateRepo tasks are running
| |
| − | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]]
| |
| − |
| |
| − | == RPM-based Kernels for Fedora ARM * ==
| |
| − |
| |
| − | On PC architecture systems (x86_64 and i386), Fedora manages and updates kernels as RPM packages, which modify [http://www.gnu.org/software/grub/grub-legacy.en.html grub] boot parameters and the initial ram disk ([http://en.wikipedia.org/wiki/Initrd initrd], configured by [[:fedora:Dracut|dracut]]).
| |
| − |
| |
| − | On Fedora-ARM systems, the kernel is not managed via RPMs, grub is not used, and the initrd system is rarely used.
| |
| − |
| |
| − | This project involves understanding how the PC (i386/x86_64) kernel/boot/initrd system works, determining which pieces can be reused on Fedora-ARM and which pieces need to be adapted or replaced, and implementing RPM-based kernel management for ARM.
| |
| − |
| |
| − | * Maximum number of students: 3
| |
| − | * Skills required: Linux system administration, script writing, RPM packaging, kernel building, initrd debugging
| |
| − | * Resources: an ARM system
| |
| − | * Expected result: RPM-based Kernels work on Fedora-ARM, with changes committed upstream; documentation about the differences between kernel management on ARM and on PCs
| |
| − | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]]
| |
| − |
| |
| − | == Fedora-ARM Communication * ==
| |
| − |
| |
| − | We're not doing a great job of communicating how the [[:fedora:Architectures/ARM|Fedora-ARM]] project is doing, especially how [http://arm.koji.fedoraproject.org current builds] are progressing. The [http://arm.koji.fedoraproject.org/status status page] is very bare-bones and doesn't convey a lot of information, the Fedora-ARM wiki pages (on the [[:fedora:Architectures/ARM Fedora]] and [[:Category:Fedora_ARM_Secondary_Architecture|Seneca]] wikis) need to be made more useful to prospective users, and we need an effective communication strategy with the rest of the [http://fedoraproject.org Fedora community]. This project involves writing some web scripts to create a easy-to-use, informative status page (showing, for example, the current state and progress of the ARM builds), creating user documentation on the Fedora wiki, and fostering effective communication within the Fedora-ARM project and the larger Fedora community.
| |
| − |
| |
| − | * Maximum number of students: 2
| |
| − | * Skills required: Apache administration, script-writing, effective written communication skills
| |
| − | * Resources: the web server on HongKong, various data sources within the Fedora-ARM build system, Fedora project communication tools, access to ARM systems
| |
| − | * Expected result: a useful (easy-to-use, informative) and automatically-updated Fedora-ARM status page; improved user documentation on the Fedora wiki (e.g., how to set up Fedora-ARM on common devices); better communication on the arm@lists.fedoraproject.org mailing list and the #fedora-arm IRC channel
| |
| − | * Initial contacts: [[User:Chris Tyler|ctyler]], [[User:Paul.W|PaulW]]
| |
| − |
| |
| − | == Automatic ExclusiveArch Addition/Removal * ==
| |
| − |
| |
| − | RPM packages may be specified as being suitable only for particular architectures through the use of ExclusiveArch and [http://www.rpm.org/max-rpm/s1-rpm-inside-tags.html#S3-RPM-INSIDE-EXCLUDEARCH-TAG ExcludeArch] tags in the spec file. If a base package, such as a language (interpreter or compiler) or a library does not build on a particular architecture, then dozens or hundreds of other packages cannot be built. These packages should all be marked as ExcludeArch in the upstream git package repository. Later, if that base package is updated to work on that arch, the ExcludeArch lines will need to be removed.
| |
| − |
| |
| − | This project involves writing a script that will mass-add or mass-remove ExcludeArch tags (or, if those tags exist, add or remove a particular architecture), pushing the changes to the upstream [[:fedora:Using_Fedora_GIT|git repo]].
| |
| − |
| |
| − | * Maximum number of students: 1
| |
| − | * Skills required: scripting (Python and/or bash), packaging
| |
| − | * Expected result: a script that will do mass adds/removals of ExcludeArch tags given a list of packages
| |
| − | * Resources: [https://fedorahosted.org/fedora-packager/ fedpkg libraries and source], Fedora package repo
| |
| − | * Initial contacts: [[User:Paul.W|PaulW]], [[User:Chris Tyler|ctyler]]
| |
| − |
| |
| − | == Fedora-ARM Package Building and Troubleshooting ==
| |
| − |
| |
| − | The [[Fedora ARM Secondary Architecture|Fedora-ARM]] project is building Fedora 13/14/15 for the ARM architecture. As this proceeds, various problems arise. For example:
| |
| − | * Some packages fail to build for ARM. They can be fixed up to build successfully on ARM, or if that's not possible, marked as unsuitable for ARM ([http://www.rpm.org/max-rpm/s1-rpm-inside-tags.html#S3-RPM-INSIDE-EXCLUDEARCH-TAG ExlcudeArch]).
| |
| − | * If a group of packages is built to work with a specific version of a library, and a newer version of that library replaces the older version, then the packages that rely on that library can break. This can often be remedied simply by rebuilding the broken package; in other cases, patches are required.
| |
| − |
| |
| − | Note that one package build issue will often block many other packages.
| |
| − |
| |
| − | This project involves working with other members of the Fedora ARM build team to resolve package build problems and get F13/F14/F15 released for ARM as soon as possible.
| |
| − |
| |
| − | * Maximum number of students: 4
| |
| − | * Skills required: packaging, troubleshooting
| |
| − | * Expected result: problems with the Fedora-ARM builds are cleared as quickly as possible; F14-ARM released by the end of the semester
| |
| − | * Resources: Fedora-ARM Koji build system, arm@lists.fedoraproject.org mailing list
| |
| − | * Initial contacts: [[User:Paul.W|PaulW]], [[User:Chris Tyler|ctyler]]
| |
| − |
| |
| − | = Fedora Projects =
| |
| − |
| |
| − | == Package the Weave Server * ==
| |
| − |
| |
| − | Mozilla Sync is a technology for synchronizing personal data (bookmarks, passwords, form values, and cookies) across multiple machines. It uses a server known as [https://wiki.mozilla.org/Labs/Weave/Sync/1.0/Setup Weave].
| |
| − |
| |
| − | This project involves packaging Weave for Fedora and getting it through the [[:fedora:Package Review Process|package review]] process. (Why package the Weave server? So that people can run a private version, either for enhanced security or for testing).
| |
| − |
| |
| − | * Maximum number of students: 1
| |
| − | * Skills required: Apache administration, packaging
| |
| − | * Resources: CDOT systems
| |
| − | * Expected result: the Weave server will be available in the main Fedora repositories (yum install weave)
| |
| − | * Initial contacts: [[User:Mhoye|mhoye]]
| |
| | | | |
| | == Package Hadoop == | | == Package Hadoop == |
| Line 176: |
Line 54: |
| | * Initial contacts: [[User:Chris Tyler|ctyler]] | | * Initial contacts: [[User:Chris Tyler|ctyler]] |
| | | | |
| − | == Package the WIX Toolchain * == | + | == Test Thumb2 with ARMv7hl * == |
| − | | |
| − | [http://wix.sourceforge.net/ WIX] is an open-source packaging system for Microsoft Windows software. It is used to prepare software packages that can be installed on a Windows machine. However, the WIX tools themselves can run on Linux, which is useful for cross-development (writing software on Linux for use on Windows, e.g., the virt-manager utilities for Windows virtual machine guests).
| |
| − | | |
| − | This project involves packaging WIX for Fedora and getting it through the [[:fedora:Package Review Process|package review]] process.
| |
| − | | |
| − | * Maximum number of students: 1
| |
| − | * Skills required: packaging
| |
| − | * Resources: [[CDOT Development Systems]]
| |
| − | * Expected result: the WIX software will be available in the main Fedora repositories (yum install wix)
| |
| − | * Initial contacts: [[User:Mhoye|mhoye]], sdowne
| |
| − | <!--
| |
| − | == fedpkg Test Suite ==
| |
| − | | |
| − | ''fedpkg'' is a new Fedora packager tool written by Jesse Keating; it's one of the main command-line tools that a packager will use. It needs a test suite, so that as new features are added, regressions can be detected.
| |
| − | | |
| − | * Maximum number of students: 2
| |
| − | * Initial contacts: [http://jkeating.livejournal.com/ Oxf13]
| |
| − | -->
| |
| − | | |
| − | == Koji Setup Documentation ==
| |
| − | | |
| − | The [[:fedora:Koji|Koji documentation]] needs an overhaul. This project involves reading the current documentation, updating and editing it, and testing it by setting up a Koji system.
| |
| − | | |
| − | * Maximum number of students: 2
| |
| − | * Skills required: writing, system administration
| |
| − | * Resources: [[CDOT Development Systems]], [[:fedora:Main Page|Fedora wiki]]
| |
| − | * Expected result: a complete, well-written guide to setting up a Koji system (from A-Z)
| |
| − | * Initial contacts: [[User:Chris Tyler|ctyler]], dgilmore
| |
| − | | |
| − | == AutoQA ==
| |
| − | [[:fedora:AutoQA|AutoQA]] is an automated test system for Fedora. At present there are event watchers for koji builds, bodhi updates, repo changes, and nightly installed images; these events trigger a small number of tests, but more tests are needed.
| |
| − | | |
| − | * Maximum number of students: 3
| |
| − | * Skills requires: Python scripting, scripting, system administration, packaging
| |
| − | * Resources: CDOT systems
| |
| − | * Expected results: additional tests for AutoQA, accepted/committed into the main AutoQA codebase
| |
| − | * Initial contacts: [[:fedora:User:JLaska|JLaska]] | |
| − | | |
| − | = Fedora-Mozilla Projects = | |
| | | | |
| − | == Repository Setup for Mozilla Nightlies and Betas * ==
| + | Thumb instructions mode is a 16-bit mode for ARM. It should provide a higher density for code (meaning that programs will be smaller). In the early versions of the ARM architecture, thumb mode was slower than regular instruction mode, but this should not be the case for newer processors (which support "Thumb2"). |
| | | | |
| − | Many web developers want access to the latest Firefox pre-releases, including the nightly builds and beta releases. Mozilla's build team wants to make these accessible as parallel-installable binaries, released through a Fedora-compatible repository. Last semester, a group of SBR600 students set most of this up; this project involves extending and improving their work.
| + | This project involves building a suitable set of packages in Thumb2 mode and in regular instruction mode and comparing: |
| − | <!-- Subprojects:
| + | # binary size |
| − | * Build configuration for the RPM files.
| + | # RPM package size |
| − | * Repository configuration RPMs.
| + | # build speed |
| − | * Getting SELinux to work with the nightlies.
| + | # execution speed |
| − | -->
| |
| − | | |
| − | See [https://bugzilla.mozilla.org/show_bug.cgi?id=600317 bug 600317].
| |
| | | | |
| | * Maximum number of students: 2 | | * Maximum number of students: 2 |
| − | * Skills required: system administration, scripting, packaging | + | * Skills required: packaging, system administration, benchmarking |
| − | * Resources: scripts and configuration from the previous semester | + | * Expected results: a recommendation to the Fedora-ARM project on whether or not to use Thumb2 for the armv7hl architecture |
| − | * Expected result: fully-functioning repository configuration ready for installation on Mozilla's systems; well-written documentation
| + | * Initial contacts: [[User:Chris Tyler|ctyler]], Jon Masters (jonmasters) |
| − | * Initial contacts: [[User:Armenzg|armenzg]], [[User:Chris Tyler|ctyler]] | |
| − | | |
| − | <!-- = Mozilla Projects =
| |
| − | -->
| |
| − | <!-- == hgtools ==
| |
| − | What if the Mozilla builders were better at managing all the different working directories (from Mercurial checkouts) that we need at any give time? If you look at [https://bugzilla.mozilla.org/show_bug.cgi?id=589885#c11 this conversation from IRC] you can see the benefits of this and [https://bug506404.bugzilla.mozilla.org/attachment.cgi?id=476270 a patch] that has the initial work.
| |
| − | Initial contacts: [[User:Armenzg|Armenzg]] -->
| |
| − | | |
| − | <!-- == MozHarness ==
| |
| − | | |
| − | Imagine that we did not have to touch the Mozilla buildbot factories but instead we maintained a bunch of script for all the different jobs they run?
| |
| − | | |
| − | It would be good if we could create scripts that told a machine how to generate an optimized build, a debug build, unit tests, talos runs, locale repackages.
| |
| − | If you look in the [http://hg.mozilla.org/build/tools/file/tip/scripts tools/scripts] repo you can see that we have a simple shell file to do this for the fuzzing automation. The buildbot factory that calls it is called [http://hg.mozilla.org/build/buildbotcustom/file/a70b38b40088/process/factory.py#l7895 ScriptFactory] and it is very simple.
| |
| − | | |
| − | Initial contacts: [[User:Armenzg|Armenzg]]
| |
| − | -->
| |
| − | <!-- == End-to-end project ==
| |
| − | How can we build faster and provide tests results faster to our developers?
| |
| − | That is what we are trying to figure out and we will be adding bugs to this [https://bugzilla.mozilla.org/show_bug.cgi?id=598175 tracking bug] to optimize
| |
| − | our infrastructure.
| |
| − | | |
| − | Initial contacts: [[User:Armenzg|Armenzg]]
| |
| − | -->
| |
| − | <!-- == I don't like waiting - give me a CPU! ==
| |
| − | We have a hundred jobs running per hour and we sometimes have jobs that have to wait for something before getting started. If we optimized the load we could use the build resources more effectively. I will be adding bugs to this [http://www.themoviemind.com/wp-content/uploads/2008/08/chuck-norris-2.jpg tracking bug] to reduce our load on our pools and therefore reduce our waiting times.
| |
| − | | |
| − | Initial contacts: [[User:Armenzg|Armenzg]]
| |
| − | -->
| |
Introduction
This is a list of potential projects related to the SBR600 course that need people.
Students: Please select a project that you're interested in and add an entry to the project table/participants page.
Open Source Community Members: We welcome your recommendations for potential projects. Please create an account on this Wiki and create a description for your proposed project below. Please list your contact info (just an IRC or FAS2 name is OK) as well as links to any related web pages as Resources for the proposed project. (Questions? Ask Chris Tyler).
Notes
Each project listing contains a general description, plus this information:
- Maximum number of students - Do not exceed this number without approval from your professor.
- Skills required - This is a rough list of some of the skills required for this project. This list may be incomplete or inaccurate, but it will give you a starting point in evaluating whether this project is a good fit for you. It is not assumed that you will have all of these skills at the outset of the project -- some of them will be picked up as you do the project.
- Resources - An initial list of computer and information resources to get started on the project.
- Expected result - A rough indication of what is expected at the conclusion of the project.
- Initial contacts - Who to initially talk to about this project. These contacts may refer you on to other people with the respective open source communities.
You will have an opportunity to investigate, expand upon, and fine-tune this information as you prepare your initial project plan. For example, you may come up with a more detail list of expected results (deliverables), resources, and contacts during your planning.
Individual Deliverables
Note that when multiple people are working on the same project, they will have independent deliverables -- it's not really group work, but rather separate, closely related projects.
This is a sample project stub. You can use the template for Sample Project in order to create a project page for one of the projects listed below. This is how you 'sign-up' for a project.
NOTE: if someone has already created the project page, speak to this person and see if you can join them. If so, simply add your name to the Project Leader(s) section on the project page. Otherwise, you can become a contributor later.
Fedora-ARM Projects
iSCSI/AoE Support *
iSCSI (SCSI over TCP/IP) and AoE (ATA-over-ethernet) are different SAN protocols that can be used on a standard ethernet network.iSCSI did not work reliably in Fedora 12 on ARM systems, but will be needed by future ARM server systems. AoE has not been well-tested on ARM systems.
Goals of this project:
(1) iSCSI and AoE need to be tested for stability and performance.
(2) The ARM builders, which currently use loopback-mounted filessystems on top of NFS, should be reconfigured to use iSCSI or AoE (whichever is the optimal solution) providing it is faster than the current solution.
- Maximium number of students: 2
- Skills required: Linux system administration, debugging and troubleshooting, kernel building, benchmarking, documentation writing
- Resources: an ARM system, CDOT PC systems
- Expected result: iSCSI on ARM fixed and tested, and changes pushed upstream; AoE tested on ARM; report comparing iSCSI and AoE performance on ARM; ARM buildsystem configured to use a high-performing iSCSI or AoE storage solution in place of the existing NFS system
- Initial contacts: ctyler, PaulW
Package Hadoop
Apache Hadoop is a set of tools used for large-scale distributed computing. It would be great to get this packaged for Fedora.
- Maximum number of students: 3
- Skills required: packaging, system administration; familiarity with Java programming/packaging
- Resources: CDOT Development Systems, devel mailing list (some work on Hadoop packaging has already been done)
- Expected result: the three Apache Hadoop subprojects (Hadoop Common, HDFS, and MapReduce) will be available in the main Fedora repositories (yum install hadoop-common hdfs mapreduce)
- Initial contacts: ctyler
Test Thumb2 with ARMv7hl *
Thumb instructions mode is a 16-bit mode for ARM. It should provide a higher density for code (meaning that programs will be smaller). In the early versions of the ARM architecture, thumb mode was slower than regular instruction mode, but this should not be the case for newer processors (which support "Thumb2").
This project involves building a suitable set of packages in Thumb2 mode and in regular instruction mode and comparing:
- binary size
- RPM package size
- build speed
- execution speed
- Maximum number of students: 2
- Skills required: packaging, system administration, benchmarking
- Expected results: a recommendation to the Fedora-ARM project on whether or not to use Thumb2 for the armv7hl architecture
- Initial contacts: ctyler, Jon Masters (jonmasters)
