Difference between revisions of "Winter 2010 Posters/ARM Intro"

From CDOT Wiki
Jump to: navigation, search
(Our Objective)
 
(41 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
 
= Project Name =
 
= Project Name =
  
Koji Hub Installation and Configuration
+
Fedora/ARM
  
 
= Name =
 
= Name =
 +
Arlene Daniel
  
Paul Whalen
+
= Our Goal =
  
= The Koji Build System =
+
To enable the current version of Fedora to be used on low-power ARM computer systems. We will do this by setting up the Fedora build system, named ''Koji'', on a cluster of ARM computers. These computers will build an ARM-compatible version of each of the approximately 15,000 Fedora software packages. The ''Koji-Shadow'' software will be used to monitor the main Fedora Koji system -- which builds software packages for PCs -- so that every time a new software package is prepared for PCs it will automatically also be built on our Koji system for ARM computers. The resulting packages will be placed in a publicly-accessible repository so that they can be used worldwide.
  
At the beginning of the Winter 2010 semester the Software Build and Release 600 class set out to create a Koji build system for the ARM architecture. Koji is the software that builds and tests packages for the Fedora Project. Currently we are aware of one other Koji Build system for the ARM architecture which is located in Germany.  
+
= About The ARM Architecture =
 +
ARM chips are the most popular 32-bit RISC (Reduced Instruction Set Computer) ''microprocessors'' --the brain of the computer that carries out its most complex functions -- and is mainly used in ''low power applications'' -- not much power is needd to operate the device --. Approximately 1.6 billion are being made each year by different manufactures and are sold under a number of different brand names, brands like ARM, StrongARM, Armada, Cortex, OMAP, Sheeva, Snapdragon and XScale to name a few. The chips created are used in devices like cellphones, routers, NAS boxes, embedded controllers, tablets, netbooks and many others.
  
The plan was to initially set up all components of the Koji build system on one server, with the Koji builder Daemons running on Virtual ARM Machines and physical hardware as it arrives. After installation and configuration of the PostgreSQL, the Koji Hub would be configured, followed by the other components of Koji including the Koji Web, Kojira and finally the set up of the actual builders themselves.
+
= The ARM Hardware =
 +
The devices listed below are some of the devices that use a ARM processor:
 +
    * One Laptop Per Child (OLPC) computers is a ''subnotebook'' -- a notebook that is smaller than the average notebook -- that's donated
 +
      to developing countries, with the hope that, every primary school child could have their own laptop to explore the world. The OLPC is
 +
      very inexpensive to produce, costing developers fifty dollars($50) for everyone produced.
 +
    * The Touch Book - developed by Always Innovating -- is a lightweight portable device that has a detachable magnetic keyboard and
 +
    * The SheevaPlug - a plug computer designed to run network-based software devices are some devices that use an ARM processor.
  
= Koji Components =
+
These devices also use a Linux based OS and since Fedora is used on the XO units, having a reliable ARM build of Fedora is increasingly important.
  
Koji Hub - an XML-RPC server that receives and relies upon calls from the Koji Builder Daemons and other Koji components to initiate communications.
+
= About Fedora =
 +
Fedora is an open source operating system using ''RPM-based'' -- linux distributions that use a tool to automate the process of installing, upgrading, configuring and removing software packages, built on top of the Linux kernel. ''Software packages'' -- are computer software that are stored in an archived format to install programs.-- ''Archived format'' -- a compressed format of storing files.-- There are presently over 15000 software packages availabe for download to linux community of users. It is developed and maintained by a ''community-support'' -- a group of programmers who contribute to the development of Fedora -- known as the Fedora Project which is sponsored by Red Hat. The Fedora Project's mission is "To lead the advancement of free and open source software and content as a collaborative community." Fedora’s main aim is to be a leader in the use and distribution on software designed as ''open source''-- the source code the proogram is readily available. -- Fedora has a release every six months and a maintenance period of about thirteen months for each of its releases. Fedora 12 was released November 11, 2009 and Fedora 13 has an expected release date of May 18, 2010.
  
Koji Web - an interactive web interface that runs on mod_python modules in the Apache Web Server. The Koji Web provides an interface for Koji users to see the status of queued builds and information for trouble shooting unsuccessful build attempts.
+
= Supported Architecture =
  
Koji Builder Daemon - the build daemon that runs on each of the Koji build machines and is responsible for polling the Koji Hub for build requests. Once a request is received for the Koji Hub the Koji Builder will initiate a test build of the software package ensuring there are no errors, if errors are detected they are logged for further analysis.
+
Fedora supports two primary architectures:
 +
  i386 - 32-bit Intel/AMD-compatible
 +
  x86_64 - 64-bit Intel/AMD-compatible
  
= Components Required By Koji =
+
There are also a number of secondary archs:
Apache Web Server - The most popular web server on the internet Apache is responsible for serving the pages provided by the Koji Web scripts. Apache is configured to use a variety of modules including SSL and mod_python. The SSL modules facilitates authentication of users and the different components of the Koji Build System.
+
  arm - A widely-used, low-power processor family commonly used for embedded and mobile applications
 +
  ia64 - Itanium
 +
  pa-risc - HP Precision Architecture
 +
  ppc - 32-bit Power PC
 +
  ppc64 - 64-bit Power PC
 +
  s390 - IBM mainframes (including z90 and z9)
 +
  sparc - Sun RISC architecture
  
PostgreSQL - PostgreSQL is the back-end database used by the Koji Build System. It stores tables of information including users, packages and build categories.
+
Primary architectures are instrumental in the design and development of any fedora release. During the development of a fedora release if any of the primary architecture builds fail, the packages cannot be released. Fedora package maintainers are required to make sure that their package builds properly for this architecture. Users of any linux distrbutions use one of these primary architectures.
  
= Problems Encountered =
+
Secondary architectures are architectures  where the hardware for the secondaries are maintained by the people producing the builds for those secondary architectures. Build failures on secondary architectures are not fatal, unlike builds in the primary architectures, the failure repairs on the secondary architectures are often released several months after the primary ones.
* Problems using the PostgreSQL database
 
  
* After what appeared to be a successful installation and configuration, Koji builders were listed as not being fully functional, this appeared to be an issue with the Koji Hub but was later determined to be a lack of disk space on the Koji builders.
+
= The ARM Architecture =
 +
The ARM architecture is increasingly important, but there's a lot more that could be done in terms of update frequency, number of packages successfully built, transparency of process, and integration with the other Fedora build processes.
 +
 +
Fedora policy requires that packages be natively compiled. We use a cluster of ARM hardware and QEMU virtual machines to build the packages natively for ARM.
  
* Authentication using SSL certificates
+
The baseline ARM CPU architecture that we have chosen to support is ARMv5TE, Little Endian, Soft-Float, EABI. We believe that this provides a nice baseline and that the pre-built packages and root file system images. You should be able to use this on many of the modern ARM CPUs, including XScale, ARM926 and ARM-11, etc.
  
* Various configuration issues
+
Although we do not provide such binaries, the sources also lend themselves for building for pre-ARMv5TE hardware. The same is true for big-endian CPUs.
  
= Results =
+
We currently do not plan to provide an installer, ISO images, or a kernel. Unlike in the x86 world, different ARM CPU families require different kernel images. Likewise, it is not entirely clear whether it makes sense to provide an installer or ISO images.
At the close of the semester we currently have a functional Koji Build system. The Koji Hub is successfully dispatching packages to the Koji builder Daemons and Kojira is creating new repositories for use with the ARM architecture. Eight VM's and the Open-RD Client are online and are listed as Ready and Enable using the Koji command line tool and the web interface through Koji Web.
+
 
 +
 
 +
= Our Objective =
 +
The plan was to set up a Koji builder on a system of eight virtual ARM machines that is based on the CDOT system HongKong. The ARM builders are using ''QEMU emulation'', -- a processor emulator --  which will be replaced by ARM Hardware when it arrives. The ARM Hardware that was purchased was an OpenRD platform. This platform is powered by the fastest ARM architecture available. The platform allows us to create and complete our designs.
 +
 
 +
The HongKong system also housed a PosegreSQL database and the Koji hub, Koji Web and Kojira. Other components like Apache and Mock also had to be installed on the system.
  
 
= Acknowledgements =
 
= Acknowledgements =
*Dennis Gilmore
 
 
 
*Chris Tyler
 
*Chris Tyler
  
 
= Links =
 
= Links =
 
+
* Fedora Project -  https://fedoraproject.org/wiki/Architectures/ARM
* Fedora Project -  http://fedoraproject.org/wiki/Koji/ServerHowTo
 
* Paul Whalen's SBR600 Blog - http://paulfedora.wordpress.com/
 
* Software Build and Release 600 wiki - http://zenit.senecac.on.ca/wiki/index.php/SBR600
 
 
 
 
* Fedora Arm Secondary Architecture wiki - http://zenit.senecac.on.ca/wiki/index.php/Fedora_ARM_Secondary_Architecture
 
* Fedora Arm Secondary Architecture wiki - http://zenit.senecac.on.ca/wiki/index.php/Fedora_ARM_Secondary_Architecture
  
= Logos =
+
= Images =
 
+
* Fedora - http://www.mynetbookworld.com/wp-content/uploads/2009/09/fedoraLogo.jpg
* Fedora
+
* ARM Processor - http://co-designsystems.com/images/arm-processor.jpg
* Seneca
+
* OLPC - http://futuresgroup.files.wordpress.com/2009/07/olpc-5.jpg
* Apache
+
* SheevaPlug - http://upload.wikimedia.org/wikipedia/commons/7/7b/SheevaPlug_with_external_drive_enclosure.jpg
 
+
* Touchbook - http://www.h-online.com/open/news/item/Touch-Book-Linux-based-touch-screen-device-announced-740317.html?view=zoom;zoom=1
* PostgreSQL
 

Latest revision as of 18:16, 20 April 2010

Project Name

Fedora/ARM

Name

Arlene Daniel

Our Goal

To enable the current version of Fedora to be used on low-power ARM computer systems. We will do this by setting up the Fedora build system, named Koji, on a cluster of ARM computers. These computers will build an ARM-compatible version of each of the approximately 15,000 Fedora software packages. The Koji-Shadow software will be used to monitor the main Fedora Koji system -- which builds software packages for PCs -- so that every time a new software package is prepared for PCs it will automatically also be built on our Koji system for ARM computers. The resulting packages will be placed in a publicly-accessible repository so that they can be used worldwide.

About The ARM Architecture

ARM chips are the most popular 32-bit RISC (Reduced Instruction Set Computer) microprocessors --the brain of the computer that carries out its most complex functions -- and is mainly used in low power applications -- not much power is needd to operate the device --. Approximately 1.6 billion are being made each year by different manufactures and are sold under a number of different brand names, brands like ARM, StrongARM, Armada, Cortex, OMAP, Sheeva, Snapdragon and XScale to name a few. The chips created are used in devices like cellphones, routers, NAS boxes, embedded controllers, tablets, netbooks and many others.

The ARM Hardware

The devices listed below are some of the devices that use a ARM processor:

   * One Laptop Per Child (OLPC) computers is a subnotebook -- a notebook that is smaller than the average notebook -- that's donated
     to developing countries, with the hope that, every primary school child could have their own laptop to explore the world. The OLPC is
     very inexpensive to produce, costing developers fifty dollars($50) for everyone produced.
   * The Touch Book - developed by Always Innovating -- is a lightweight portable device that has a detachable magnetic keyboard and
   * The SheevaPlug - a plug computer designed to run network-based software devices are some devices that use an ARM processor.

These devices also use a Linux based OS and since Fedora is used on the XO units, having a reliable ARM build of Fedora is increasingly important.

About Fedora

Fedora is an open source operating system using RPM-based -- linux distributions that use a tool to automate the process of installing, upgrading, configuring and removing software packages, built on top of the Linux kernel. Software packages -- are computer software that are stored in an archived format to install programs.-- Archived format -- a compressed format of storing files.-- There are presently over 15000 software packages availabe for download to linux community of users. It is developed and maintained by a community-support -- a group of programmers who contribute to the development of Fedora -- known as the Fedora Project which is sponsored by Red Hat. The Fedora Project's mission is "To lead the advancement of free and open source software and content as a collaborative community." Fedora’s main aim is to be a leader in the use and distribution on software designed as open source-- the source code the proogram is readily available. -- Fedora has a release every six months and a maintenance period of about thirteen months for each of its releases. Fedora 12 was released November 11, 2009 and Fedora 13 has an expected release date of May 18, 2010.

Supported Architecture

Fedora supports two primary architectures:

  i386 - 32-bit Intel/AMD-compatible
  x86_64 - 64-bit Intel/AMD-compatible

There are also a number of secondary archs:

 arm - A widely-used, low-power processor family commonly used for embedded and mobile applications
 ia64 - Itanium
 pa-risc - HP Precision Architecture
 ppc - 32-bit Power PC
 ppc64 - 64-bit Power PC
 s390 - IBM mainframes (including z90 and z9)
 sparc - Sun RISC architecture

Primary architectures are instrumental in the design and development of any fedora release. During the development of a fedora release if any of the primary architecture builds fail, the packages cannot be released. Fedora package maintainers are required to make sure that their package builds properly for this architecture. Users of any linux distrbutions use one of these primary architectures.

Secondary architectures are architectures where the hardware for the secondaries are maintained by the people producing the builds for those secondary architectures. Build failures on secondary architectures are not fatal, unlike builds in the primary architectures, the failure repairs on the secondary architectures are often released several months after the primary ones.

The ARM Architecture

The ARM architecture is increasingly important, but there's a lot more that could be done in terms of update frequency, number of packages successfully built, transparency of process, and integration with the other Fedora build processes.

Fedora policy requires that packages be natively compiled. We use a cluster of ARM hardware and QEMU virtual machines to build the packages natively for ARM.

The baseline ARM CPU architecture that we have chosen to support is ARMv5TE, Little Endian, Soft-Float, EABI. We believe that this provides a nice baseline and that the pre-built packages and root file system images. You should be able to use this on many of the modern ARM CPUs, including XScale, ARM926 and ARM-11, etc.

Although we do not provide such binaries, the sources also lend themselves for building for pre-ARMv5TE hardware. The same is true for big-endian CPUs.

We currently do not plan to provide an installer, ISO images, or a kernel. Unlike in the x86 world, different ARM CPU families require different kernel images. Likewise, it is not entirely clear whether it makes sense to provide an installer or ISO images.


Our Objective

The plan was to set up a Koji builder on a system of eight virtual ARM machines that is based on the CDOT system HongKong. The ARM builders are using QEMU emulation, -- a processor emulator -- which will be replaced by ARM Hardware when it arrives. The ARM Hardware that was purchased was an OpenRD platform. This platform is powered by the fastest ARM architecture available. The platform allows us to create and complete our designs.

The HongKong system also housed a PosegreSQL database and the Koji hub, Koji Web and Kojira. Other components like Apache and Mock also had to be installed on the system.

Acknowledgements

  • Chris Tyler

Links

Images