Difference between revisions of "Buildbot and Cairo"

From CDOT Wiki
Jump to: navigation, search
(Added release notes)
m (added prerequisites of project)
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
== Name ==
 
== Name ==
  
Cairo on Buildbot
+
Cairo on Buildbot is a project to build the Cairo graphics library in an automated manner with Buildbot and be able to run tests and performance tests
 
 
== Description ==
 
 
 
Automate Cairo builds with testing suites on Buildbot.
 
Automate Cairo builds with buildbot, and add in support for running test suites with buildbot.
 
  
 
== Leader(s) ==
 
== Leader(s) ==
Line 15: Line 10:
  
 
* [http://blog.vlad1.com/ Vladimir Vukicevic]
 
* [http://blog.vlad1.com/ Vladimir Vukicevic]
 +
* Ben Hearshum
  
NOTE: only Project Leader(s) should add names here. You '''can’t''' add your own name to the Contributor list.
+
== Project News ==
 +
* Mar 06 2008 - Started writing cairobuild.py grouping steps and CairoTests class which generates summary
 +
* Feb 29 2008 - '''Why does the TESTS take so long on MacOS X? It is ridiculous!!'''
 +
* Feb 29 2008 - Added steps to build on MacOS X
 +
* Feb 16 2008 - Updated Cairo project page and first proper release
 +
* Feb 11 2008 - Steps are written but some issues have to fixed
 +
* Jan 19 2008 - Project Proposal from Vladimir Vukicevic.
  
== Release Feb. 15th ==
+
== Release ==
'''Where will I put code and screenshots?'''
+
<blockquote>
* [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/ http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/] - Buildbot configs (look for "cairo")
+
=== Requirements ===
* [http://matrix.senecac.on.ca/%7Eazambran/mozilla/cairo/ http://matrix.senecac.on.ca/%7Eazambran/mozilla/cairo/] - Archived files
+
* [http://buildbot.net/trac buildbot (I am using 0.7.5)]
* [http://matrix.senecac.on.ca/%7Eazambran/mozilla/screenshots/ http://matrix.senecac.on.ca/%7Eazambran/mozilla/screenshots/] - Screenshots (look for "cairo" in the filenames)
+
* [http://twistedmatrix.com/trac/ twisted (I am using 2.5.0)]
 +
* [http://git.or.cz/ Git (I am using 1.5.3.6)]
 +
=== Mar. 7th ===
 +
* The steps to build on Mac have been added to [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/ Seneca Hera Try Server Setup/buildbot-configs/]
 +
and the blog post explaining a lot of details about how much has been done is here: ([http://armenzg.blogspot.com/2008/02/armenzg-sess14-building-cairo-on-mac-no.html Sess14-Buildbot on Cairo])  
  
'''What has been released this week:'''
+
# svn co http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/cairomaster/
* [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/cairo-one-ubuntu-slave/master.cfg master.cfg] - It builds Cairo and runs automated tests on one linux machine
+
# buildbot create-slave ./cairoslave localhost:9876 macslave slavepassword (change macslave for linuxslave when trying to run in Linux)
* [http://matrix.senecac.on.ca/%7Eazambran/mozilla/screenshots/buildbot-status-cairo.jpg buildbot-status-cairo.jpg] - After build #8 (no tests at this point)
+
# buildbot start cairomaster
* [http://matrix.senecac.on.ca/%7Eazambran/mozilla/cairo/Buildbot:Cairo-results.tar.gz Buildbot:Cairo-results.tar.gz] - After build #11 (tests were run)
+
# buildbot start cairoslave
 
+
# firefox http://localhost:8020 //OR open it with any browser
'''Steps to meet requirements'''
+
</blockquote>
* sudo apt-get install buildbot
 
* sudo apt-get install git-core //This is to get the version control that cairo uses, it is called GIT
 
 
 
'''Steps to make it happen'''
 
* buildbot create-master ~/cairomaster
 
* Put the [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/cairo-one-ubuntu-slave/master.cfg master.cfg] to the the created master
 
* buildbot create-slave ~/cairoslave localhost:9876 cairoslave
 
* buildbot start ~/cairomaster
 
* buildbot start ~/cairoslave
 
* firefox http://localhost:8020 //This is to be able to see the results of the build
 
  
 
== Tasks ==
 
== Tasks ==
Line 83: Line 79:
 
<!-- Task -->        <td>Write the steps for Mac</td>
 
<!-- Task -->        <td>Write the steps for Mac</td>
 
<!-- Details -->    <td></td>
 
<!-- Details -->    <td></td>
 +
<!-- Priority -->    <td style="background-color: orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
 +
<!-- Contributors--> <td style="text-align: center;">cworth #cairo</td>
 +
<!-- Status -->      <td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Done</td>
 +
<!-- Target -->      <td style="text-align: center;">29 Feb</td>
 +
<!-- Link -->        <td>[http://armenzg.blogspot.com/2008/02/armenzg-sess14-building-cairo-on-mac-no.html Sess14-Buildbot on Cairo]</td>
 +
    </tr>
 +
    <tr style="height:42px;">
 +
<!-- Task -->        <td>Write the steps for Windows</td>
 +
<!-- Details -->    <td></td>
 +
<!-- Priority -->    <td style="background-color: orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
 +
<!-- Contributors--> <td style="text-align: center;"></td>
 +
<!-- Status -->      <td style="background-color: white; text-align: center; font-weight: bold; font-size: bigger">&nbsp;</td>
 +
<!-- Target -->      <td style="text-align: center;">7 Mar</td>
 +
<!-- Link -->        <td>&nbsp;</td>
 +
    </tr>
 +
    <tr style="height:42px;">
 +
<!-- Task -->        <td>Write cairobuild.py file</td>
 +
<!-- Details -->    <td>This will centralize and group steps together as in [http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/testing/unittest/mozbuild.py mozbuild.py]</td>
 
<!-- Priority -->    <td style="background-color: orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
 
<!-- Priority -->    <td style="background-color: orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
 
<!-- Contributors--> <td style="text-align: center;"></td>
 
<!-- Contributors--> <td style="text-align: center;"></td>
 
<!-- Status -->      <td style="background-color: white; text-align: center; font-weight: bold; font-size: bigger">&nbsp;</td>
 
<!-- Status -->      <td style="background-color: white; text-align: center; font-weight: bold; font-size: bigger">&nbsp;</td>
<!-- Target -->      <td style="text-align: center;">&nbsp;</td>
+
<!-- Target -->      <td style="text-align: center;">7 Mar</td>
 
<!-- Link -->        <td>&nbsp;</td>
 
<!-- Link -->        <td>&nbsp;</td>
 
     </tr>
 
     </tr>
Line 100: Line 114:
 
     <tr style="height:42px;">
 
     <tr style="height:42px;">
 
<!-- Task -->        <td>Find out how to use PKG_CONFIG_PATH</td>
 
<!-- Task -->        <td>Find out how to use PKG_CONFIG_PATH</td>
<!-- Details -->    <td></td>
+
<!-- Details -->    <td>* PKG_CONFIG_PATH=(the --prefix value)+"/lib/pkgconfig"(where the .pc file is)</td>
 
<!-- Priority -->    <td style="background-color: orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
 
<!-- Priority -->    <td style="background-color: orange; text-align: center; font-weight: bold; font-size: bigger">Med</td>
 
<!-- Contributors--> <td style="text-align: center;"></td>
 
<!-- Contributors--> <td style="text-align: center;"></td>
<!-- Status -->      <td style="background-color: white; text-align: center; font-weight: bold; font-size: bigger">&nbsp;</td>
+
<!-- Status -->      <td style="background-color: green; text-align: center; font-weight: bold; font-size: bigger">Fixed</td>
<!-- Target -->      <td  style="text-align: center;">&nbsp;</td>
+
<!-- Target -->      <td  style="text-align: center;">Feb 29</td>
 
<!-- Link -->        <td>&nbsp;</td>
 
<!-- Link -->        <td>&nbsp;</td>
 
     </tr>
 
     </tr>
Line 155: Line 169:
  
 
== Problems ==
 
== Problems ==
 +
* A lot of problems when trying to build on MacOS are mentioned here: [http://armenzg.blogspot.com/2008/02/armenzg-sess14-building-cairo-on-mac-no.html Sess14-Buildbot on Cairo]
 +
* <strike>The first time the command GIT CLONE works because there is nothing before that execution but the second run should be an update instead of a checkout. What could be the proper GIT command to have as the step?</strike>
 +
I have decided to remove the checked out code and checkout the trunk before each build
 +
* <strike>I need to do SUDO MAKE INSTALL, but this requires user's input which breaks the automated concept. What could be the right step?</strike>
 +
Used ./configure --prefix=/path/to/somewhere to avoid building somewhere where I require to have sudo privileges
  
* The first time the command GIT CLONE works because there is nothing before that execution but the second run should be an update instead of a checkout. What could be the proper GIT command to have as the step?
+
== Notes ==
* I need to do SUDO MAKE INSTALL, but this requires user's input which breaks the automated concept. What could be the right step?
+
'''These notes are for learning purposes and documentation'''
 +
<blockquote>
 +
=== Feb. 15th release===
 +
'''Where will I put code and screenshots?'''
 +
* [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/ http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/] - Buildbot configs (look for "cairo")
 +
* [http://matrix.senecac.on.ca/%7Eazambran/mozilla/cairo/ http://matrix.senecac.on.ca/%7Eazambran/mozilla/cairo/] - Archived files
 +
* [http://matrix.senecac.on.ca/%7Eazambran/mozilla/screenshots/ http://matrix.senecac.on.ca/%7Eazambran/mozilla/screenshots/] - Screenshots (look for "cairo" in the filenames)
  
== Project News ==
+
'''What has been released this week:'''
Feb 11 2008 - Steps are written but some issues have to fixed
+
* [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/cairo-one-ubuntu-slave/master.cfg master.cfg] - It builds Cairo and runs automated tests on one linux machine
Jan 19 2008 - Project Proposal from Vladimir Vukicevic.
+
* [http://matrix.senecac.on.ca/%7Eazambran/mozilla/screenshots/buildbot-status-cairo.jpg buildbot-status-cairo.jpg] - After build #8 (no tests at this point)
 +
* [http://matrix.senecac.on.ca/%7Eazambran/mozilla/cairo/Buildbot:Cairo-results.tar.gz Buildbot:Cairo-results.tar.gz] - After build #11 (tests were run)
  
== Notes ==
+
'''Steps to meet requirements'''
=== master.cfg draft ===
+
* sudo apt-get install buildbot
 
+
* sudo apt-get install git-core //This is to get the version control that cairo uses, it is called GIT
# -*- python -*-
 
# ex: set syntax=python:
 
 
# This master.cfg defines how to build Cairo and run the tests it has
 
 
from buildbot.scheduler import Scheduler, Periodic
 
from buildbot.process import step, factory
 
from buildbot.status import html
 
s = factory.s
 
c = BuildmasterConfig = {}
 
 
####### PROJECT IDENTITY
 
 
c['projectName'] = "Cairo"
 
c['projectURL'] = "http://www.cairographics.org/"
 
# changed buildbot's default port
 
c['buildbotURL'] = "http://localhost:8020/"
 
#modified from default's buildbot port
 
c['slavePortnum'] = 9876
 
 
####### BUILDSLAVES
 
 
c['bots'] = [("cairoslave", "cairoslavepassword")]
 
 
####### STATUS TARGETS
 
 
c['status'] = []
 
 
from buildbot.status import html
 
c['status'].append(html.Waterfall(http_port=8010))
 
 
####### CHANGESOURCES
 
 
c['sources'] = []
 
 
# Armen - Research GIT source changes
 
 
####### SCHEDULERS
 
 
## configure the Schedulers
 
from buildbot.scheduler import Scheduler
 
c['schedulers'] = []
 
c['schedulers'].append(Periodic(name="30 minutes build scheduler",
 
                                builderNames=["Linux Ubuntu7.1 dep unit test"],
 
                                periodicBuildTimer=30*60))
 
 
####### BUILDERS
 
 
builders = []
 
 
ubuntuFactory = factory.BuildFactory()
 
ubuntuFactory.addStep(step.ShellCommand, name="checkout pixman",
 
        command=["git","clone","git://git.cairographics.org/git/pixman"],
 
        workdir="")
 
ubuntuFactory.addStep(step.ShellCommand, name="checkout cairo",
 
        command=["git","clone","git://git.cairographics.org/git/cairo"],
 
        workdir="")
 
ubuntuFactory.addStep(step.ShellCommand, name="autogen for pixman",
 
        command=["./autogen.sh"],
 
        workdir="pixman")
 
ubuntuFactory.addStep(step.ShellCommand, name="configure pixman",
 
        command=["./configure"],
 
        workdir="pixman")
 
ubuntuFactory.addStep(step.ShellCommand, name="make pixman",
 
        command=["make"],
 
        workdir="pixman")
 
ubuntuFactory.addStep(step.ShellCommand, name="make install pixman",
 
        command=["sudo","make","install"],
 
        workdir="pixman")
 
#let's prepare cairo
 
ubuntuFactory.addStep(step.ShellCommand, name="autogen for cairo",
 
        command=["./autogen.sh"],
 
        workdir="cairo")
 
ubuntuFactory.addStep(step.ShellCommand, name="configure cairo",
 
        command=["./configure"],
 
        workdir="cairo")
 
ubuntuFactory.addStep(step.ShellCommand, name="make cairo",
 
        command=["make"],
 
        workdir="cairo")
 
ubuntuFactory.addStep(step.ShellCommand, name="make install cairo",
 
        command=["sudo","make","install"],
 
        workdir="cairo")
 
 
cairo_trunk_ubuntu_builder = {
 
      'name': "Linux Ubuntu7.1 dep unit test",
 
      'slavenames': ['cairoslave'],
 
      'builddir': "trunk_ubuntu",
 
      'factory': ubuntuFactory}
 
 
builders.append(cairo_trunk_ubuntu_builder)
 
c['builders'] = builders
 
# c['builders'] = [cairo_trunk_ubuntu_builder]
 
  
 +
'''Steps to make it happen'''
 +
* buildbot create-master ~/cairomaster
 +
* Put the [http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/cairo-one-ubuntu-slave/master.cfg master.cfg] to the the created master
 +
* buildbot create-slave ~/cairoslave localhost:9876 cairoslave
 +
* buildbot start ~/cairomaster
 +
* buildbot start ~/cairoslave
 +
* firefox http://localhost:8020 //This is to be able to see the results of the build
 +
</blockquote>
  
 +
<blockquote>
 
=== Build Cairo manually ===
 
=== Build Cairo manually ===
 
* mkdir cairo & cd cairo
 
* mkdir cairo & cd cairo
Line 295: Line 238:
 
* cd test && make html
 
* cd test && make html
 
* firefox index.html
 
* firefox index.html
 +
</blockquote>

Latest revision as of 21:55, 7 March 2008

Name

Cairo on Buildbot is a project to build the Cairo graphics library in an automated manner with Buildbot and be able to run tests and performance tests

Leader(s)

Adam Delyea
Armen Zambrano

Contributor(s)

Project News

  • Mar 06 2008 - Started writing cairobuild.py grouping steps and CairoTests class which generates summary
  • Feb 29 2008 - Why does the TESTS take so long on MacOS X? It is ridiculous!!
  • Feb 29 2008 - Added steps to build on MacOS X
  • Feb 16 2008 - Updated Cairo project page and first proper release
  • Feb 11 2008 - Steps are written but some issues have to fixed
  • Jan 19 2008 - Project Proposal from Vladimir Vukicevic.

Release

Requirements

Mar. 7th

and the blog post explaining a lot of details about how much has been done is here: (Sess14-Buildbot on Cairo)

  1. svn co http://svn2.assembla.com/svn/Seneca_Hera_Try_Server_Setup/buildbot-configs/cairomaster/
  2. buildbot create-slave ./cairoslave localhost:9876 macslave slavepassword (change macslave for linuxslave when trying to run in Linux)
  3. buildbot start cairomaster
  4. buildbot start cairoslave
  5. firefox http://localhost:8020 //OR open it with any browser

Tasks

Task
Details
Priority
Contrib.
Status
Target
Link(s)
Build Cairo manually Manually build Cairo Med armenzg Done   Blog: Building Cairo
Buildbot and Cairo on Linux Build on my laptop Cairo and run tests; Missing perf tests High armenzg Done 15 Feb Blog: Buildbot with Cairo
Fix GIT and SUDO issue * Fixed by delete checked out source code and ./configure --prefix High armenzg Fixed 15 Feb
Write the steps for Mac Med cworth #cairo Done 29 Feb Sess14-Buildbot on Cairo
Write the steps for Windows Med   7 Mar  
Write cairobuild.py file This will centralize and group steps together as in mozbuild.py Med   7 Mar  
Run the "performance" steps Med      
Find out how to use PKG_CONFIG_PATH * PKG_CONFIG_PATH=(the --prefix value)+"/lib/pkgconfig"(where the .pc file is) Med Fixed Feb 29  
Build upon source changes in the GIT repository Med      
Try to use the GIT source class that buildbot has Med      
Package results and make them available Pass the results of "make test" and "make html" to the master and allow to download them Med      
Write a CSS file for the waterfall view   Med      
Email notifications? IRC notifications?   Med      

Problems

  • A lot of problems when trying to build on MacOS are mentioned here: Sess14-Buildbot on Cairo
  • The first time the command GIT CLONE works because there is nothing before that execution but the second run should be an update instead of a checkout. What could be the proper GIT command to have as the step?
I have decided to remove the checked out code and checkout the trunk before each build
  • I need to do SUDO MAKE INSTALL, but this requires user's input which breaks the automated concept. What could be the right step?
Used ./configure --prefix=/path/to/somewhere to avoid building somewhere where I require to have sudo privileges

Notes

These notes are for learning purposes and documentation

Feb. 15th release

Where will I put code and screenshots?

What has been released this week:

Steps to meet requirements

  • sudo apt-get install buildbot
  • sudo apt-get install git-core //This is to get the version control that cairo uses, it is called GIT

Steps to make it happen

  • buildbot create-master ~/cairomaster
  • Put the master.cfg to the the created master
  • buildbot create-slave ~/cairoslave localhost:9876 cairoslave
  • buildbot start ~/cairomaster
  • buildbot start ~/cairoslave
  • firefox http://localhost:8020 //This is to be able to see the results of the build

Build Cairo manually

make[2]: Entering directory `/home/armen/sandbox/cairo2/pixman/pixman' test -z "/usr/local/lib" || /bin/mkdir -p "/usr/local/lib" /bin/bash ../libtool --mode=install /usr/bin/install -c 'libpixman-1.la' '/usr/local/lib/libpixman-1.la' /usr/bin/install -c .libs/libpixman-1.so.0.9.6 /usr/local/lib/libpixman-1.so.0.9.6 /usr/bin/install: cannot remove `/usr/local/lib/libpixman-1.so.0.9.6': Permission denied make[2]: *** [install-libLTLIBRARIES] Error 1

  • sudo make install //I had to use SUDO
  • whereis libpixman-1
  • cd ../cairo && ./autogen.sh
  • ./configure
  • make
  • make install //the same problem with SUDO

make[2]: Entering directory `/home/armen/sandbox/cairo2/cairo/src' test -z "/usr/local/lib" || /bin/mkdir -p "/usr/local/lib" /bin/bash ../libtool --mode=install /usr/bin/install -c 'libcairo.la' '/usr/local/lib/libcairo.la' /usr/bin/install -c .libs/libcairo.so.2.14.0 /usr/local/lib/libcairo.so.2.14.0 /usr/bin/install: cannot create regular file `/usr/local/lib/libcairo.so.2.14.0': Permission denied

  • sudo make install

Running the tests and getting an index.html file with them

  • make test //takes long time
  • cd test && make html
  • firefox index.html