Open main menu

CDOT Wiki β

Add support for more compilers to distcc

Revision as of 19:10, 13 December 2007 by Pcallaghan (talk | contribs) (Added 0.3 release)

Project Name

Add support for more compilers to distcc

Project Description

An earlier project added multi-compiler support to distcc, and then MSVC support. The framework is now in place to support even more compilers. Add support for another compiler; either Java or C#.

Necessary Skills: C, Python, Makefile

Project Leader(s)

Project Contributor(s)

Some options for contributions:

  • publish/find some more c# files to test with
  • NOTE: Please ensure the files compile with Mono c# before sending them.
  • test this on different OSs & networks
  • Make a comprehensive list of the set-up steps on one page, unlike the scattered links there are now. No coding required, but time consuming.

Project Details/News

0.3 release (Under construction)

Code

All the code for the 0.3 release can be found here. This contains three files:

  • pcallaghanzeropointthreeWHOLE.txt - the entire patch
  • pcallaghanzeropointthreeCODE.txt - the changes to the src directory
  • pcallaghanzeropointthreeTEST.txt - the changes to test/distcctest.py

These should be applied from the trunk directory. NOTE: use either pcallaghanzeropointthreeWHOLE.txt OR both the others, depending on your preference. The end result is the same. For more details see the instructions section.

Instructions

In order to use this program, several steps must be completed.

  1. download the Mono C# compiler
  • put mono in your PATH
  • set up an environment variable DISTCC_MCS_HOSTS=localhost
  • Checkout the distcc source
  • If you are using windows get cygwin
    • configure cygwin by running the setup file that gets downloaded. I used the mirror: http://mirror.rhsmith.umd.edu
    • NOTE: This is the step where most errors will stem from, so pay careful attention to detail.
      • In addition to the default, get the following packages:
        • under development
      • modify the cygwin.bat file
      • Copy and paste the following file, then change the paths so they accurately represent your system. It's VERY important
      • The particular lines to change (if yours are different) are: 4,9,12,16,19,22
      • If you copy and paste you should not have to make any changes past line 22.
    @echo off
    
    C:
    chdir C:\cygwin\bin
    
    rem --- Set HOME so that cvs and ssh work correctly
    rem --- cvs uses HOME to locate your .cvspass file, and ssh to locate your .ssh file
    rem --- if you are using ssh, your HOME should match the home directory specified in /etc/passwd. See http://www.cygwin.com/faq/faq0.html.
    set HOME=C:\home
    
    rem --- Set VCVARS to wherever the MSVC vcvars.bat file is found
    set VCVARS=C:\ff\vcvars32.bat
    
    rem --- Set MSSDK to wherever the MS SDK is installed
    rem --- Only required for MSVC7 or the Free MSVC editions that don't come with an SDK
    set MSSDK=C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2
    
    rem --- Set MOZ_TOOLS to wherever you have the moztools packaged installed
    set MOZ_TOOLS=C:\moztools
    rem --- Set MONO to wherever you have mono installed.
    rem --- Make sure there are no spaces in the path, otherwise the unit tests won't work
    set MONO=C:\Mono-1.2.5.1
    
    rem --- Set CYGWINBASE to wherever cygwin is installed
    rem --- Do not use CYGWIN or else cygserver, cygrunsrv, and Cygwin services will not function properly
    rem --- Variable CYGWIN is also used to modify Cygwin's behaviour a little bit.
    set CYGWINBASE=C:\cygwin
    rem --- Make sure Cygwin does not print out a DOS style path warning
    set CYGWIN=nodosfilewarning
    
    rem --- Prepend Cygwin path
    rem --- This is necessary so that cygwin find is ahead of windows find.exe in the PATH, but cgywin link is after MSVC link.exe.
    set PATH=%CYGWINBASE%\bin;%PATH%
    set PATH=%MONO%\bin;%PATH%
    rem --- Set MSVC environment vars
    call "%VCVARS%"
     
    rem --- Prepend SDK paths
    rem --- Only required for MSVC7 or the Free MSVC editions that don't come with an SDK
    set PATH=%MSSDK%\bin;%PATH%;
    set INCLUDE=%MSSDK%\include;%MSSDK%\include\atl;%INCLUDE%
    set LIB=%MSSDK%\lib;%LIB%
    
    rem --- moztools comes last
    set PATH=%PATH%;%MOZ_TOOLS%\bin;C:\ff\distcc\src;
    
    
    rem --- Now the PATH variable contains:
    rem MS-SDK; MSVC; Cygwin; Windows; glib/libIDL; Moztools
    
    rem --- Typically the last thing the script does is launch a cygwin shell
    rem --- watch for your ~/.profile and /etc/profile which may overwrite your carefully setup PATH!
    bash --login -i
    
  • get the 0.3 patch
  • apply the patch
    • from the trunk directory
    • patch -p0 < pcallaghanzeropointthreeWHOLE.txt
    • OR
      • patch -p0 < pcallaghanzeropointthreeCODE.txt
      • patch -p0 < pcallaghanzeropointthreeTEST.txt
  • configure and build distcc
    • from the trunk directory
    • ./configure && make && make install
  • get a simple C# file
  • compile it
    • distcc mcs hello.cs
  • run it
    • ./hello
  • As an alternative to steps 8 through 10, find or write some other c# files and test them.
  • Tell me how it went.
  • Plans for 0.3 release

    While all the required functions exist, unit tests would streamline the testing process and ensure each function does exaclty what it's supposed to. Therefore, the major component of the 0.3 release is going to be a series of tests for the functions that can be run after any changes. In addition, (Time permitting) I'll clean up the instructions and make them clearer.

    Having found that these instructions are inadequate, I'm setting out to fix them. First up is configuring cygwin. The Instructions given on the linked page aren't enough to compile distcc with. There's a list of some 165 packages I had. Most of them were obviously extraneous and quite a few were downloaded by default. So I'm down to about 40 to test. The required ones will be listed with the 0.3 release instructions.

    0.25 Release (Obsolete)

    A new patch has been created. All required functions are now complete for the Mono C# compiler. Now is a great time for more contributors to test this project and try to find (or fix) any bugs that pop up. I'm still having trouble with SVN, but I expect to get that sorted out soon.

    to use this patch:

    1. get cygwin
    2. configure cygwin
    3. download the Mono C# compiler
    • put mono in your PATH
  • set up an environment variable DISTCC_MCS_HOSTS=localhost
  • Checkout the distcc source
  • get the 0.25 patch
  • apply the patch
    • from the trunk directory
    • patch -p0 < pcallaghanzeropointtwofive.txt
  • configure and build distcc
    • from the trunk directory
    • ./configure && make && make install
  • get a simple C# file
  • compile it
    • distcc mcs hello.cs
  • run it
    • ./hello
  • As an alternative to steps 9 through 11, find or write some other c# files and test them.
  • Tell me how it went.

  • 0.2 Release (Obsolete)

    A new patch has been created. All except one of the required functions (dcc_mcs_scan_args) are now complete. As the work required for this update turned out to be a lot less than I anticipated, a 0.25 patch should be released soon that will implement that last function. An SVN branch is not available yet, but should also be ready in time for 0.25.

    to use this patch:

    • Either reverse the first patch or get a non-patched version of the trunk
    • Apply the new patch


    Goal for 0.2 release

    Continue work on the previously identified functions. As of 0.1 several were still little more than shells, by the completion of 0.2 all except dcc_mcs_scan_args should be fully completed. A supplementary goal is to set up the source as a branch of the existing trunk on cdot, so testers may just check out the new code rather than having to check out the trunk and patch it.

    0.1 Release (Obsolete)

    A compiler has been created. Note that it is by no means anywhere near completion. That's why this is the 0.1 release.

    This patch has been tested on Windows XP, using cygwin to tell distcc to compile a single, simple c# file. To use it:

    1. get cygwin
    2. configure cygwin
    3. download the Mono C# compiler
    • put mono in your PATH
  • set up an environment variable DISTCC_MCS_HOSTS=localhost
  • Checkout the distcc source
  • get the 0.1 patch
  • apply the patch
    • from the trunk directory
    • patch -p0 < pcallaghanzeropointone.txt
  • configure and build distcc
    • from the trunk directory
    • ./configure && make && make install
  • get a simple C# file
  • compile it
    • distcc mcs hello.cs
  • run it
    • ./hello

    Goal for 0.1 Release

    - Add support for Mono's cSharp compiler[1]. By the 0.1 release, Distcc should be able to at least compile C# locally. While the earlier work done on Distcc With MSVC suggests that only a hand-full of functions need to be coded in order for it to add a new compiler, both Java and C# compile in a manner significantly different than C. This will likely cause some trouble shortly...regardless, the following functions are the minimum which need to be created:

    • arg.c
      • dcc_<compilerName>_set_action_opt
      • dcc_<compilerName>_set_output
      • dcc_<compilerName>_scan_args
    • strip.c
      • dcc_<compilerName>_strip_local_args
      • dcc_<compilerName>_strip_dasho
    • filename.c
      • dcc_<compilerName>_is_source
      • dcc_<compilerName>_is_preprocessed
      • dcc_<compilerName>_is_object
      • dcc_<compilerName>_preproc_exten
    • hosts.c
      • dcc_<compilerName>_get_hostlist

    Known Bugs

    See the list within Distcc With MSVC