Open main menu

CDOT Wiki β

Distcc With MSVC

Revision as of 15:01, 20 March 2007 by Cdolivei (talk | contribs)

Project Name

distcc, distributed compilations for C/C++ compilers.

Project Description

The goal of this project is to speed up Windows Mozilla Builds by spreading object compilations over several machines. This can be done by using an open source product called distcc created by Martin Pool. Unfortunately this is only available on *nix machines using gcc. Our project involves porting this software to Windows and use Microsoft's Visual C++ compiler using cygwin.

Project Leader(s)

Project Contributor(s)

We have stuff for you to do!

  • Dave Liebreich - Thanks for checkin whether cygwin's ln will create working links to cl.exe (e.g. a link name clfoo.exe to cl.exe).
  • Ben Hearsum - for the windows box and support! Thank you!
  • Jason Spiro - For helping figure out why cl wouldn't execute in the unit testing framework and assisting in finding the right path to solve this issue.
  • Phil - helped with changing the page on helping people do research into cl options for us.
  • Moe
  • Mark P. (aka RealMarkP)
  • David Humphrey
  • Jeff
  • Melz

Product Documentation

slave distccd --allow '' --daemon --log-file="logfile" --verbose mater export DISTCC_LOG="filename" export DISTCC_VERBOSE=1 export DISTCC_CL_HOSTS='' distcc cl /c foo.c

mk_add_options MOZ_OBJDIR=/cygdrive/c/home/cesar/mozobj ac_add_options --enable-application=browser ac_add_options --enable-optimize ac_add_options --disable-debug ac_add_options --enable-debugger-info-modules=no mk_add_options MOZ_CO_PROJECT=browser mk_add_options MOZ_MAKE_FLAGS="CC='distcc cl' CXX='distcc cl' -j4"

Current Status

  • distcc seems to be distributing, but something is not working on the server side which is causing distcc to fallback to localhost. This problem has not been solved as of Dec 7, 2006.
  • Some of the Test Cases may be incorrect. Neither Tom nor Cesar have extensive experience in CL options. So it may not distribute when it should, or it may distribute when it shouldn't.

What's still to do

Need to port unit tests to ensure same functionality for cl as for gcc. medium
Get distcc working with Moz
check why MSVC Preprocessor seems to be dying with firefox 2.0 release source.

Possible Solution: We're doing unix to windows path conversion on the daemon, move that code to the client and everything might be good.

high (currently Cesar & Tom)
Figure out why distributed builds die but nondistributed builds don't. This issue lies in the Moz Build System not in distcc code. High
Get a list of programs that work with our distcc
Application Reporter
NLED Cesar
mozilla Tom + Cesar
boy there sure are lot of memory leaks... would be nice to fix that low
Make sure that there are no orphan distcc processes left after make maintainer-check low

How to get distcc to compile use cl

Tom and Cesar created this patch keeping backwards compatibility in mind. You should be able to still use distcc to compile your c programs if you wish. But if you applied the patch, you probably want to do more than that.

Firstly, the environment variables for distributing with cl is different. DISTCC_HOSTS is still valid for gcc, but cl will be looking for DISCC_CL_HOSTS instead. In fact, that is the only thing it looks for. Unlike gcc, cl will not look for ~/.distcc directory. This might be implemented later on.

Secondly, masquerade mode works only for gcc. If you want want to use cl, you must explicitly say so (distcc cl hello.c). If you are using gnu make, you must also explictly tell it to use cl as well (eg using CC, CXX, etc).


NOTE :: If you run the checks, please check your processes (ps aux, or the task manager) for leaked distcc/distccd programs. We will fix this ASAP

You may ignore this section if you installed Microsoft's CL in the default C:\ directory and your tests work fine.

In order to properly run the tests, you may need to make a few changes. The PATH variable is reinitalized when the test cases are run. Normally, the tests will fail when they try the cl tests. This is because PATH does not look for the cl executable and the libraries. Tom and Cesar have made certain assemptions when extending the include path, and included these following paths :

/cygdrive/c/Program Files/Microsoft Visual Studio 8/VC/bin
/cygdrive/c/Program Files/Microsoft Visual Studio 8/Common7/IDE

If it is different for your machine, you need to find these in the or Makefile and change them to your settings.

Extending the program

This will be postponed until we get distcc working properly (not fully) with cl.

Project Details

Mozilla (and others :) ) is interested in speeding up their builds by using a distributed compiler (distcc). The problem is that distcc only works with gcc but not with the msvc compiler (cl).

The plan has two stages:

  1. change distcc to support multiple compilers
  2. add cl support

Initially, we planned to use two vtables (one for distcc and another for distccd) to keep track of which compiler specific operation to use when. However that bombed out so we're currently using a single vtable.

We have kept the two vtable branch because we actually got a lot of the cl specific stuff done (before we realized we missed some functions).

Help Us Out

Here's what you can do to help us out...:

The First Patch (Oops)

This was released to the distcc mailing list. This has a few functions missing. It was meant to provide the two vtable support without cl. It should be ignored and we'll be re-releasing a working version of this soon.

Project News

See what's up with this project! Older news can be found here

March 19, 2007

There was a subtle bug in Mozilla's code base that bug 274563 deals with that made it impossible for distcc to build Moz. If you apply the patch attached to the bug then you'll be able to do distcc builds of Mozilla. We will soon be releasing our first beta to reflect our confidence in the code.

Feb 27, 2007

We are able to do distributed builds of Mozilla but it dies when trying to link stuff together (issues surrounding ATL?). So in order to take advantage of Ben Smedberg's wonderful Mozilla Build and be able to get help figuring out weird linkage bugs and dependancies without having to deal with cygwin issues (which are galore), we've started a branch with a mingw port: svn://

Jan 4, 2007

The post made to the build newgroup seems to have gathered dust. If mozilla is limited to only 4 jobs, than distcc is somewhat limited. But that's not our problem.

Tom has found one problem with using cygwin. Using absolute pathnames. For example :

$ cl hello.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

Microsoft (R) Incremental Linker Version 8.00.50727.42
Copyright (C) Microsoft Corporation.  All rights reserved.


Works. But this doesn't :

$ cl /cygdrive/c/home/temp/hello.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '/cygdrive/c/home/temp
cl : Command line error D8003 : missing source filename

There are a few things we can do about this :

  • replace /cygdrive/c/ with C:\
  • user must replace the enviroment variable TMPDIR which defaults to /tmp

Cesar found another problem. Both C/C++ preprocessed file ends in .i. cl has no way of knowning if it's a C/C++ preprocessed file. So it refuses to compile it (it treats it as an object file). So when we preprocess, we have to redirect to the same extension as the source file.

$ cl hello.i
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9024 : unrecognized source file type 'hello.i', objec
t file assumed
Microsoft (R) Incremental Linker Version 8.00.50727.42
Copyright (C) Microsoft Corporation.  All rights reserved.

hello.i : fatal error LNK1107: invalid or corrupt file: cannot read at 0xEE7F