1
edit
Changes
m
|-
</pre>|-|Didn't note the time.
</pre>
|-
ac_add_options --disable-libxulac_add_options --enable-debug --disable-optimizeac_add_options --enable-shared --disable-staticac_add_options --with-macos-sdkapplication=/Developer/SDKs/MacOSX10.4u.sdkmk_add_options MOZ_MAKE_FLAGS="-s -j4"</pre>|-browser
→November 2008
Resources: ted
=== Bugzilla ===
A bug has been submitted for enhancement on [[https://bugzilla.mozilla.org/show_bug.cgi?id=459891 Bugzilla]].
== Leader(s) ==
== Project Contributor(s) ==
* Ted Mielczarek
* [[User:twlai1 | Tony Lai]]
* [[User:Zghansar|Zaid Ghansar]]
* [[User:Evmarbella | Ezadkiel V. Marbella]]
* [[User:Scott | Scott Lunel]]
* [[User:kwpark1 | Park,KiWon]]
* [[User:Felis | Irina Shoykher]]
* [[User:JesseV| Jesse Valianes]]
== Project Details ==
* Get where time is spent during the build
** 1 set of data from scratch builds and 1 set of data from rebuilds
* Collect Data
** Get build time info from others
* -j specifies how many concurrent jobs you can run, so for some parts of the tree you get more makes going at once
** mk_add_options MOZ_MAKE_FLAGS=-j4
== Releases ==
=== Release 0.1 - Oct 13 ===
==== Goal ====
* Try out different build options
** See how they affect built
* Try profiling the run-time build
** Time how long it takes on each file/directory during the build
==== Results for Rebuilds with No Changes====
* Looking at the breakdown of the time, it can be seen that dom, browser, content, toolkit, netwerk, and xpcom take up a large portion of the build time
* The remaining 39.3% of the build time is divided into layout, intl, embedding and other 23 different directories
===== Charts =====
''Data from 10 Rebuilds were used to create the following charts:''
[[Image:TimeChartCondensed.jpg]]
[[Image:TimeChartAll.jpg]]
===== Data =====
{| class="mediawiki sortable" border="1" cellpadding="5" cellspacing="0"
|+ Rebuild Profile Data
|-
| style="background:#DDDDDD;" align="center" | Folders
| style="background:#DDDDDD;" align="center" | 1 (sec)
| style="background:#DDDDDD;" align="center" | 2 (sec)
| style="background:#DDDDDD;" align="center" | 3 (sec)
| style="background:#DDDDDD;" align="center" | 4 (sec)
| style="background:#DDDDDD;" align="center" | 5 (sec)
| style="background:#DDDDDD;" align="center" | 6 (sec)
| style="background:#DDDDDD;" align="center" | 7 (sec)
| style="background:#DDDDDD;" align="center" | 8 (sec)
| style="background:#DDDDDD;" align="center" | 9 (sec)
| style="background:#DDDDDD;" align="center" | 10 (sec)
| style="background:#DDDDDD;" align="center" | AVG (sec)
| style="background:#DDDDDD;" align="center" | %
|-
| dom
| 100.359
| 78.483
| 78.530
| 83.454
| 85.608
| 105.346
| 121.647
| 94.457
| 86.579
| 81.220
| 91.568
| 12.46%
|-
| content
| 107.504
| 72.521
| 76.064
| 72.693
| 72.175
| 84.936
| 101.252
| 75.230
| 76.487
| 70.748
| 80.961
| 11.02%
|-
| browser
| 79.352
| 78.662
| 85.827
| 73.574
| 75.665
| 79.911
| 82.024
| 71.726
| 72.534
| 71.356
| 77.063
| 10.49%
|-
| netwerk
| 73.646
| 58.749
| 68.171
| 61.736
| 61.747
| 77.301
| 89.820
| 66.909
| 65.089
| 59.837
| 68.301
| 9.30%
|-
| toolkit
| 67.833
| 63.092
| 70.699
| 64.975
| 66.265
| 72.723
| 76.762
| 67.938
| 67.456
| 63.604
| 68.135
| 9.27%
|-
| xpcom
| 63.494
| 57.616
| 54.563
| 50.040
| 51.008
| 71.673
| 77.560
| 61.735
| 63.407
| 48.072
| 59.917
| 8.16%
|-
| layout
| 67.544
| 41.636
| 43.673
| 43.222
| 44.494
| 49.144
| 76.850
| 45.768
| 43.555
| 42.665
| 49.855
| 6.79%
|-
| intl
| 50.631
| 40.152
| 42.585
| 40.364
| 40.779
| 52.670
| 59.998
| 44.475
| 40.644
| 38.891
| 45.119
| 6.14%
|-
| embedding
| 39.713
| 33.534
| 33.525
| 34.321
| 35.067
| 39.370
| 44.234
| 36.592
| 34.942
| 33.886
| 36.518
| 4.97%
|-
| editor
| 26.544
| 20.365
| 23.146
| 20.031
| 20.868
| 22.450
| 26.921
| 21.583
| 22.397
| 19.463
| 22.377
| 3.05%
|-
| gfx
| 22.279
| 14.212
| 14.587
| 14.673
| 16.167
| 17.449
| 26.159
| 15.635
| 14.128
| 14.231
| 16.952
| 2.31%
|-
| docshell
| 15.935
| 12.512
| 13.759
| 13.092
| 13.857
| 15.978
| 19.985
| 14.177
| 12.954
| 12.567
| 14.482
| 1.97%
|-
| rdf
| 15.287
| 12.746
| 12.636
| 13.285
| 13.497
| 15.735
| 19.098
| 14.593
| 13.312
| 12.964
| 14.315
| 1.95%
|-
| xpfe
| 11.707
| 10.972
| 11.337
| 11.267
| 11.531
| 12.907
| 13.344
| 11.754
| 11.429
| 10.915
| 11.716
| 1.59%
|-
| widget
| 14.399
| 9.812
| 10.078
| 10.383
| 10.472
| 12.203
| 14.493
| 10.947
| 12.513
| 10.354
| 11.565
| 1.57%
|-
| nsprpub
| 9.411
| 8.017
| 7.098
| 7.356
| 7.802
| 16.656
| 10.980
| 7.587
| 7.347
| 7.373
| 8.963
| 1.22%
|-
| uriloader
| 8.707
| 7.224
| 7.098
| 7.859
| 8.264
| 10.349
| 11.157
| 8.413
| 7.926
| 7.313
| 8.431
| 1.15%
|-
| storage
| 7.811
| 6.412
| 6.708
| 6.909
| 6.848
| 7.746
| 9.900
| 7.387
| 6.413
| 6.513
| 7.265
| 0.99%
|-
| caps
| 6.484
| 5.304
| 5.288
| 5.663
| 6.321
| 6.668
| 6.985
| 6.335
| 5.529
| 5.406
| 5.998
| 0.82%
|-
| profile
| 5.775
| 5.461
| 5.897
| 5.449
| 5.604
| 6.239
| 6.266
| 5.755
| 5.404
| 5.317
| 5.717
| 0.78%
|-
| xpinstall
| 4.772
| 4.072
| 4.570
| 4.243
| 4.264
| 5.161
| 5.592
| 4.349
| 4.183
| 4.068
| 4.527
| 0.62%
|-
| sun-java
| 3.528
| 3.307
| 3.214
| 3.353
| 3.557
| 4.090
| 5.230
| 3.630
| 3.271
| 3.332
| 3.651
| 0.50%
|-
| extensions
| 3.212
| 3.358
| 4.555
| 3.236
| 3.209
| 3.381
| 3.321
| 2.930
| 3.218
| 3.225
| 3.365
| 0.46%
|-
| chrome
| 3.428
| 3.121
| 3.026
| 3.268
| 3.209
| 3.412
| 3.733
| 3.284
| 3.113
| 3.047
| 3.264
| 0.44%
|-
| db
| 2.898
| 2.558
| 2.543
| 2.613
| 2.672
| 2.915
| 3.232
| 2.978
| 2.554
| 2.642
| 2.761
| 0.38%
|-
| view
| 2.768
| 2.497
| 2.434
| 2.609
| 2.570
| 2.926
| 3.120
| 2.626
| 2.832
| 2.538
| 2.692
| 0.37%
|-
| webshell
| 2.585
| 2.477
| 2.709
| 2.541
| 2.565
| 2.946
| 3.311
| 2.745
| 2.601
| 2.437
| 2.692
| 0.37%
|-
| build
| 2.252
| 2.066
| 2.162
| 2.139
| 2.191
| 2.201
| 3.217
| 2.232
| 2.486
| 2.054
| 2.300
| 0.31%
|-
| jpeg
| 3.053
| 1.529
| 2.110
| 1.651
| 1.617
| 2.396
| 2.641
| 1.647
| 1.526
| 1.573
| 1.974
| 0.27%
|-
| config
| 2.131
| 1.373
| 1.466
| 1.489
| 1.422
| 2.659
| 2.424
| 1.425
| 2.003
| 1.364
| 1.776
| 0.24%
|-
| probes
| 0.423
| 0.426
| 0.437
| 0.455
| 0.447
| 0.422
| 0.543
| 0.435
| 0.468
| 0.412
| 0.447
| 0.06%
|-
|}
* File with data used to generate charts and the charts can be found here: [[Media:RebuildAnalysis.zip | Rebuild Analysis xls]]
* A copy of all the logs I used or generated for the results can be found here: [[Media:BuildLogs1.zip | Build Logs Part 1]] and [[Media:BuildLogs2.zip | Build Logs Part 2]]
* A copy of the rules.mk I've edited from the config folder can be found here: [[Media:rules.zip | config/rules.mk]]
* An interesting and note worthy point in the analysis, is that most of these folders are visited multiple times during the build
=== Release 0.2 - Nov 10 ===
==== Goal ====
* Capture the processes being spawned during build along with its pid, ppid, time, duration, and working directory
* Try identifying the cause for slow Windows build
=== Release 0.3 - Dec 1 ===
* TBA
== Project Data ==
=== Contribution Instructions ===* Ideas on how to log the following during build time: pid, ppid, start time, end time, process name, current working directory* <strike>Please feel free to fill in the table to help me collect more data</strike>* <strike>YOU WILL BE CONSIDERED AS A PROJECT CONTRIBUTOR FOR PROVIDING DATA</strike>
* you can time your build by adding the "time" command in front of your build command like so:
** time make -f client.mk build
=== Sample Build Data ===
{| class="mediawiki sortable" border="1" cellpadding="5" cellspacing="0"
|+'''Build Time Data'''
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_MAKE_FLAGS=-j5
</pre>
|-
| 1:02:35
| 0:20:09
| 5
| 4 GB
| 2.0 Duo
| 7200
| Vista Business
| [[User:Pplam3|Patrick]]
| <pre>mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_MAKE_FLAGS=-j5
ac_add_options --disable-vista-sdk-requirements
ac_add_options --disable-tests
ac_add_options --disable-accessibility
ac_add_options --disable-mochitest
</pre>
|-
|-
| 1:03:00
|0:18:14
| N/A
| 1 GB
| 1.6 Duo
|-
| XP Media
| [[User:twlai1|Tony]]
|-
| 0:58:40
| -
| N/A
| 1 GB
| 1.6 Duo
| -
| Ubuntu (VirtualBox)
| [[User:twlai1|Tony]]
|-
| 0:55:00
|-
| 4
| 2.5 GB
| 1.6 AMD TurionX2
|-
| XP Professional
| [[User:zghansar|Zaid]]
| 2.5 GB
| 1.6 AMD TurionX2
|-
| XP Professional
| [[User:zghansar|Zaid]]
| 2.5 GB
| 1.6 AMD TurionX2
|-
| XP Professional
| [[User:zghansar|Zaid]]
ac_add_options --disable-optimize
ac_add_options --disable-static --enable-shared
</pre>
|-
| real 0:18:16<br />user 0:10:34<br />sys 0:03:0| real 0:02:59<br />user 0:04:15<br />sys 0:01:23| 5| 2 GB (Upgraded to more/faster rame)| 2.0 Duo ( OC'd to 2.41 )| 7200 (src) to External (dest) - speed bump doing this| Windows XP Professional ( 32-bit) - Visual Studio 2005| [[User:cbishop2 | Chris Bishop]]<!-- I hope this is ok since the mozconfig can also affect performance -->| <pre> . $topsrcdir/browser/config/mozconfigmk_add_options MOZ_OBJDIR=/i/mozilla/ff-optmk_add_options MOZ_MAKE_FLAGS=-j5ac_add_options --disable-vista-sdk-requirementsac_add_options --disable-testsac_add_options --disable-debugac_add_options --disable-accessibility</pre>|-|real 0:29:13<br />user 0:11:26<br />sys 0:03:47|real 0:05:8<br />user 0:04:47<br />sys 0:01:43|5|1 GB|2.0 Duo ( OC'd to 2.41 )|7200|Windows XP Professional ( 32-bit)|[[User:cbishop2 | Chris Bishop]]
<!-- I hope this is ok since the mozconfig can also affect performance -->
|<pre>. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-opt
mk_add_options MOZ_MAKE_FLAGS=-j5
ac_add_options --disable-tests
</pre>
|-
|real 0:36:50<br />user 0:10:44<br />sys 0:03:30
</pre>
|-
|
'''FireFox:'''<br />real 16m12.402s<br />user 2m23.381s<br />sys 5m11.815s <br />
ac_add_options --enable-debug
mk_add_options MOZ_MAKE_FLAGS=-j5
</pre>
|-
|[[User:Evmarbella | Ezadkiel V. Marbella]]
|<pre>. $topsrcdir/browser/config/mozconfig
export MOZCONFIG=~/mozilla/mozconfig-firefoxmk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-ff@CONFIG_GUESS@mk_add_options MOZ_MAKE_FLAGS=-j5mk_add_options MOZ_CO_PROJECT=browserac_add_options --enable-debug
ac_add_options --enable-tests
</pre>
|-
ac_add_options --enable-debug
ac_add_options --disable-tests
ac_add_options --disable-vista-sdk-requirements
</pre>
|-
|1Hr 15min
|_
| None
|1G
|Intel 1.8 GHZ Duo Core
|7200
|Microsoft Windows XP Service Pack 2
|[[User:kwpark1 | Park,KiWon]]
|<pre>
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-firefox
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --enable-application=browser
ac_add_options --disable-optimize
ac_add_options --enable-debug
ac_add_options --enable-tests
</pre>
|-
|1:05:32
|0:11:14
|8
|4G
|Quad-core 2.4
|7200
|Microsoft Windows XP Prof
|[[User:Felis| Irina Shoykher]]
|<pre>
ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --disable-vista-sdk-requirements
ac_add_options --enable-debug
ac_add_options --disable-optimize
ac_add_options --enable-libxul
ac_add_options --disable-tests
ac_add_options --disable-mochitest
ac_add_options --disable-accessibility
mk_add_options MOZ_MAKE_FLAGS=-j8
</pre>
|-
|real 01:25:35<br />user 00:14:08<br />sys 00:18:37
|real 00:19:02<br />user 00:05:44<br />sys 00:07:58
|4
|1 GB
|Pentium 4 2.6C
|7200
|XP Professional
|[[User:JesseV| Jesse Valianes]]
|<pre>
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../obj-@CONFIG_GUESS@
mk_add_options MOZ_MAKE_FLAGS=-j4
ac_add_options --enable-application=browser
ac_add_options --enable-libxul
ac_add_options --disable-tests
ac_add_options --disable-mochitest
ac_add_options --disable-vista-sdk-requirements
</pre>
== Project News / Updates ==
=== November 2008 ===<i>11/11/2008 -</i>* Managed to get process duration<i>11/06/2008 -</i>* Captured processes from a "do-nothing rebuild"* Posted description of CSV log on [[https://bugzilla.mozilla.org/show_bug.cgi?id=459891 Bugzilla]]* Trying to get more accurate data for Benjamin<i>11/05/2008 -</i>* Found a similar app to [[http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx Process Explorer]] called [[http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx Process Monitor]]** Captures: pid, ppid, start time, end time, process name** Can build log file* Posted finding on [[https://bugzilla.mozilla.org/show_bug.cgi?id=459891 Bugzilla]]* Asking for further directions/hints for project<i>11/04/2008 -</i>* Played with [[http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx Process Explorer]]** Captured: pid, ppid, start time, end time, process name** Finding ways to log it === October 2008 ===<i>10/20/2008 -</i>* Redo Pie Chart** Used all 10 Rebuild Data* Changed the app to generate better extractions from logs<i>10/17/2008 -</i>* Finished making app to extract times from log** App breaks down log into 3 files*** File with all lines starting with 'make' (directory path) and 'real' (time)*** File with just 1 'make' and 1 'real' line for each time - there are times for multiple 'make'*** File with just level 1 directory times* Adding times up for amount spent in each level 1 directory* Created diagram to illustrate time breakdown<i>10/15/2008 -</i>* Creating app to extract time from log* Timing the build multiple times for both scratch and rebuilds (aiming for 10 builds each)<i>10/13/2008 -</i>* Timed time spent in each directory* Working on extracting the data from log<i>10/02/2008 -</i>* Talked with Ted, guided me to look into rules.mk* Continuing testing for timing the build === September 2008 ===<i>09/30/2008 -</i>* Looked through many files and still haven't found where code for timing can go* Suspect most options don't affect build speed, except -j and optimize<i>09/25/2008 -</i>* After couple more testing with the build options, starting to notice they don't make a big difference<i>09/18/2008 -</i>
* Use time command to time build time
** ie. time make -f client.mk build
* Start collecting data
<i>09/17/2008 -</i>
* Built the first time in 1h 20m
* Rebuilt second time in 35m with all apps off
** Understand what each part of the build is taking
** Collect win32 build times from colleagues
<i>09/16/2008 -</i>
* Talked with Ted Mielczarek, briefly discussed about where to start and how to approach this project.
** Start by timing how long each make file is
** Identify which make file takes a long time
<i>09/15/2008 -</i>
* Created Project page for this project.
----