# Introduction

This documentation is still in progress.

• This document assumes that you have the following:
• msvcp71.dll (NOTE: If you have Visual Studio 2003 installed, you've already got it.)
• Microsoft® C Runtime Library, v. 7.10.3077.0
• You need a copy of this file to run Python with all the libraries.
• This documentation is the revised version of Annie Sullivan's Readme.txt for the Firefox Performance Testing.
• Unfortunately, the Firefox Performance Testing is only catered to Windows users at this time.

# Quick start

3 easy steps to run Firefox Performance Testing:

# Cygwin Setup

• Cygwin
• Install cygwin with the DOS-style line endings. (Use the default root directory, which is C:\)
• At minimum, you are required to have the following additional tools:
• unzip (Archive category)
• wget (Web category)

unzip (Archive category)

wget (Web category)

Note: For some reason, we can't get output from Firefox unless it's wrapped in a Cygwin sh shell. So we use cygwin for that, and to sync between Firefox runs.

# Automated Configuration

## Save the script

• Save this script at a desired location
• Edit the script to your own choosing. You may provide a list of extensions to install and test in the config.yaml section.
• Note: This script will run the performance testing. Running performance testing will close all opened Firefox browsers! If don't wish for that to happen, simply comment out the last line of the script (to comment out, use #):
#./run_tests.py config.yaml;


## Run the script

For example:

$./perfconfig.sh ## What the script does • Creates directories needed for the performance testing • Directory that holds the files for the performance testing • Reports directory to hold the results generated • Downloads and installs pre-requisites • Write to config.yaml ## Config File (config.yaml) ### Config File Info When you run the perfconfig.sh script, a config.yaml file will be written to C:\mozilla\testing\performance\win32\ by default. Your config.yaml should consist of these items. • filename • title • (Test Profile name) - any name • firefox • preferences • extensions Refer to sample.config:  # Filename will be appended to the timestamp in the report filename. # Use letters and underscores only filename: yourfilename # The title of the report title: 1 GHz Celeron # Name of profile to test Test profile 1: # Path to Firefox to test firefox: C:\Program Files\Mozilla Firefox 2 Beta 1\firefox.exe # Preferences to set in the test (use "preferences: {}" for no prefs) preferences: javascript.options.showInConsole : true xpinstall.enabled : true # Extensions to install in test (use "extensions: {}" for none) extensions: # Make sure that the path is directed to the desired location of the extension # Need quotes around guid because of curly braces "{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi foo@sample.com : c:\path\to\other\unzipped\xpi  ### Config File Instructions 1. Verify that the firefox path is pointing to the right Firefox instance on your system. • You will hit a zero division error and some other errors if you don't point it to the right path 2. preferences: • If you don't have preferences, change to: preferences: {} (make sure there is a space between ":" and "{" or else it will cause a syntax error) • If you do have preferences, refer to sample.config 3. extensions: • If you don't have extensions to install, change to: extensions: {} (make sure there is a space between ":" and "{" or else it will cause a syntax error) • If you do have extensions to install, refer to this sample:  extensions: # Make sure that the path is directed to the desired location of the extension # Need quotes around guid because of curly braces "{12345678-1234-1234-1234-abcd12345678}" : c:\path\to\unzipped\xpi foo@sample.com : c:\path\to\other\unzipped\xpi  • Make sure that the path to the extension is right 4. Change filename and title to whatever you desire # Run Firefox Performance Tests • By default, the run_tests.py file that you need to execute to run the Firefox Performance Testing is located at c:\mozilla\testing\performance\win32\ • You can use a space-separated list of config files, to generate a report of startup and page load times • Before you run Firefox Performance Testing, refer to What happens when you run Firefox Performance Tests • You can run run_tests.py either from the DOS prompt or Cygwin DOS $run_tests.py config.yaml <anotherConfig.yaml>

Cygwin

./run_tests.py config.yaml <anotherConfig.yaml>

# Things you can configure manually

## Configuring the paths (paths.py)

• You may set up the appropriate paths for the framework manually.
• The current paths are set by default
• After you run perfconfig.sh, the paths.py file is located in C:\mozilla\testing\performance\win32\ by default
• The following are the constants in paths.py:

### CYGWIN

By default: CYGWIN = r'c:\cygwin\bin\bash.exe -c'

• The path to cygwin bash to get output from dump() in Firefox
• If you didn't specify a different root directory for the Cygwin to be installed, it will be installed in C:\ by default

### SYNC

By default: SYNC = r'c:\cygwin\bin\sync'

• Runs sync between Ts runs
• If you didn't specify a different root directory for the Cygwin to be installed, it is in C:\

### BASE_PROFILE_DIR

By default: BASE_PROFILE_DIR = r'C:\mozilla\testing\performance\win32\base_profile'

• IMPORTANT: Check that there is content in the base_profile directory
• (Optional) For the page load test to work, modify the hostperm.1 file
• Set to allow urls with scheme:file to open in new windows
• Set the preference to open new windows in a tab should be off.

### REPORTS_DIR

By default: REPORTS_DIR = r'C:\mozilla\testing\performance\win32\extension_perf_reports'

• The perfconfig.sh script creates a extension_perf_reports dir in C:\mozilla\testing\performance\win32\

### INIT_URL

By default: INIT_URL = 'file:///c:/mozilla/testing/performance/win32/initialize.html'

• The path to the file url to load when initializing a new profile
• Note: Don't change the local file url to a file path

### TS_URL

By default: TS_URL = 'file:///c:/mozilla/testing/performance/win32/startup_test/startup_test.html?begin='

• The path to the file url to load for startup test (Ts)
• Note: Don't change the local file url to a file path

### TP_URL

• The path to the file url to load for page load test (Tp)
• Note: Don't change the local file url to a file path

# What happens when you run Firefox Performance Tests

• All opened browser windows will be closed automatically!
• You might run into this message on your browser: This page should close Firefox. If it does not, please make sure that the dom.allow_scripts_to_close_windows preference is set to true in about:config
• Type auto:config in the address bar and double click on dom.allow_scripts_to_close_windows to change it to true
• If you see processing... Performance Testing is starting, performance testing is running..
• You can check the progress by the progress bar
• After the performance testing is done, your reports will be placed in the REPORTS_DIR directory you specified in paths.py
• By default, it your reports will go to: C:\mozilla\testing\performance\win32\extension_perf_reports
• You can use a space-separated list of config files, to generate a report of startup and page load times.