Things to fix for Performance Testing Framework

From CDOT Wiki
Jump to: navigation, search
Task Details Status
Fix the Documentation for Performance Testing Framework Configuration
  • Clarify that the paths are default and could be modified -
  • Confusion - contradiction for hostperm.1 file in documentation
(Optional) For the page load test to work, modify the hostperm.1 file

    * Set to allow urls with scheme:file to open in new windows
    * Set the preference to open new windows in a tab should be off. 
    # Permission File
    # This is a generated file! Do not edit.

  • Highlight sections that (may) need to be modified
    • Student comment: Within code, could be hard to understand.
  • BOLD and HIGHLIGHT the part: Running the test will close any existing Firefox windows.
  • Clarify any concerns about skewing the test
    • How much of this test is reliant on applications currently running on the system? Can one skew the test results to give inaccurate readings by overloading the system?
    • By running a quick test I realized that by overloading the processor, the results of the test become much less useful. Before running the test framework I started a multitude of applications to occupy the processor:
      • 2 x YouTube Flash Video Sites
      • 1 x Disk Defragmenter
      • 1 x Flash Game
      • 1 x Flash Media Site
      • 1 x Windows Media Streamed Video
  • Student commented that users want to quickly setup the environment and get the performance testing running
  • Clarify yaml config setup
    • Do mention that indentation is important :)
    • Bold (use "preferences : {}" for no prefs) and (use "extensions: {}" for none)
  • Should creation of the 'extension_perf_reports' directory be incorporated into the zip with the python files to save the user from having to make the directory?
  • Student Comment: Mention that you need to "download and install them, in this order". The reason is because I assumed this was an installation walkthrough guide so I presume you may need us to configure things manually in the guide.
  • Step-by-step guide
  • Organize by section
  • Clearer in these areas:
    • Be more specific in the "Configure the paths" section.
    • Explain what preferences : {} is.
    • if you're going to change the dom.allow_scripts_to_close_windows you have to cancel the py script before u change the setting.
    • firefox should not be open before the test starts.

In progress

Incorporated some of the items in here:

New Firefox Performance Testing Documentation

Fix the Performance Testing Framework
  • Give some sort of feedback on test conclusions, it currently just dumps you back to the DOS prompt.
  • Just say tests completed successfully/ failed etc.
  • Default configuration
  • Ask if the user wants to make a reports dir if it doesn't exist