OLD | NEW |
(Empty) | |
| 1 Setup to run all performance tests: |
| 2 |
| 3 TODO(efortuna): run through these steps again on a clean machine to make sure I |
| 4 didn't leave anything out! |
| 5 |
| 6 Overview: |
| 7 These are the instructions to run a wide variety of performance tests using |
| 8 dart/tools/testing/perf_testing/create_graph.py. Currently the results are |
| 9 uploaded to https://dartperf.googleplex.com/. |
| 10 |
| 11 A variant of these tests are running on our buildbots to test Chrome and |
| 12 Firefox (since the buildbots are Linux) that simply ensure that no changes to |
| 13 frog have broken updating the dom, but not for testing performance. |
| 14 |
| 15 This file details how to set up configurations for each setup. It is long, |
| 16 because there are many variants depending on what platform you're on, and what |
| 17 you want to set up. |
| 18 |
| 19 ============ Windows Installation ============= |
| 20 |
| 21 Setting up Windows on Mac hardware: |
| 22 If you need to install Windows 7 via dual-boot mac, use BootCamp to install |
| 23 Windows 7 Professional. You can obtain a copy of Windows 7 (Google internal) |
| 24 by getting an MSDN subscription via GUTS. When installing via BootCamp, don't |
| 25 forget to also install the "WindowsSupport" drivers that are optional -- |
| 26 you'll need them to be able to connect to any wireless network in Windows. |
| 27 |
| 28 Great! You have a shiny dual-booting machine! |
| 29 |
| 30 Use the following instructions to set up for Windows builds: |
| 31 http://www.chromium.org/developers/how-tos/build-instructions-windows |
| 32 |
| 33 NOTE: DO NOT USE Visual Studio 2010 to try to build the project!! Right now Dart |
| 34 only builds with VS2008 versions! |
| 35 |
| 36 Then: |
| 37 1) Download installer for OpenSSL for Windows via |
| 38 http://openssl.org/related/binaries.html |
| 39 2) Install OpenSSL |
| 40 3) Copy the directory OpenSSL-Win32/include/openssl to |
| 41 C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include |
| 42 4) Copy the ***contents*** of the directory OpenSSL-Win32/lib/VC/static/ to |
| 43 C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\lib |
| 44 |
| 45 ============= All Platforms ================ |
| 46 |
| 47 1) Ensure Python 2.7 is installed and in your path. |
| 48 |
| 49 2) Run the following command while standing in this directory: |
| 50 $> sudo buildbot_browser_test_setup.sh |
| 51 |
| 52 If that doesn't work, or you're running Windows, here are the manual steps: |
| 53 a) Install selenium library python bindings |
| 54 (http://pypi.python.org/pypi/selenium) |
| 55 b) Ensure Firefox is installed. |
| 56 |
| 57 3) Mac only: |
| 58 Download and install xcode 3.2: |
| 59 https://developer.apple.com/devcenter/download.action?path=/Developer_Tools
/xcode_3.2.6_and_ios_sdk_4.3__final/xcode_3.2.6_and_ios_sdk_4.3.dmg |
| 60 a) Mount dmg |
| 61 b) LION only: At a terminal type: |
| 62 TODO(efortuna): verify |
| 63 $> export COMMAND_LINE_INSTALL=1 |
| 64 $> open "/Volumes/Xcode and iOS SDK/Xcode and iOS SDK.mpkg" |
| 65 c) Install. |
| 66 |
| 67 4) (Mac OS Lion only) Install xcode 4.0 and then run: |
| 68 $> xcode-select -switch /path/to/xcode3.2.6/ |
| 69 |
| 70 5) Ensure Java is installed and in your path. If not, install the Java jdk |
| 71 (so that we can run the webdriver server from its jar) |
| 72 |
| 73 6) Download selenium-server-standalone-2.15.0.jar (only to run Safari) |
| 74 http://selenium.googlecode.com/files/selenium-server-standalone-2.15.0.jar |
| 75 and run it: |
| 76 $> java -jar selenium-server-standalone-2.15.0.jar |
| 77 |
| 78 7) Ensure that Chrome, Safari and IE (Windows only) are installed. |
| 79 |
| 80 8)Download the Chrome Driver: http://code.google.com/p/chromium/downloads/list |
| 81 and make sure it is in your path. |
| 82 |
| 83 9) a) Disable pop-up blocking in Safari: |
| 84 Preferences -> Security -> (unselect) Block pop-up windows. |
| 85 b) copy the file in /Library/Preferences/com.apple.Safari.plist to |
| 86 $DARTDIR/tools/testing/com.apple.Safari.plist |
| 87 (We do this because Safari deletes our preferences (on no pop-up |
| 88 blocking) if it crashes (aka times out) two times in a row.) |
| 89 |
| 90 If you just want smoketests, you're done! Run them by typing: |
| 91 |
| 92 $> tools/testing/bin/$YOUR_OS_DIR/dart tools/test.dart --component=webdriver |
| 93 --report --timeout=20 --mode=release --browser=[ff | safari | chrome | ie] |
| 94 [--frog=path/to/frog/executable/like/Release_ia32/dart-sdk/frogc |
| 95 --froglib=path/to/frog/lib/like/dart/frog/lib] test_to_run(like "language" or |
| 96 "corelib") |
| 97 |
| 98 (If you don't specify frog and froglib arguments, we default to using the frogsh |
| 99 living in your frog directory.) |
| 100 |
| 101 ========= Proceed further only if you also want to run performance tests.====== |
| 102 |
| 103 10)Download appengine for Python and place it in third_party (http://code.google
.com/appengine/downloads.html#Google_App_Engine_SDK_for_Python): |
| 104 "dart/third_party/appengine-python/" |
| 105 |
| 106 11)Install matplotlib http://matplotlib.sourceforge.net/ |
| 107 |
| 108 12)Pull down benchmarks from internal repo (Google only): |
| 109 http://chromegw.corp.google.com/viewvc/dash/trunk/internal/browserBenchmarks/
README.txt?view=markup |
| 110 |
| 111 13)TODO(efortuna): Deal with appengine check in! Run |
| 112 '../../../third_party/appengine-python/1.5.4/appcfg.py update appengine/' |
| 113 while standing in dart/tools/testing/perf_tests. |
| 114 |
| 115 14) Run the tests! While standing in dart/tools/testing/perf_testing, run |
| 116 $> python create_graph.py --forever --verbose |
| 117 to run all the tests (browser performance, language correctness in the |
| 118 browser, command line performance, and self-hosted compile time and compiled |
| 119 code size). |
| 120 |
| 121 You can run individual tests by adding the particular option (such as |
| 122 --language) when running create_graph.py. Type "create_graph.py -h" for a |
| 123 full list of the options. |
OLD | NEW |