| OLD | NEW |
| 1 Setup to run all performance tests: | 1 Overview: |
| 2 These are the instructions to run a wide variety of browser tests using |
| 3 test.dart --component=webdriver or |
| 4 dart/tools/testing/perf_testing/create_graph.py. Currently the results of |
| 5 create_graph are uploaded to https://dartperf.googleplex.com/. |
| 6 |
| 7 ============== Windows Setup ============== |
| 2 | 8 |
| 3 TODO(efortuna): run through these steps again on a clean machine to make sure I | 9 Open Internet Explorer. Open the Internet Options dialog, and move to the |
| 4 didn't leave anything out! | 10 "Security" tab. Click through Internet, Local Intranet, Trusted sites, and |
| 11 Restricted site and ensure the "Enable Protected Mode" is checked for all zones |
| 12 or unchecked for all zones. Then click to the "Advanced" tab, scroll down to the |
| 13 "Security" section, and check the checkbox "Allow active content to run in files |
| 14 on My Computer." |
| 5 | 15 |
| 6 Overview: | 16 ================ Mac Setup ================ |
| 7 These are the instructions to run a wide variety of performance tests using | |
| 8 dart/tools/testing/perf_testing/create_graph.py. Currently the results are | |
| 9 uploaded to https://dartperf.googleplex.com/. | |
| 10 | |
| 11 A variant of these tests are running on our buildbots to test Chrome and | |
| 12 Firefox (since the buildbots are Linux) that simply ensure that no changes to | |
| 13 frog have broken updating the dom, but not for testing performance. | |
| 14 | 17 |
| 15 This file details how to set up configurations for each setup. It is long, | 18 1) Ensure Java is installed and in your path (if you want to run Safari). If |
| 16 because there are many variants depending on what platform you're on, and what | 19 not, install the Java jdk (so that we can run the webdriver server from its |
| 17 you want to set up. | 20 jar) |
| 18 | 21 |
| 19 ============ Windows Installation ============= | 22 2) a) Disable pop-up blocking in Safari: |
| 20 | 23 Preferences -> Security -> (unselect) Block pop-up windows. |
| 21 Setting up Windows on Mac hardware: | 24 b) copy the file in /Library/Preferences/com.apple.Safari.plist to |
| 22 If you need to install Windows 7 via dual-boot mac, use BootCamp to install | 25 $DARTDIR/tools/testing/com.apple.Safari.plist |
| 23 Windows 7 Professional. You can obtain a copy of Windows 7 (Google internal) | 26 (We do this because Safari deletes our preferences (on no pop-up |
| 24 by getting an MSDN subscription via GUTS. When installing via BootCamp, don't | 27 blocking) if it crashes (aka times out) two times in a row.) |
| 25 forget to also install the "WindowsSupport" drivers that are optional -- | |
| 26 you'll need them to be able to connect to any wireless network in Windows. | |
| 27 | |
| 28 Great! You have a shiny dual-booting machine! | |
| 29 | |
| 30 Use the following instructions to set up for Windows builds: | |
| 31 http://www.chromium.org/developers/how-tos/build-instructions-windows | |
| 32 | |
| 33 NOTE: DO NOT USE Visual Studio 2010 to try to build the project!! Right now Dart | |
| 34 only builds with VS2008 versions! | |
| 35 | |
| 36 Then: | |
| 37 1) Download installer for OpenSSL for Windows via | |
| 38 http://openssl.org/related/binaries.html | |
| 39 2) Install OpenSSL | |
| 40 3) Copy the directory OpenSSL-Win32/include/openssl to | |
| 41 C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include | |
| 42 4) Copy the ***contents*** of the directory OpenSSL-Win32/lib/VC/static/ to | |
| 43 C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\lib | |
| 44 | 28 |
| 45 ============= All Platforms ================ | 29 ============= All Platforms ================ |
| 46 | 30 |
| 31 THE EASISET WAY: |
| 47 1) Ensure Python 2.7 is installed and in your path. | 32 1) Ensure Python 2.7 is installed and in your path. |
| 33 2) Download and install Chrome in the default location if you haven't done so |
| 34 already. |
| 35 3) Run the following script as root/administrator while standing in this |
| 36 directory: |
| 37 $> [sudo] python dart/tools/testing/perf_testing/webdriver_test_setup.py |
| 38 4) Profit! |
| 48 | 39 |
| 49 2) Run the following command while standing in this directory: | 40 Example Run: |
| 50 $> sudo buildbot_browser_test_setup.sh | 41 $> tools/test.py --component=webdriver --report --timeout=20 --mode=release |
| 51 | 42 --browser=[ff | safari | chrome | ie] |
| 52 If that doesn't work, or you're running Windows, here are the manual steps: | |
| 53 a) Install selenium library python bindings | |
| 54 (http://pypi.python.org/pypi/selenium) | |
| 55 b) Ensure Firefox is installed. | |
| 56 | |
| 57 3) Mac only: | |
| 58 Download and install xcode 3.2: | |
| 59 https://developer.apple.com/devcenter/download.action?path=/Developer_Tools
/xcode_3.2.6_and_ios_sdk_4.3__final/xcode_3.2.6_and_ios_sdk_4.3.dmg | |
| 60 a) Mount dmg | |
| 61 b) LION only: At a terminal type: | |
| 62 TODO(efortuna): verify | |
| 63 $> export COMMAND_LINE_INSTALL=1 | |
| 64 $> open "/Volumes/Xcode and iOS SDK/Xcode and iOS SDK.mpkg" | |
| 65 c) Install. | |
| 66 | |
| 67 4) (Mac OS Lion only) Install xcode 4.0 and then run: | |
| 68 $> xcode-select -switch /path/to/xcode3.2.6/ | |
| 69 | |
| 70 5) Ensure Java is installed and in your path. If not, install the Java jdk | |
| 71 (so that we can run the webdriver server from its jar) | |
| 72 | |
| 73 6) Download selenium-server-standalone-2.15.0.jar (only to run Safari) | |
| 74 http://selenium.googlecode.com/files/selenium-server-standalone-2.15.0.jar | |
| 75 and run it: | |
| 76 $> java -jar selenium-server-standalone-2.15.0.jar | |
| 77 | |
| 78 7) Ensure that Chrome, Safari and IE (Windows only) are installed. | |
| 79 | |
| 80 8)Download the Chrome Driver: http://code.google.com/p/chromium/downloads/list | |
| 81 and make sure it is in your path. | |
| 82 | |
| 83 9) a) Disable pop-up blocking in Safari: | |
| 84 Preferences -> Security -> (unselect) Block pop-up windows. | |
| 85 b) copy the file in /Library/Preferences/com.apple.Safari.plist to | |
| 86 $DARTDIR/tools/testing/com.apple.Safari.plist | |
| 87 (We do this because Safari deletes our preferences (on no pop-up | |
| 88 blocking) if it crashes (aka times out) two times in a row.) | |
| 89 | |
| 90 If you just want smoketests, you're done! Run them by typing: | |
| 91 | |
| 92 $> tools/testing/bin/$YOUR_OS_DIR/dart tools/test.dart --component=webdriver | |
| 93 --report --timeout=20 --mode=release --browser=[ff | safari | chrome | ie] | |
| 94 [--frog=path/to/frog/executable/like/Release_ia32/dart-sdk/frogc | 43 [--frog=path/to/frog/executable/like/Release_ia32/dart-sdk/frogc |
| 95 --froglib=path/to/frog/lib/like/dart/frog/lib] test_to_run(like "language" or | 44 --froglib=path/to/frog/lib/like/dart/frog/lib] test_to_run(like "language" or |
| 96 "corelib") | 45 "corelib") |
| 97 | 46 |
| 98 (If you don't specify frog and froglib arguments, we default to using the frogsh | 47 (If you don't specify frog and froglib arguments, we default to using frog with |
| 99 living in your frog directory.) | 48 the VM.) |
| 49 |
| 50 |
| 51 ============================================ |
| 52 Okay, so you're still here? Here's the long version of what that script does if |
| 53 it didn't work for you: |
| 54 a) Install selenium library python bindings |
| 55 (http://pypi.python.org/pypi/selenium) |
| 56 |
| 57 b) Ensure that Firefox is installed in the default location. |
| 58 |
| 59 c) Download the Chrome Driver: http://code.google.com/p/chromium/downloads/list |
| 60 and make sure it is in your path. |
| 61 |
| 62 d) Download the selenium server (version 2.15 or newer) and place it in this |
| 63 directory: |
| 64 http://selenium.googlecode.com/files/selenium-server-standalone-2.15.0.jar |
| 65 |
| 100 | 66 |
| 101 ========= Proceed further only if you also want to run performance tests.====== | 67 ========= Proceed further only if you also want to run performance tests.====== |
| 102 | 68 |
| 103 10)Download appengine for Python and place it in third_party (http://code.google
.com/appengine/downloads.html#Google_App_Engine_SDK_for_Python): | 69 10)Download appengine for Python and place it in third_party (http://code.google
.com/appengine/downloads.html#Google_App_Engine_SDK_for_Python): |
| 104 "dart/third_party/appengine-python/" | 70 "dart/third_party/appengine-python/" |
| 105 | 71 |
| 106 11)Install matplotlib http://matplotlib.sourceforge.net/ | 72 11)Install matplotlib http://matplotlib.sourceforge.net/ |
| 107 | 73 |
| 108 12)Pull down benchmarks from internal repo (Google only): | 74 12)Pull down benchmarks from internal repo (Google only): |
| 109 http://chromegw.corp.google.com/viewvc/dash/trunk/internal/browserBenchmarks/
README.txt?view=markup | 75 http://chromegw.corp.google.com/viewvc/dash/trunk/internal/browserBenchmarks/
README.txt?view=markup |
| 110 | 76 |
| 111 13)TODO(efortuna): Deal with appengine check in! Run | 77 13)TODO(efortuna): Deal with appengine check in! Run |
| 112 '../../../third_party/appengine-python/1.5.4/appcfg.py update appengine/' | 78 '../../../third_party/appengine-python/1.5.4/appcfg.py update appengine/' |
| 113 while standing in dart/tools/testing/perf_tests. | 79 while standing in dart/tools/testing/perf_tests. |
| 114 | 80 |
| 115 14) Run the tests! While standing in dart/tools/testing/perf_testing, run | 81 14) Run the tests! While standing in dart/tools/testing/perf_testing, run |
| 116 $> python create_graph.py --forever --verbose | 82 $> python create_graph.py --forever --verbose |
| 117 to run all the tests (browser performance, language correctness in the | 83 to run all the tests (browser performance, language correctness in the |
| 118 browser, command line performance, and self-hosted compile time and compiled | 84 browser, command line performance, and self-hosted compile time and compiled |
| 119 code size). | 85 code size). |
| 120 | 86 |
| 121 You can run individual tests by adding the particular option (such as | 87 You can run individual tests by adding the particular option (such as |
| 122 --language) when running create_graph.py. Type "create_graph.py -h" for a | 88 --language) when running create_graph.py. Type "create_graph.py -h" for a |
| 123 full list of the options. | 89 full list of the options. |
| OLD | NEW |