OLD | NEW |
1 Setup to run all performance tests: | 1 Overview: |
| 2 These are the instructions to run a wide variety of browser tests using |
| 3 test.dart --component=webdriver or |
| 4 dart/tools/testing/perf_testing/create_graph.py. Currently the results of |
| 5 create_graph are uploaded to https://dartperf.googleplex.com/. |
| 6 |
| 7 ============== Windows Setup ============== |
2 | 8 |
3 TODO(efortuna): run through these steps again on a clean machine to make sure I | 9 Open Internet Explorer. Open the Internet Options dialog, and move to the |
4 didn't leave anything out! | 10 "Security" tab. Click through Internet, Local Intranet, Trusted sites, and |
| 11 Restricted site and ensure the "Enable Protected Mode" is checked for all zones |
| 12 or unchecked for all zones. Then click to the "Advanced" tab, scroll down to the |
| 13 "Security" section, and check the checkbox "Allow active content to run in files |
| 14 on My Computer." |
5 | 15 |
6 Overview: | 16 ================ Mac Setup ================ |
7 These are the instructions to run a wide variety of performance tests using | |
8 dart/tools/testing/perf_testing/create_graph.py. Currently the results are | |
9 uploaded to https://dartperf.googleplex.com/. | |
10 | |
11 A variant of these tests are running on our buildbots to test Chrome and | |
12 Firefox (since the buildbots are Linux) that simply ensure that no changes to | |
13 frog have broken updating the dom, but not for testing performance. | |
14 | 17 |
15 This file details how to set up configurations for each setup. It is long, | 18 1) Ensure Java is installed and in your path (if you want to run Safari). If |
16 because there are many variants depending on what platform you're on, and what | 19 not, install the Java jdk (so that we can run the webdriver server from its |
17 you want to set up. | 20 jar) |
18 | 21 |
19 ============ Windows Installation ============= | 22 2) a) Disable pop-up blocking in Safari: |
20 | 23 Preferences -> Security -> (unselect) Block pop-up windows. |
21 Setting up Windows on Mac hardware: | 24 b) copy the file in /Library/Preferences/com.apple.Safari.plist to |
22 If you need to install Windows 7 via dual-boot mac, use BootCamp to install | 25 $DARTDIR/tools/testing/com.apple.Safari.plist |
23 Windows 7 Professional. You can obtain a copy of Windows 7 (Google internal) | 26 (We do this because Safari deletes our preferences (on no pop-up |
24 by getting an MSDN subscription via GUTS. When installing via BootCamp, don't | 27 blocking) if it crashes (aka times out) two times in a row.) |
25 forget to also install the "WindowsSupport" drivers that are optional -- | |
26 you'll need them to be able to connect to any wireless network in Windows. | |
27 | |
28 Great! You have a shiny dual-booting machine! | |
29 | |
30 Use the following instructions to set up for Windows builds: | |
31 http://www.chromium.org/developers/how-tos/build-instructions-windows | |
32 | |
33 NOTE: DO NOT USE Visual Studio 2010 to try to build the project!! Right now Dart | |
34 only builds with VS2008 versions! | |
35 | |
36 Then: | |
37 1) Download installer for OpenSSL for Windows via | |
38 http://openssl.org/related/binaries.html | |
39 2) Install OpenSSL | |
40 3) Copy the directory OpenSSL-Win32/include/openssl to | |
41 C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include | |
42 4) Copy the ***contents*** of the directory OpenSSL-Win32/lib/VC/static/ to | |
43 C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\lib | |
44 | 28 |
45 ============= All Platforms ================ | 29 ============= All Platforms ================ |
46 | 30 |
| 31 THE EASISET WAY: |
47 1) Ensure Python 2.7 is installed and in your path. | 32 1) Ensure Python 2.7 is installed and in your path. |
| 33 2) Download and install Chrome in the default location if you haven't done so |
| 34 already. |
| 35 3) Run the following script as root/administrator while standing in this |
| 36 directory: |
| 37 $> [sudo] python dart/tools/testing/perf_testing/webdriver_test_setup.py |
| 38 4) Profit! |
48 | 39 |
49 2) Run the following command while standing in this directory: | 40 Example Run: |
50 $> sudo buildbot_browser_test_setup.sh | 41 $> tools/test.py --component=webdriver --report --timeout=20 --mode=release |
51 | 42 --browser=[ff | safari | chrome | ie] |
52 If that doesn't work, or you're running Windows, here are the manual steps: | |
53 a) Install selenium library python bindings | |
54 (http://pypi.python.org/pypi/selenium) | |
55 b) Ensure Firefox is installed. | |
56 | |
57 3) Mac only: | |
58 Download and install xcode 3.2: | |
59 https://developer.apple.com/devcenter/download.action?path=/Developer_Tools
/xcode_3.2.6_and_ios_sdk_4.3__final/xcode_3.2.6_and_ios_sdk_4.3.dmg | |
60 a) Mount dmg | |
61 b) LION only: At a terminal type: | |
62 TODO(efortuna): verify | |
63 $> export COMMAND_LINE_INSTALL=1 | |
64 $> open "/Volumes/Xcode and iOS SDK/Xcode and iOS SDK.mpkg" | |
65 c) Install. | |
66 | |
67 4) (Mac OS Lion only) Install xcode 4.0 and then run: | |
68 $> xcode-select -switch /path/to/xcode3.2.6/ | |
69 | |
70 5) Ensure Java is installed and in your path. If not, install the Java jdk | |
71 (so that we can run the webdriver server from its jar) | |
72 | |
73 6) Download selenium-server-standalone-2.15.0.jar (only to run Safari) | |
74 http://selenium.googlecode.com/files/selenium-server-standalone-2.15.0.jar | |
75 and run it: | |
76 $> java -jar selenium-server-standalone-2.15.0.jar | |
77 | |
78 7) Ensure that Chrome, Safari and IE (Windows only) are installed. | |
79 | |
80 8)Download the Chrome Driver: http://code.google.com/p/chromium/downloads/list | |
81 and make sure it is in your path. | |
82 | |
83 9) a) Disable pop-up blocking in Safari: | |
84 Preferences -> Security -> (unselect) Block pop-up windows. | |
85 b) copy the file in /Library/Preferences/com.apple.Safari.plist to | |
86 $DARTDIR/tools/testing/com.apple.Safari.plist | |
87 (We do this because Safari deletes our preferences (on no pop-up | |
88 blocking) if it crashes (aka times out) two times in a row.) | |
89 | |
90 If you just want smoketests, you're done! Run them by typing: | |
91 | |
92 $> tools/testing/bin/$YOUR_OS_DIR/dart tools/test.dart --component=webdriver | |
93 --report --timeout=20 --mode=release --browser=[ff | safari | chrome | ie] | |
94 [--frog=path/to/frog/executable/like/Release_ia32/dart-sdk/frogc | 43 [--frog=path/to/frog/executable/like/Release_ia32/dart-sdk/frogc |
95 --froglib=path/to/frog/lib/like/dart/frog/lib] test_to_run(like "language" or | 44 --froglib=path/to/frog/lib/like/dart/frog/lib] test_to_run(like "language" or |
96 "corelib") | 45 "corelib") |
97 | 46 |
98 (If you don't specify frog and froglib arguments, we default to using the frogsh | 47 (If you don't specify frog and froglib arguments, we default to using frog with |
99 living in your frog directory.) | 48 the VM.) |
| 49 |
| 50 |
| 51 ============================================ |
| 52 Okay, so you're still here? Here's the long version of what that script does if |
| 53 it didn't work for you: |
| 54 a) Install selenium library python bindings |
| 55 (http://pypi.python.org/pypi/selenium) |
| 56 |
| 57 b) Ensure that Firefox is installed in the default location. |
| 58 |
| 59 c) Download the Chrome Driver: http://code.google.com/p/chromium/downloads/list |
| 60 and make sure it is in your path. |
| 61 |
| 62 d) Download the selenium server (version 2.15 or newer) and place it in this |
| 63 directory: |
| 64 http://selenium.googlecode.com/files/selenium-server-standalone-2.15.0.jar |
| 65 |
100 | 66 |
101 ========= Proceed further only if you also want to run performance tests.====== | 67 ========= Proceed further only if you also want to run performance tests.====== |
102 | 68 |
103 10)Download appengine for Python and place it in third_party (http://code.google
.com/appengine/downloads.html#Google_App_Engine_SDK_for_Python): | 69 10)Download appengine for Python and place it in third_party (http://code.google
.com/appengine/downloads.html#Google_App_Engine_SDK_for_Python): |
104 "dart/third_party/appengine-python/" | 70 "dart/third_party/appengine-python/" |
105 | 71 |
106 11)Install matplotlib http://matplotlib.sourceforge.net/ | 72 11)Install matplotlib http://matplotlib.sourceforge.net/ |
107 | 73 |
108 12)Pull down benchmarks from internal repo (Google only): | 74 12)Pull down benchmarks from internal repo (Google only): |
109 http://chromegw.corp.google.com/viewvc/dash/trunk/internal/browserBenchmarks/
README.txt?view=markup | 75 http://chromegw.corp.google.com/viewvc/dash/trunk/internal/browserBenchmarks/
README.txt?view=markup |
110 | 76 |
111 13)TODO(efortuna): Deal with appengine check in! Run | 77 13)TODO(efortuna): Deal with appengine check in! Run |
112 '../../../third_party/appengine-python/1.5.4/appcfg.py update appengine/' | 78 '../../../third_party/appengine-python/1.5.4/appcfg.py update appengine/' |
113 while standing in dart/tools/testing/perf_tests. | 79 while standing in dart/tools/testing/perf_tests. |
114 | 80 |
115 14) Run the tests! While standing in dart/tools/testing/perf_testing, run | 81 14) Run the tests! While standing in dart/tools/testing/perf_testing, run |
116 $> python create_graph.py --forever --verbose | 82 $> python create_graph.py --forever --verbose |
117 to run all the tests (browser performance, language correctness in the | 83 to run all the tests (browser performance, language correctness in the |
118 browser, command line performance, and self-hosted compile time and compiled | 84 browser, command line performance, and self-hosted compile time and compiled |
119 code size). | 85 code size). |
120 | 86 |
121 You can run individual tests by adding the particular option (such as | 87 You can run individual tests by adding the particular option (such as |
122 --language) when running create_graph.py. Type "create_graph.py -h" for a | 88 --language) when running create_graph.py. Type "create_graph.py -h" for a |
123 full list of the options. | 89 full list of the options. |
OLD | NEW |