Run Chrome Endure tests with network simulation via Web Page Replay.

chrome/test/functional/record_endure.py

+#!/usr/bin/env python
+# Copyright (c) 2012 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+"""Record webapp interactions using Web Page Replay for Endure test.

[dennis_jeffrey, 2012/08/03 19:39:54]

add a blank line right above this

[fdeng1, 2012/08/06 20:58:24]

Done.

+
+Example:

[dennis_jeffrey, 2012/08/03 19:39:54]

before giving specific examples, mention the general form of the command to run
this script, something like this:

./record_endure.py [options] archive_file_name tests_to_run

And mention what happens by default (e.g., each test runs for 30 seconds and the
archive file gets stored into a some default location).

[fdeng1, 2012/08/06 20:58:24]

Done.

+1) ./record_endure.py ChromeEndureGmailTest.wpr
+   perf.ChromeEndureGmailTest
+
+   The above command runs each test in perf.ChromeEndureGmailTest for
+   30 secs and saves archive file to 'src/chrome/test/data/pyauto_private/
+   webpagereplay/ChromeEndureGmailTest.wpr' (default setting).
+
+2) ./record_endure.py --test-length=40 --out-dir='.'
+   ChromeEndureGmailTest.wpr perf.ChromeEndureGmailTest
+
+   The above command runs each test for 40 seconds and saves the archive to
+   current directory.
+
+Please make sure that the archive name matches the one that your Endure
+test will use.
+"""
+import optparse

[dennis_jeffrey, 2012/08/03 19:39:54]

add a blank line right above this

[fdeng1, 2012/08/06 20:58:24]

Done.

+import os
+import subprocess
+import sys
+import perf

[dennis_jeffrey, 2012/08/03 19:39:54]

add a blank line right above this to separate standard library imports from
application-specific imports

[fdeng1, 2012/08/06 20:58:24]

Done.

+import webpagereplay
+
+
+class PlainHelpFormatter(optparse.IndentedHelpFormatter):
+  """Format the help message of this script."""
+
+  def format_description(self, description):
+    if description:
+      return description + '\n'
+    else:
+      return ''
+
+
+class Recorder(object):
+  """Record webapp interactions of Endure tests via Web Page Replay."""
+
+  _PATH = {
+      'out_dir':

[dennis_jeffrey, 2012/08/03 19:39:54]

rather than having 2 inputs to represent the archive file (one optional
parameter for the path and one required argument for the archive file name),
could we just combine the two into one optional input named "--archive-file"
that has both the path and the file name in the string?

By default, we can have it create a file "archive.wpr" in the current working
directory.  Otherwise the user can specify a new file like this:

./record_endure --archive-file=<full path and file name of the wpr file to
create>

[fdeng1, 2012/08/06 20:58:24]

Done.

+      'src/chrome/test/data/pyauto_private/webpagereplay',
+      'scripts':
+      'src/chrome/test/data/chrome_endure/webpagereplay/wpr_deterministic.js',
+      'perf_endure':

[dennis_jeffrey, 2012/08/03 19:39:54]

maybe we can remove this and just assume that perf_endure.py exists in the
current working directory.  We can just list that as an assumption of this
script in the docstring at the top of the file

[fdeng1, 2012/08/06 20:58:24]

Done.

+      'src/chrome/test/functional/perf_endure.py',
+      }
+
+  @classmethod
+  def _ParseArgs(cls):
+    parser = optparse.OptionParser(
+        usage='%prog [options] archive_name test1 [test2]...',

[dennis_jeffrey, 2012/08/03 19:39:54]

test1 --> test_or_suite_1
test2 --> test_or_suite_2

(just to highlight the fact that they can be individual test names or groups of
tests, since pyauto allows for both to be specified on the command line)

[fdeng1, 2012/08/06 20:58:24]

Done.

+        formatter=PlainHelpFormatter(),
+        description=__doc__)
+    message = 'Length of each test in seconds. Default value is 30 secs.'
+    parser.add_option('--test-length', type='int', default=30,
+                      help=message)
+    message = ('Output directory of the archive. Default direcotry is ' +
+               cls._PATH['out_dir'])
+    parser.add_option('--out-dir', type='string', default=None,
+                      help=message)
+    options, args = parser.parse_args()
+    if len(args) <= 1:
+      parser.error('Invalid arguments. Use --help to see usage.')
+    return options, args
+
+  @classmethod
+  def Main(cls):
+    """Set up Web Page Replay and run Endure tests."""
+    options, args = cls._ParseArgs()
+    test_length = options.test_length
+    # Use a greater value than test length to disable stats.
+    perf_stats_interval = test_length + 60
+    user_dir = os.path.abspath(options.out_dir) if options.out_dir else None
+    out_dir = user_dir or perf.FormatChromePath(
+        os.path.join(cls._PATH['out_dir']))
+    archive_path = os.path.join(out_dir, args[0])
+    tests = args[1:]
+    replay_options = [
+        '--record',
+        '--inject_scripts', perf.FormatChromePath(cls._PATH['scripts']),
+        ]
+    with webpagereplay.ReplayServer(archive_path, replay_options):
+      env = os.environ.copy()
+      env.update({
+          'TEST_LENGTH': str(test_length),
+          'PERF_STATS_INTERVAL': str(perf_stats_interval),
+          'WPR_RECORD': '1',
+          'DO_NOT_RESTART_PYTHON_FOR_PYAUTO': '1',
+      })
+
+      cmd = ['python', perf.FormatChromePath(cls._PATH['perf_endure'])]
+      cmd.extend(tests)
+      subprocess.call(cmd, env=env)

[dennis_jeffrey, 2012/08/03 19:39:54]

i think it's possible that an archive file might still be created even though
one or more pyauto tests fail.  We should probably check the return code of
pyauto itself to make sure it's 0.  If it's not 0, then some pyauto test
probably failed and we should alert the user that even though an archive file
was created, it might be invalid.

[fdeng1, 2012/08/06 20:58:24]

Done.

+      print 'Shutting down server.'
+    if os.path.exists(archive_path):
+      print 'Archive is saved to %s' % archive_path
+      return 0
+    else:
+      print 'Error in recording. See log: %s' % webpagereplay.LOG_PATH
+      return 1
+
+
+if __name__ == '__main__':
+  sys.exit(Recorder().Main())