Automate Chrome Endure setup process.

chrome/test/functional/perf/endure_setup.py

+"""Automate the setup process of Chrome Endure environment.

[Nirnimesh, 2012/08/09 17:28:05]

license header?

[fdeng1, 2012/08/14 19:11:40]

Done.

+
+Usage:
+  python endure_setup.py [fetch|server] [option]
+
+We use <ENDURE_DIR> to refer to the root directory in which Chrome Endure
+is set up. By default, <ENDURE_DIR> is the current working directory.
+
+First, run "python endure_setup.py fetch". This command will automatically setup
+Chrome Endure in <ENDURE_DIR>.
+
+Next, try to run your first endure test by:
+  TEST_LENGTH=60 LOCAL_GRAPH='<ENDURE_DIR>/chrome_graph' \\
+  python <ENDURE_DIR>/src/chrome/test/functional/perf_endure_setup.py \\
+  perf_endure.ChromeEndureGmailTest.testGmailComposeDiscard \\
+The above commands runs a Chrome Endure test for 60 seconds and saves
+the results to <ENDURE_DIR>/chrome_graph.
+
+Last, run "python endure_setup.py server". This command will start a local
+HTTP server to serve <ENDURE_DIR>/chrome_graph. A port will be automatically
+picked. You can now view the result graphs via http://localhost:<GIVEN_PORT>.
+
+Use python endure_setup.py [fetch|server] --help for more options.
+"""

[dennis_jeffrey, 2012/08/08 23:41:07]

in this docstring, also list the dependencies that this module has.  For
example, it requires fetch_prebuilt_pyauto.py, among other things.

[fdeng1, 2012/08/14 19:11:40]

Add some text about dependencies on depot_tools and fetch_prebuilt_pyauto.py.

On 2012/08/08 23:41:07, dennis_jeffrey wrote:

+
+import BaseHTTPServer
+import optparse
+import os
+import shutil
+import SimpleHTTPServer
+import subprocess
+import sys
+import urllib
+import zipfile
+
+
+class SetupError(Exception):
+  """Catch errors in setting up Chrome Endure."""
+  pass
+
+
+class PlainHelpFormatter(optparse.IndentedHelpFormatter):
+  """Format the help message of this script."""
+
+  def format_description(self, description):
+    if description:
+      return description + '\n'
+    else:
+      return ''
+
+
+class CmdFetch(object):
+  """Fetch Chrome Endure.
+
+  Usage:
+    python endure_setup.py fetch [options]

[Nirnimesh, 2012/08/09 17:28:05]

leave a blank line

[fdeng1, 2012/08/14 19:11:40]

Done.

+  Examples:
+    python endure_setup.py fetch
+      Fetch the latest version of Chrome Endure to the current
+      working directory.
+
+    python endure_setup.py fetch --endure-dir=/home/user/endure_dir
+      Fetch the latest version of Chrome Endure to /home/user/endure_dir.
+  """
+  _URLS = {'depot_tools': ('http://src.chromium.org'
+                           '/chrome/trunk/tools/depot_tools'),
+           'pyauto': ('https://src.chromium.org/'
+                      'chrome/trunk/src/chrome/test/functional.DEPS'),
+           # TODO(fdeng): change to an external address after graph code
+           # is checked in
+           'graph': ('http://www.corp.google.com/'
+                     '~dennisjeffrey/chrome_perf/local_graphs.zip'),
+          }
+  NAME = 'fetch'
+  DESCRIPTION = 'Fetch Chrome Endure.'
+
+  def _ParseArgs(self, argv):
+    parser = optparse.OptionParser(
+        usage='%%prog %s [options]' % self.NAME,
+        formatter=PlainHelpFormatter(),
+        description=self.__doc__)
+    parser.add_option(
+        '-d', '--endure-dir', type='string', default=os.getcwd(),
+        help='Directory in which to setup or update. ' \
+             'Default value is the current working directory')
+    return parser.parse_args(argv)
+
+  def Run(self, argv):
+    """Run this command."""
+    options, _ = self._ParseArgs(argv)
+    self._endure_dir = os.path.abspath(options.endure_dir)
+    self._depot_dir = os.path.join(self._endure_dir, 'depot_tools')
+    self._gclient = os.path.join(self._depot_dir, 'gclient')
+    self._fetch_py = os.path.join(self._endure_dir, 'src', 'chrome',
+                                  'test', 'pyautolib',
+                                  'fetch_prebuilt_pyauto.py')
+    self._binary_dir = os.path.join(self._endure_dir, 'src', 'out', 'Release')
+    self._graph_dir = os.path.join(self._endure_dir, 'chrome_graph')
+
+    if not os.path.isdir(self._endure_dir):
+      os.makedirs(self._endure_dir)
+
+    print 'Checking depot tools...'

[dennis_jeffrey, 2012/08/08 23:41:07]

i recommend using the "logging" module instead of print so we can choose/control
the verbosity level with which we output logging messages.  We may want some of
these messages printed on every run, but there might be other less-important
messages that we may only want to output when debugging or when we explicitly
want verbose output

[fdeng1, 2012/08/14 19:11:40]

Switch from "print" to "logging".
On 2012/08/08 23:41:07, dennis_jeffrey wrote:

+    self._FetchDepot()
+    print 'Fetching PyAuto (python code)...'
+    self._FetchPyAuto()
+    print 'Fetching binaries(chrome, pyautolib, chrome driver)...'
+    self._FetchBinaries()
+    # TODO(fdeng): remove this after it is check into the chrome tree.
+    print 'Fetching chrome graphing files...'
+    self._FetchGraph()
+    return 0
+
+  def _FetchDepot(self):
+    """Fetch depot_tool if not installed in the system."""
+    try:
+      subprocess.call(['gclient', '--version'])
+      self._gclient = 'gclient'
+    except OSError:
+      # TODO(fdeng): how can I know gclient is not installed
+      # without checking OSError, a better way?

[dennis_jeffrey, 2012/08/08 23:41:07]

it's possible depot_tools may already be installed on the system somewhere, but
the "gclient" command may just not be in the path.  so it's possible you may end
up installing a second copy of depot tools here.

Why not skip the check entirely and just install depot tools into the known
location in self._depot_dir regardless of whether it might already be installed
on the system? 

[fdeng1, 2012/08/14 19:11:40]

Done.

+      url = self._URLS['depot_tools']
+      print 'Fetching depot tools: %s' % url
+      subprocess.call(['svn', 'co', self._URLS['depot_tools'],
+                       self._depot_dir])
+      subprocess.call([self._gclient, '--version'])

[dennis_jeffrey, 2012/08/08 23:41:07]

is there any way to verify that we were successful in fetching depot tools? 
Maybe check to make sure the expected directory exists?

[fdeng1, 2012/08/14 19:11:40]

Added code to check that gclient/gclient.bat(win) and gclient.py exist.

On 2012/08/08 23:41:07, dennis_jeffrey wrote:

+
+  def _FetchPyAuto(self):
+    """Use gclient to fetch python code."""
+    cur_dir = os.getcwd()
+    os.chdir(self._endure_dir)
+    # gclient config
+    config_cmd = [self._gclient, 'config', self._URLS['pyauto']]

[dennis_jeffrey, 2012/08/08 23:41:07]

Did Nirnimesh say it's not sufficient to use pyauto's functional.DEPS here?

[fdeng1, 2012/08/14 19:11:40]

I think we were talking about webpagereplay was not added to functinoal.DEPS. I
can submit a separate CL about this. I'll confirm with him whether there is
anything else. 

On 2012/08/08 23:41:07, dennis_jeffrey wrote:

[dennis_jeffrey, 2012/08/15 17:53:08]

Ok.  I thought there was some problem where the functional.DEPS won't be able to
pull in the correct version of WPR.  If that's the case, we'll have to figure
out another way to do it.

+    code = subprocess.call(config_cmd)
+    if code != 0:
+      raise SetupError('Running "%s" failed.' % ' '.join(config_cmd))
+    # gclient sync
+    sync_cmd = [self._gclient, 'sync']
+    code = subprocess.call(sync_cmd)
+    if code != 0:
+      raise SetupError('Running "%s" failed.' % ' '.join(sync_cmd))
+    os.chdir(cur_dir)

[dennis_jeffrey, 2012/08/08 23:41:07]

should we also check to see whether an expected directory exists, to verify that
pyauto was likely fetched properly?

[fdeng1, 2012/08/14 19:11:40]

Add _CheckPyAuto()
Check src/chrome/test/pyatuolib/pyauto.py
and src/chrome/test/pyatuolib/fetch_prebuilt_pyauto.py exist.
I was thinking checking more files such as perf.py perf_endure.py too but I
remember we had a plan to move them in to functional/perf?

On 2012/08/08 23:41:07, dennis_jeffrey wrote:

[dennis_jeffrey, 2012/08/15 17:53:08]

I think this is good.  I just thought a simple sanity check would be good to
make sure things were likely successful in checking out the code.
We probably don't need to check for those.  And you're right, eventually I'd
like to move the other perf files into the perf folder.

+
+  def _FetchBinaries(self):
+    """Get the prebuilt binaries from continuous build archive."""
+    if not os.path.exists(self._fetch_py):
+      raise SetupError(
+          'Unable to find %s, did fetching python code succeed?'
+          % self._fetch_py)
+    print 'Cleaning %s' % self._binary_dir
+    if os.path.exists(self._binary_dir):
+      shutil.rmtree(self._binary_dir)
+    print 'Downloading...'
+    cmd = [self._fetch_py, '-d', self._binary_dir, '--latest']
+    code = subprocess.call(cmd)

[dennis_jeffrey, 2012/08/08 23:41:07]

if this is successful, maybe we could log a message saying which version of
chrome was downloaded.

[fdeng1, 2012/08/14 19:11:40]

Done.

+    if code != 0:
+      raise SetupError('Running "%s" failed.' % ' '.join(cmd))
+
+  def _FetchGraph(self):
+    """Fetch graph code."""
+    graph_zip = urllib.urlretrieve(self._URLS['graph'])[0]
+    if graph_zip == None or not os.path.exists(graph_zip):
+      raise SetupError('Unable to retrieve %s' % self._URLS['graph'])
+    if os.path.exists(self._graph_dir):
+      print 'Cleaning %s, ' % self._graph_dir,

[dennis_jeffrey, 2012/08/08 23:41:07]

Do we need to do all this work to clean the old directory?  What happens if we
just extract the new graph files into the destination directory?  If it simply
overwrites the files that need to be overwritten and ignores the other files
already there (e.g., .dat files), then that's likely what we want.

[fdeng1, 2012/08/14 19:11:40]

Done.

+      print 'data files(.dat) will be preserved.'
+      files = os.listdir(self._graph_dir)
+      for f in files:
+        path = os.path.join(self._graph_dir, f)
+        if os.path.isdir(path):
+          shutil.rmtree(path)
+        elif os.path.splitext(f) != 'dat':
+          os.remove(path)
+    else:
+      os.mkdir(self._graph_dir)
+    self._UnzipFilenameToDir(graph_zip, self._graph_dir)
+
+  @classmethod
+  def _UnzipFilenameToDir(cls, filename, directory):
+    """Unzip |filename| to directory |directory|.
+
+    This works with as low as python2.4 (used on win).
+    """
+    zf = zipfile.ZipFile(filename)
+    pushd = os.getcwd()
+    if not os.path.isdir(directory):
+      os.mkdir(directory)
+    os.chdir(directory)
+    # Extract files.
+    for info in zf.infolist():
+      name = info.filename
+      print name
+      if name.endswith('/'):  # dir
+        if not os.path.isdir(name):
+          os.makedirs(name)
+      else:  # file
+        directory = os.path.dirname(name)
+        if directory and not os.path.isdir(directory):
+          os.makedirs(directory)
+        out = open(name, 'wb')
+        out.write(zf.read(name))
+        out.close()
+      # Set permissions. Permission info in external_attr is shifted 16 bits.
+      os.chmod(name, info.external_attr >> 16L)

[dennis_jeffrey, 2012/08/08 23:41:07]

did you get this code from somewhere?  should we cite the source if so?

[fdeng1, 2012/08/14 19:11:40]

Mentioned it in docstring that it is adapted from fetch_prebuilt_pyauto.py
On 2012/08/08 23:41:07, dennis_jeffrey wrote:

+    os.chdir(pushd)
+
+
+class CmdServer(object):
+  """Start an http server which serves the Chrome Endure test results.

[dennis_jeffrey, 2012/08/08 23:41:07]

'test results' --> 'graphs'.

[fdeng1, 2012/08/14 19:11:40]

Done.

+
+  Usage:
+    python endure_setup.py server [options]
+  Examples:
+    python endure_setup.py server
+      Start a server which serves the default location
+      <CURRENT_WORKING_DIR>/chrome_graph.
+
+    python endure_setup.py server --graph-dir=/home/user/Document/graph_dir
+      Start a server which serves /home/user/Document/graph_dir which
+      is where your graph code and test results are.
+  """
+  NAME = 'server'
+  DESCRIPTION = 'Start an http server for viewing the' \
+                'tests results from a browser.'
+
+  def _ParseArgs(self, argv):
+    parser = optparse.OptionParser(
+        usage='%%prog %s [options]' % self.NAME,
+        formatter=PlainHelpFormatter(),
+        description=self.__doc__)
+    parser.add_option(
+        '-g', '--graph-dir', type='string',
+        default=os.path.join(os.getcwd(), 'chrome_graph'),
+        help='The directory that contains graph code ' \
+             'and data files of test results. Default value is ' \
+             '<CURRENT_WORKING_DIR>/chrome_graph')
+    return parser.parse_args(argv)
+
+  def Run(self, argv):
+    """Run this command."""
+    options, _ = self._ParseArgs(argv)
+    self._graph_dir = os.path.abspath(options.graph_dir)
+    cur_dir = os.getcwd()
+    os.chdir(self._graph_dir)
+    httpd = BaseHTTPServer.HTTPServer(('', 0),
+                                      EndureHTTPRequestHandler)
+    try:
+      print 'Serving %s at port %d' % (self._graph_dir, httpd.server_port)
+      print 'View test results at http://localhost:%d' % httpd.server_port
+      print 'Press Ctrl-C to stop the server.'
+      httpd.serve_forever()
+    except KeyboardInterrupt:
+      print 'Shutting down ...'
+      httpd.shutdown()
+    finally:
+      os.chdir(cur_dir)
+    return 0
+
+
+class EndureHTTPRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
+  """A simple HTTP request handler for showing Chrome Endure test results."""
+
+  def send_head(self):
+    """Override send_head so it won't redirect if path is not ended with "/".
+
+    Common code for GET and HEAD commands. This sends the response
+    code and MIME headers.
+
+    Returns:
+      either a file object (which has to be copied
+      to the output file by the caller unless the command was HEAD,
+      and must be closed by the caller under all circumstances), or
+      None, in which case the caller has nothing further to do.
+    """
+    path = self.translate_path(self.path)
+    f = None
+    if os.path.isdir(path):
+      for index in 'index.html', 'index.htm':
+        index = os.path.join(path, index)
+        if os.path.exists(index):
+          path = index
+          break
+      else:
+        return self.list_directory(path)
+    ctype = self.guess_type(path)
+    try:
+      # Always read in binary mode. Opening files in text mode may cause
+      # newline translations, making the actual size of the content
+      # transmitted *less* than the content-length!
+      f = open(path, 'rb')
+    except IOError:
+      self.send_error(404, 'File not found')
+      return None
+    self.send_response(200)
+    self.send_header('Content-type', ctype)
+    fs = os.fstat(f.fileno())
+    self.send_header('Content-Length', str(fs[6]))
+    self.send_header('Last-Modified', self.date_time_string(fs.st_mtime))
+    self.end_headers()
+    return f
+
+
+class CmdHelp(object):
+  """Print a list of commands or help for a specific command."""
+  NAME = 'help'
+  DESCRIPTION = __doc__
+
+  def __init__(self, cmds):
+    """Initialize help command.
+
+    Args:
+      cmds: Commands for which help information will be printed.
+    """
+    self.cmds = cmds
+
+  def Run(self, args):
+    """Run this command."""
+    if len(args) == 1:
+      return Main(args + ['--help'])
+    #print 'Usage:\n  ./endure_setup.py [options] command'
+    print __doc__
+    print 'Commands are:'
+    for cmd in self.cmds:
+      print '\t%s\t\t%s' % (cmd.NAME, cmd.DESCRIPTION)
+
+    return 0
+
+
+def Main(argv):

[dennis_jeffrey, 2012/08/08 23:41:07]

maybe we should split up this script into 2 separate scripts.  One could be
called endure_setup.py, which will just do the "fetch" command.  Another could
be called endure_graph_server.py, which will just do the "server" command.  I
think it's worthwhile to make 2 different scripts for both of those primary
functions, since they're independent of each other anyway.

[fdeng1, 2012/08/09 03:26:16]

Since we are able to solve the problem brought by extra '/' (by modifying the
graph code), the user can simply call "python -m SimpleHTTPServer 8000" under
the graph dir to start a server. The difference is that he needs to go inside
that folder as SimpleHTTPServer don't taken a directory as input. He also need
to pick a free port by himself. Maybe we can get rid of the server feature and
just document it somewhere(e.g. the help message)? 

[dennis_jeffrey, 2012/08/09 17:16:27]

I like the feature of having a free port automatically selected for the user
though, and not having to go into that directory manually to start the server. 
It might still be worthwhile to write a small, separate script that will
automatically launch the server for the user and tell him/her the URL to go to,
to see the graphs.

[fdeng1, 2012/08/14 19:11:40]

That sounds reasonable. I've moved it to a separate script. 

+  if argv and argv[0] == CmdFetch.NAME:
+    command = CmdFetch()
+  elif argv and argv[0] == CmdServer.NAME:
+    command = CmdServer()
+  else:
+    cmds = [CmdFetch, CmdServer, CmdHelp]

[dennis_jeffrey, 2012/08/08 23:41:07]

I think it's easier to use optparse for parsing args.  That'll eliminate the
need for the CmdHelp class too.  If you split up this file into 2 separate
scripts, we can just use optparse to parse the args in both scripts.

[fdeng1, 2012/08/14 19:11:40]

Done.

+    command = CmdHelp(cmds)
+  return command.Run(argv[1:])
+
+
+if '__main__' == __name__:
+  sys.exit(Main(sys.argv[1:]))