Unittest filtering

Moved Main class to a new file called run_install_tests.py and modified it by adding the ability to filter unittests. This will allow users to run individual unittests, a set of tests, or the entire suite.

BUG=none
NOTRY=true

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=164090

[nkang, 2012-10-23 23:26:09]

Added support for running individual unittests or a set of unittests.

[kkania, 2012-10-24 15:19:54]

https://codereview.chromium.org/11234068/diff/1/install_test/install_test.py
File install_test/install_test.py (right):

https://codereview.chromium.org/11234068/diff/1/install_test/install_test.py#...
install_test/install_test.py:272: if os.path.dirname(sys.argv[0]) not in
sys.path:
I think this is unnecessary. sys.argv[0] was just executed, so the module should
already be in sys.modules

https://codereview.chromium.org/11234068/diff/1/install_test/install_test.py#...
install_test/install_test.py:298: def _GetFilteredTests(self):
move this whole Main class out to a new file, called run_install_tests.py;
delete the ability to run the tests by executing a single test file.

[nkang, 2012-10-24 23:35:14]

Addressed the comments made by Ken. The main change has to do with the Main
class, which was moved out of install_test.py and into a new file called
run_install_tests.py.

https://codereview.chromium.org/11234068/diff/1/install_test/install_test.py
File install_test/install_test.py (right):

https://codereview.chromium.org/11234068/diff/1/install_test/install_test.py#...
install_test/install_test.py:272: if os.path.dirname(sys.argv[0]) not in
sys.path:
On 2012/10/24 15:19:54, kkania wrote:
> I think this is unnecessary. sys.argv[0] was just executed, so the module
should
> already be in sys.modules

I agree. I was thinking the same thing that it should already be in sys.modules.
But, before removing this code, I wanted to test it so I could be certain that
the removal won't cause any problems. However, due to time constraints, I wasn't
able to do any testing. So I just erred on the side of caution and decided to
leave it in there. It is now removed.

https://codereview.chromium.org/11234068/diff/1/install_test/install_test.py#...
install_test/install_test.py:298: def _GetFilteredTests(self):
On 2012/10/24 15:19:54, kkania wrote:
> move this whole Main class out to a new file, called run_install_tests.py;
> delete the ability to run the tests by executing a single test file.

Moved Main to a new file called run_install_tests.py

[kkania, 2012-10-24 23:44:05]

http://codereview.chromium.org/11234068/diff/5001/install_test/run_install_te...
File install_test/run_install_tests.py (right):

http://codereview.chromium.org/11234068/diff/5001/install_test/run_install_te...
install_test/run_install_tests.py:65: '-f', '--filter', type='string',
default=None, dest='filter',
set default to '*'

http://codereview.chromium.org/11234068/diff/5001/install_test/run_install_te...
install_test/run_install_tests.py:194: if self._opts.filter:
i don't like the if and else here, instead just do:

all_tests = unittest.defaultTestLoader.loadTestsFromModule(sample_updater)
tests = self._FilterTestSuite(all_tests, self._opts.filter)

Delete the _GetFilteredTests method.  Always call the filter func, because the
default is '*'

http://codereview.chromium.org/11234068/diff/5001/install_test/run_install_te...
install_test/run_install_tests.py:207: sys.exit(Main())
remove the sys.exit() and just keep Main()

http://codereview.chromium.org/11234068/diff/5001/install_test/sample_updater.py
File install_test/sample_updater.py (right):

http://codereview.chromium.org/11234068/diff/5001/install_test/sample_updater...
install_test/sample_updater.py:6: """Runs sample updater and install tests using
one or more Chrome builds.
Change this comment to
"""Tests that demonstrate use of install test framework."""

Remove all the other comments

[nkang, 2012-10-24 23:52:45]

Addressed all comments.

http://codereview.chromium.org/11234068/diff/5001/install_test/run_install_te...
File install_test/run_install_tests.py (right):

http://codereview.chromium.org/11234068/diff/5001/install_test/run_install_te...
install_test/run_install_tests.py:65: '-f', '--filter', type='string',
default=None, dest='filter',
On 2012/10/24 23:44:05, kkania wrote:
> set default to '*'

Done.

http://codereview.chromium.org/11234068/diff/5001/install_test/run_install_te...
install_test/run_install_tests.py:194: if self._opts.filter:
On 2012/10/24 23:44:05, kkania wrote:
> i don't like the if and else here, instead just do:
> 
> all_tests = unittest.defaultTestLoader.loadTestsFromModule(sample_updater)
> tests = self._FilterTestSuite(all_tests, self._opts.filter)
> 
> Delete the _GetFilteredTests method.  Always call the filter func, because the
> default is '*'

Got rid of the if/else statement and the _GetFilteredTests method.

http://codereview.chromium.org/11234068/diff/5001/install_test/run_install_te...
install_test/run_install_tests.py:207: sys.exit(Main())
On 2012/10/24 23:44:05, kkania wrote:
> remove the sys.exit() and just keep Main()

Done.

http://codereview.chromium.org/11234068/diff/5001/install_test/sample_updater.py
File install_test/sample_updater.py (right):

http://codereview.chromium.org/11234068/diff/5001/install_test/sample_updater...
install_test/sample_updater.py:6: """Runs sample updater and install tests using
one or more Chrome builds.
On 2012/10/24 23:44:05, kkania wrote:
> Change this comment to
> """Tests that demonstrate use of install test framework."""
> 
> Remove all the other comments

Changed first line as suggested. Removed all other comments, except for the
first line.

[kkania, 2012-10-24 23:59:11]

lgtm

[commit-bot: I haz the power, 2012-10-25 16:45:59]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/nkang@chromium.org/11234068/9001

[commit-bot: I haz the power, 2012-10-25 16:46:01]

Failed to apply patch for chrome/test/install_test/sample_updater.py:
While running patch -p0 --forward --force --no-backup-if-mismatch;
  patching file chrome/test/install_test/sample_updater.py
  Hunk #1 FAILED at 3.
  1 out of 1 hunk FAILED -- saving rejects to file
chrome/test/install_test/sample_updater.py.rej

Patch:       chrome/test/install_test/sample_updater.py
Index: install_test/sample_updater.py
===================================================================
--- chrome/test/install_test/sample_updater.py	(revision 163097)
+++ chrome/test/install_test/sample_updater.py	(working copy)
@@ -3,37 +3,37 @@
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
-"""Runs sample updater test using two Chrome builds.
+"""Tests that demonstrate use of install test framework."""
 
-Test requires two arguments: build url and chrome versions to be used, rest of
-the arguments are optional. Builds must be specified in ascending order, with
-the lower version first and the higher version last, each separated by commas.
-The setUp method creates a ChromeDriver instance, which can be used to run
-tests. ChromeDriver is shutdown, upon completion of the testcase.
-
-Example:
-    $ python sample_updater.py --url=<URL> --builds=19.0.1069.0,19.0.1070.0
-"""
-
 import os
 import sys
 import unittest
 
 from install_test import InstallTest
-from install_test import Main
 
 
 class SampleUpdater(InstallTest):
   """Sample update tests."""
 
   def testCanOpenGoogle(self):
-    """Simple Navigation."""
+    """Simple Navigation.
+
+    This test case illustrates how to run an update test. Update tests require
+    two or more builds.
+    """
+    self.Install(self.GetUpdateBuilds()[0])
     self.StartChrome()
     self._driver.get('http://www.google.com/')
-    self.UpdateBuild()
+    self.Install(self.GetUpdateBuilds()[1])
     self.StartChrome()
     self._driver.get('http://www.google.org/')
 
+  def testCanOpenGoogleOrg(self):
+    """Simple Navigation.
 
-if __name__ == '__main__':
-  Main()
+    This test case illustrates how to run a fresh install test. Simple install
+    tests, unlike update test, require only a single build.
+    """
+    self.Install(self.GetInstallBuild())
+    self.StartChrome()
+    self._driver.get('http://www.google.org/')

[commit-bot: I haz the power, 2012-10-25 16:58:45]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/nkang@chromium.org/11234068/4004

[commit-bot: I haz the power, 2012-10-25 17:00:16]

Change committed as 164090