Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(221)

Side by Side Diff: chrome/test/chromedriver/test_expectations

Issue 14301024: [chromedriver] Clean tmp directory and kill Chrome processes before each run cycle. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Address comments. Created 7 years, 7 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « chrome/test/chromedriver/run_buildbot_steps.py ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright (c) 2013 The Chromium Authors. All rights reserved. 1 # Copyright (c) 2013 The Chromium Authors. All rights reserved.
2 # Use of this source code is governed by a BSD-style license that can be 2 # Use of this source code is governed by a BSD-style license that can be
3 # found in the LICENSE file. 3 # found in the LICENSE file.
4 4
5 """Test expectation list for WebDriver Java acceptance tests. 5 """Test expectation list for WebDriver Java acceptance tests.
6 6
7 It is evaluated through Python. 7 It is evaluated through Python.
8 """ 8 """
9 9
10 import fnmatch 10 import fnmatch
11 11
12 12
13 _REVISION_NEGATIVE_FILTER = {} 13 _REVISION_NEGATIVE_FILTER = {}
14 _REVISION_NEGATIVE_FILTER['HEAD'] = [ 14 _REVISION_NEGATIVE_FILTER['HEAD'] = [
15 'AlertsTest.*', 15 'AlertsTest.*',
16 'PerformanceLoggingTest.*',
17 'ChromeOptionsFunctionalTest.canStartChromeWithCustomOptions',
18 'AuthenticatedPageLoadingTest.*', 16 'AuthenticatedPageLoadingTest.*',
19 'AvailableLogsTest.browserLogShouldBeEnabledByDefault', 17 'AvailableLogsTest.browserLogShouldBeEnabledByDefault',
20 'AvailableLogsTest.clientLogShouldBeEnabledByDefault', 18 'AvailableLogsTest.clientLogShouldBeEnabledByDefault',
21 'AvailableLogsTest.driverLogShouldBeEnabledByDefault', 19 'AvailableLogsTest.driverLogShouldBeEnabledByDefault',
22 'AvailableLogsTest.profilerLogShouldBeDisabledByDefault', 20 'AvailableLogsTest.profilerLogShouldBeDisabledByDefault',
23 'AvailableLogsTest.shouldBeAbleToEnableProfilerLog', 21 'AvailableLogsTest.shouldBeAbleToEnableProfilerLog',
24 'BasicMouseInterfaceTest.testContextClick', 22 'BasicMouseInterfaceTest.testContextClick',
25 'BasicMouseInterfaceTest.testMousePositionIsNotPreservedInActionsChain', 23 'BasicMouseInterfaceTest.testMousePositionIsNotPreservedInActionsChain',
26 'BasicMouseInterfaceTest.testMovingIntoAnImageEnclosedInALink', 24 'BasicMouseInterfaceTest.testMovingIntoAnImageEnclosedInALink',
27 'BasicMouseInterfaceTest.testMovingMouseBackAndForthPastViewPort', 25 'BasicMouseInterfaceTest.testMovingMouseBackAndForthPastViewPort',
28 'BasicMouseInterfaceTest.testMovingMousePastViewPort', 26 'BasicMouseInterfaceTest.testMovingMousePastViewPort',
29 'ClickScrollingTest.testShouldNotBeAbleToClickElementThatIsOutOfViewInANonSc rollableFrame', 27 'ClickScrollingTest.testShouldNotBeAbleToClickElementThatIsOutOfViewInANonSc rollableFrame',
30 'ClickScrollingTest.testShouldNotScrollIfAlreadyScrolledAndElementIsInView', 28 'ClickScrollingTest.testShouldNotScrollIfAlreadyScrolledAndElementIsInView',
31 'ClickTest.testCanClickAnImageMapArea', 29 'ClickTest.testCanClickAnImageMapArea',
32 'ClickTest.testCanClickOnALinkThatContainsEmbeddedBlockElements',
33 'CombinedInputActionsTest.testClickingOnFormElements', 30 'CombinedInputActionsTest.testClickingOnFormElements',
34 'CombinedInputActionsTest.testHoldingDownShiftKeyWhileClicking',
35 'CoordinatesTest.testShouldScrollPageAndGetCoordinatesOfAnElementThatIsOutOf ViewPort', 31 'CoordinatesTest.testShouldScrollPageAndGetCoordinatesOfAnElementThatIsOutOf ViewPort',
36 'CorrectEventFiringTest.testShouldEmitOnClickEventsWhenSelectingElements',
37 'CorrectEventFiringTest.testShouldFireEventsInTheRightOrder', 32 'CorrectEventFiringTest.testShouldFireEventsInTheRightOrder',
38 'CorrectEventFiringTest.testShouldFireFocusEventWhenClicking', 33 'CorrectEventFiringTest.testShouldFireFocusEventWhenClicking',
39 'DragAndDropTest.testDragTooFar', 34 'DragAndDropTest.testDragTooFar',
40 'ExecutingAsyncJavascriptTest.includesAlertTextInUnhandledAlertException', 35 'ExecutingAsyncJavascriptTest.includesAlertTextInUnhandledAlertException',
41 'ExecutingAsyncJavascriptTest.throwsIfAlertHappensDuringScript', 36 'ExecutingAsyncJavascriptTest.throwsIfAlertHappensDuringScript',
42 'ExecutingAsyncJavascriptTest.throwsIfAlertHappensDuringScriptWhichTimesOut' , 37 'ExecutingAsyncJavascriptTest.throwsIfAlertHappensDuringScriptWhichTimesOut' ,
43 'ExecutingAsyncJavascriptTest.throwsIfScriptTriggersAlert', 38 'ExecutingAsyncJavascriptTest.throwsIfScriptTriggersAlert',
44 'ExecutingAsyncJavascriptTest.throwsIfScriptTriggersAlertWhichTimesOut', 39 'ExecutingAsyncJavascriptTest.throwsIfScriptTriggersAlertWhichTimesOut',
45 'ExecutingJavascriptTest.testShouldThrowExceptionIfExecutingOnNoPage', 40 'ExecutingJavascriptTest.testShouldThrowExceptionIfExecutingOnNoPage',
46 'FormHandlingTest.handleFormWithJavascriptAction', 41 'FormHandlingTest.handleFormWithJavascriptAction',
47 'FormHandlingTest.testShouldNotBeAbleToSubmitAFormThatDoesNotExist', 42 'FormHandlingTest.testShouldNotBeAbleToSubmitAFormThatDoesNotExist',
48 'FrameSwitchingTest.testFrameSearchesShouldBeRelativeToTheCurrentlySelectedF rame', 43 'FrameSwitchingTest.testFrameSearchesShouldBeRelativeToTheCurrentlySelectedF rame',
49 'FrameSwitchingTest.testShouldNotBeAbleToDoAnythingTheFrameIsDeletedFromUnde rUs', 44 'FrameSwitchingTest.testShouldNotBeAbleToDoAnythingTheFrameIsDeletedFromUnde rUs',
50 'FrameSwitchingTest.testShouldReturnWindowTitleInAFrameset', 45 'FrameSwitchingTest.testShouldReturnWindowTitleInAFrameset',
51 'FrameSwitchingTest.testShouldThrowFrameNotFoundExceptionLookingUpSubFramesW ithSuperFrameNames', 46 'FrameSwitchingTest.testShouldThrowFrameNotFoundExceptionLookingUpSubFramesW ithSuperFrameNames',
52 'GetLogsTest.differentLogsShouldNotContainTheSameLogEntries', 47 'GetLogsTest.differentLogsShouldNotContainTheSameLogEntries',
53 'GetLogsTest.logBufferShouldBeResetAfterEachGetLogCall', 48 'GetLogsTest.logBufferShouldBeResetAfterEachGetLogCall',
54 'GetLogsTest.turningOffLogShouldMeanNoLogMessages', 49 'GetLogsTest.turningOffLogShouldMeanNoLogMessages',
55 'I18nTest.testShouldBeAbleToActivateIMEEngine', 50 'I18nTest.testShouldBeAbleToActivateIMEEngine',
56 # Broken because AddWebStorage.java is broken. 51 # Broken because AddWebStorage.java is broken.
57 'LocalStorageTest.*', 52 'LocalStorageTest.*',
58 'LocationContextTest.*', 53 'LocationContextTest.*',
59 'MiscTest.testShouldBeAbleToGetTheSourceOfAnXmlDocument', 54 'MiscTest.testShouldBeAbleToGetTheSourceOfAnXmlDocument',
60 'PageLoadingTest.shouldBeAbleToDisableAcceptOfInsecureSslCertsWithRequiredCa pability', 55 'PageLoadingTest.shouldBeAbleToDisableAcceptOfInsecureSslCertsWithRequiredCa pability',
61 'PageLoadingTest.testShouldNotWaitIndefinitelyIfAnExternalResourceFailsToLoa d', 56 'PageLoadingTest.testShouldNotWaitIndefinitelyIfAnExternalResourceFailsToLoa d',
62 'PageLoadingTest.testShouldTimeoutIfAPageTakesTooLongToLoad', 57 'PageLoadingTest.testShouldTimeoutIfAPageTakesTooLongToLoad',
63 'PerformanceLoggingTest.testDisabledProfilingDoesNotLog', 58 # PerformanceLoggingTest is for firefox only.
59 'PerformanceLoggingTest.*',
64 'ProxySettingTest.canConfigureProxyWithRequiredCapability', 60 'ProxySettingTest.canConfigureProxyWithRequiredCapability',
65 'ProxySettingTest.requiredProxyCapabilityShouldHavePriority', 61 'ProxySettingTest.requiredProxyCapabilityShouldHavePriority',
66 'RenderedWebElementTest.testGetCssValueShouldReturnStandardizedColour',
67 'RenderedWebElementTest.testShouldAllowInheritedStylesToBeUsed',
68 'RenderedWebElementTest.testShouldHandleNonIntegerPositionAndSize', 62 'RenderedWebElementTest.testShouldHandleNonIntegerPositionAndSize',
69 'RenderedWebElementTest.testShouldPickUpStyleOfAnElement',
70 'SessionHandlingTest.callingQuitAfterClosingTheLastWindowIsANoOp', 63 'SessionHandlingTest.callingQuitAfterClosingTheLastWindowIsANoOp',
71 # Broken because AddWebStorage.java is broken. 64 # Broken because AddWebStorage.java is broken.
72 'SessionStorageTest.*', 65 'SessionStorageTest.*',
73 'SqlDatabaseTest.*', 66 'SqlDatabaseTest.*',
74 'TextHandlingTest.testShouldNotReturnLtrMarks', 67 'TextHandlingTest.testShouldNotReturnLtrMarks',
75 'TextHandlingTest.testShouldReturnEmptyStringWhenTagIsSelfClosing',
76 'TextPagesTest.testShouldBeAbleToLoadASimplePageOfText', 68 'TextPagesTest.testShouldBeAbleToLoadASimplePageOfText',
77 'TextPagesTest.testShouldThrowExceptionWhenAddingCookieToAPageThatIsNotHtml' , 69 'TextPagesTest.testShouldThrowExceptionWhenAddingCookieToAPageThatIsNotHtml' ,
78 'TypingTest.testNonPrintableCharactersShouldWorkWithContentEditableOrDesignM odeSet', 70 'TypingTest.testNonPrintableCharactersShouldWorkWithContentEditableOrDesignM odeSet',
79 'TypingTest.testShouldBeAbleToTypeIntoContentEditableElementWithExistingValu e', 71 'TypingTest.testShouldBeAbleToTypeIntoContentEditableElementWithExistingValu e',
80 'TypingTest.testShouldNotTypeIntoElementsThatPreventKeyDownEvents', 72 'TypingTest.testShouldNotTypeIntoElementsThatPreventKeyDownEvents',
81 'TypingTest.testTypingIntoAnIFrameWithContentEditableOrDesignModeSet', 73 'TypingTest.testTypingIntoAnIFrameWithContentEditableOrDesignModeSet',
82 'UnexpectedAlertBehaviorTest.*', 74 'UnexpectedAlertBehaviorTest.*',
83 'VisibilityTest.tooSmallAWindowWithOverflowHiddenIsNotAProblem', 75 'VisibilityTest.tooSmallAWindowWithOverflowHiddenIsNotAProblem',
84 'WebElementTest.testElementReturnsOriginDriver',
85 'WindowTest.*', 76 'WindowTest.*',
86 ] 77 ]
87 _REVISION_NEGATIVE_FILTER['27'] = [] + _REVISION_NEGATIVE_FILTER['HEAD'] 78 _REVISION_NEGATIVE_FILTER['27'] = [] + _REVISION_NEGATIVE_FILTER['HEAD']
88 _REVISION_NEGATIVE_FILTER['26'] = [ 79 _REVISION_NEGATIVE_FILTER['26'] = [
89 'UploadTest.testFileUploading', 80 'UploadTest.testFileUploading',
90 'CorrectEventFiringTest.testUploadingFileShouldFireOnChangeEvent', 81 'CorrectEventFiringTest.testUploadingFileShouldFireOnChangeEvent',
91 'FormHandlingTest.testShouldBeAbleToAlterTheContentsOfAFileUploadInputElemen t', 82 'FormHandlingTest.testShouldBeAbleToAlterTheContentsOfAFileUploadInputElemen t',
92 'FormHandlingTest.testShouldBeAbleToUploadTheSameFileTwice', 83 'FormHandlingTest.testShouldBeAbleToUploadTheSameFileTwice',
93 'FormHandlingTest.testShouldBeAbleToSendKeysToAFileUploadInputElementInAnXht mlDocument', 84 'FormHandlingTest.testShouldBeAbleToSendKeysToAFileUploadInputElementInAnXht mlDocument',
94 ] + _REVISION_NEGATIVE_FILTER['HEAD'] 85 ] + _REVISION_NEGATIVE_FILTER['HEAD']
95 86
96 _OS_NEGATIVE_FILTER = {} 87 _OS_NEGATIVE_FILTER = {}
97 _OS_NEGATIVE_FILTER['win'] = [ 88 _OS_NEGATIVE_FILTER['win'] = [
98 # https://code.google.com/p/chromedriver/issues/detail?id=303
99 'CombinedInputActionsTest.testChordControlCutAndPaste',
100 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=282 89 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=282
101 'PageLoadingTest.testShouldNotHangIfDocumentOpenCallIsNeverFollowedByDocumen tCloseCall', 90 'PageLoadingTest.testShouldNotHangIfDocumentOpenCallIsNeverFollowedByDocumen tCloseCall',
102 ] 91 ]
103 _OS_NEGATIVE_FILTER['linux'] = [ 92 _OS_NEGATIVE_FILTER['linux'] = []
104 # https://code.google.com/p/chromedriver/issues/detail?id=284
105 'TypingTest.testArrowKeysAndPageUpAndDown',
106 'TypingTest.testHomeAndEndAndPageUpAndPageDownKeys',
107 'TypingTest.testNumberpadKeys',
108 ]
109 _OS_NEGATIVE_FILTER['mac'] = [] 93 _OS_NEGATIVE_FILTER['mac'] = []
110 _OS_NEGATIVE_FILTER['android'] = [ 94 _OS_NEGATIVE_FILTER['android'] = [
111 'BasicKeyboardInterfaceTest.*', 95 'BasicKeyboardInterfaceTest.*',
112 'BasicMouseInterfaceTest.*', 96 'BasicMouseInterfaceTest.*',
97 'ChromeOptionsFunctionalTest.canStartChromeWithCustomOptions',
113 'ClickScrollingTest.testShouldBeAbleToClickElementThatIsOutOfViewInANestedFr ameThatIsOutOfView', 98 'ClickScrollingTest.testShouldBeAbleToClickElementThatIsOutOfViewInANestedFr ameThatIsOutOfView',
114 'ClickScrollingTest.testShouldBeAbleToClickElementThatIsOutOfViewInANestedFr ame', 99 'ClickScrollingTest.testShouldBeAbleToClickElementThatIsOutOfViewInANestedFr ame',
115 'ClickTest.testCanClickOnAnElementWithTopSetToANegativeNumber', 100 'ClickTest.testCanClickOnAnElementWithTopSetToANegativeNumber',
116 'ClickTest.testShouldOnlyFollowHrefOnce', 101 'ClickTest.testShouldOnlyFollowHrefOnce',
117 'CombinedInputActionsTest.*', 102 'CombinedInputActionsTest.*',
118 'CoordinatesTest.testShouldGetCoordinatesInViewPortOfAnElementInAFrame', 103 'CoordinatesTest.testShouldGetCoordinatesInViewPortOfAnElementInAFrame',
119 'CoordinatesTest.testShouldGetCoordinatesInViewPortOfAnElementInANestedFrame ', 104 'CoordinatesTest.testShouldGetCoordinatesInViewPortOfAnElementInANestedFrame ',
120 'CoordinatesTest.testShouldGetCoordinatesOfAHiddenElement', 105 'CoordinatesTest.testShouldGetCoordinatesOfAHiddenElement',
121 'CorrectEventFiringTest.testSendingKeysToAFocusedElementShouldNotBlurThatEle ment', 106 'CorrectEventFiringTest.testSendingKeysToAFocusedElementShouldNotBlurThatEle ment',
122 'CorrectEventFiringTest.testSendingKeysToAnElementShouldCauseTheFocusEventTo Fire', 107 'CorrectEventFiringTest.testSendingKeysToAnElementShouldCauseTheFocusEventTo Fire',
(...skipping 74 matching lines...) Expand 10 before | Expand all | Expand 10 after
197 182
198 Returns: 183 Returns:
199 Set of passed test names. 184 Set of passed test names.
200 """ 185 """
201 filters = (_OS_NEGATIVE_FILTER[operating_system] + 186 filters = (_OS_NEGATIVE_FILTER[operating_system] +
202 _REVISION_NEGATIVE_FILTER[chrome_version]) 187 _REVISION_NEGATIVE_FILTER[chrome_version])
203 passed = set(tests) 188 passed = set(tests)
204 for f in filters: 189 for f in filters:
205 passed.difference_update(set(t for t in tests if fnmatch.fnmatch(t, f))) 190 passed.difference_update(set(t for t in tests if fnmatch.fnmatch(t, f)))
206 return passed 191 return passed
OLDNEW
« no previous file with comments | « chrome/test/chromedriver/run_buildbot_steps.py ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698