Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(92)

Side by Side Diff: chrome/test/chromedriver/test_expectations

Issue 15994017: [chromedriver] Disable flaky tests. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Created 7 years, 6 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright (c) 2013 The Chromium Authors. All rights reserved. 1 # Copyright (c) 2013 The Chromium Authors. All rights reserved.
2 # Use of this source code is governed by a BSD-style license that can be 2 # Use of this source code is governed by a BSD-style license that can be
3 # found in the LICENSE file. 3 # found in the LICENSE file.
4 4
5 """Test expectation list for WebDriver Java acceptance tests. 5 """Test expectation list for WebDriver Java acceptance tests.
6 6
7 It is evaluated through Python. 7 It is evaluated through Python.
8 """ 8 """
9 9
10 import fnmatch 10 import fnmatch
(...skipping 19 matching lines...) Expand all
30 'ClickScrollingTest.testShouldBeAbleToClickOnAnElementHiddenByDoubleOverflow ', 30 'ClickScrollingTest.testShouldBeAbleToClickOnAnElementHiddenByDoubleOverflow ',
31 'ClickScrollingTest.testShouldNotBeAbleToClickElementThatIsOutOfViewInANonSc rollableFrame', 31 'ClickScrollingTest.testShouldNotBeAbleToClickElementThatIsOutOfViewInANonSc rollableFrame',
32 'ClickScrollingTest.testShouldNotScrollIfAlreadyScrolledAndElementIsInView', 32 'ClickScrollingTest.testShouldNotScrollIfAlreadyScrolledAndElementIsInView',
33 'ClickTest.testCanClickAnImageMapArea', 33 'ClickTest.testCanClickAnImageMapArea',
34 'ClickTest.testShouldBeAbleToClickOnAnElementInFrameGreaterThanTwoViewports' , 34 'ClickTest.testShouldBeAbleToClickOnAnElementInFrameGreaterThanTwoViewports' ,
35 'CombinedInputActionsTest.testClickingOnFormElements', 35 'CombinedInputActionsTest.testClickingOnFormElements',
36 'CoordinatesTest.testShouldScrollPageAndGetCoordinatesOfAnElementThatIsOutOf ViewPort', 36 'CoordinatesTest.testShouldScrollPageAndGetCoordinatesOfAnElementThatIsOutOf ViewPort',
37 'CorrectEventFiringTest.testShouldFireEventsInTheRightOrder', 37 'CorrectEventFiringTest.testShouldFireEventsInTheRightOrder',
38 'CorrectEventFiringTest.testShouldFireFocusEventWhenClicking', 38 'CorrectEventFiringTest.testShouldFireFocusEventWhenClicking',
39 'DragAndDropTest.testDragTooFar', 39 'DragAndDropTest.testDragTooFar',
40 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=374
41 'ElementFindingTest.testShouldNotBeAbleToFindAnElementOnABlankPage',
40 'ExecutingAsyncJavascriptTest.includesAlertTextInUnhandledAlertException', 42 'ExecutingAsyncJavascriptTest.includesAlertTextInUnhandledAlertException',
41 'ExecutingAsyncJavascriptTest.shouldCatchErrorsWithMessageAndStacktraceWhenE xecutingInitialScript', 43 'ExecutingAsyncJavascriptTest.shouldCatchErrorsWithMessageAndStacktraceWhenE xecutingInitialScript',
42 'ExecutingAsyncJavascriptTest.throwsIfAlertHappensDuringScript', 44 'ExecutingAsyncJavascriptTest.throwsIfAlertHappensDuringScript',
43 'ExecutingAsyncJavascriptTest.throwsIfAlertHappensDuringScriptWhichTimesOut' , 45 'ExecutingAsyncJavascriptTest.throwsIfAlertHappensDuringScriptWhichTimesOut' ,
44 'ExecutingAsyncJavascriptTest.throwsIfScriptTriggersAlert', 46 'ExecutingAsyncJavascriptTest.throwsIfScriptTriggersAlert',
45 'ExecutingAsyncJavascriptTest.throwsIfScriptTriggersAlertWhichTimesOut', 47 'ExecutingAsyncJavascriptTest.throwsIfScriptTriggersAlertWhichTimesOut',
46 'ExecutingJavascriptTest.testShouldThrowExceptionIfExecutingOnNoPage', 48 'ExecutingJavascriptTest.testShouldThrowExceptionIfExecutingOnNoPage',
47 'ExecutingJavascriptTest.testShouldThrowAnExceptionWithMessageAndStacktraceW henTheJavascriptIsBad', 49 'ExecutingJavascriptTest.testShouldThrowAnExceptionWithMessageAndStacktraceW henTheJavascriptIsBad',
48 'FormHandlingTest.testShouldNotBeAbleToSubmitAFormThatDoesNotExist', 50 'FormHandlingTest.testShouldNotBeAbleToSubmitAFormThatDoesNotExist',
49 'FrameSwitchingTest.testShouldNotBeAbleToDoAnythingTheFrameIsDeletedFromUnde rUs', 51 'FrameSwitchingTest.testShouldNotBeAbleToDoAnythingTheFrameIsDeletedFromUnde rUs',
(...skipping 36 matching lines...) Expand 10 before | Expand all | Expand 10 after
86 'VisibilityTest.tooSmallAWindowWithOverflowHiddenIsNotAProblem', 88 'VisibilityTest.tooSmallAWindowWithOverflowHiddenIsNotAProblem',
87 'WindowTest.*', 89 'WindowTest.*',
88 ] 90 ]
89 _REVISION_NEGATIVE_FILTER['28'] = [] + _REVISION_NEGATIVE_FILTER['HEAD'] 91 _REVISION_NEGATIVE_FILTER['28'] = [] + _REVISION_NEGATIVE_FILTER['HEAD']
90 _REVISION_NEGATIVE_FILTER['27'] = [] + _REVISION_NEGATIVE_FILTER['HEAD'] 92 _REVISION_NEGATIVE_FILTER['27'] = [] + _REVISION_NEGATIVE_FILTER['HEAD']
91 93
92 _OS_NEGATIVE_FILTER = {} 94 _OS_NEGATIVE_FILTER = {}
93 _OS_NEGATIVE_FILTER['win'] = [ 95 _OS_NEGATIVE_FILTER['win'] = [
94 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=282 96 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=282
95 'PageLoadingTest.testShouldNotHangIfDocumentOpenCallIsNeverFollowedByDocumen tCloseCall', 97 'PageLoadingTest.testShouldNotHangIfDocumentOpenCallIsNeverFollowedByDocumen tCloseCall',
98 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=373
99 'RenderedWebElementTest.testHoverPersists',
96 ] 100 ]
97 _OS_NEGATIVE_FILTER['linux'] = [] 101 _OS_NEGATIVE_FILTER['linux'] = []
98 _OS_NEGATIVE_FILTER['mac'] = [ 102 _OS_NEGATIVE_FILTER['mac'] = [
99 # https://code.google.com/p/chromedriver/issues/detail?id=26 103 # https://code.google.com/p/chromedriver/issues/detail?id=26
100 'AlertsTest.testAlertShouldNotAllowAdditionalCommandsIfDismissed', 104 'AlertsTest.testAlertShouldNotAllowAdditionalCommandsIfDismissed',
101 'AlertsTest.testShouldAllowUsersToDismissAnAlertManually', 105 'AlertsTest.testShouldAllowUsersToDismissAnAlertManually',
102 'FormHandlingTest.handleFormWithJavascriptAction', 106 'FormHandlingTest.handleFormWithJavascriptAction',
103 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=354 107 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=354
104 'AlertsTest.testShouldAllowUsersToAcceptAnAlertInAFrame', 108 'AlertsTest.testShouldAllowUsersToAcceptAnAlertInAFrame',
109 # Flaky: https://code.google.com/p/chromedriver/issues/detail?id=375
110 'PageLoadingTest.testShouldBeAbleToNavigateBackInTheBrowserHistoryInPresence OfIframes',
105 ] 111 ]
106 _OS_NEGATIVE_FILTER['android'] = [ 112 _OS_NEGATIVE_FILTER['android'] = [
107 'AlertsTest.*', 113 'AlertsTest.*',
108 'BasicKeyboardInterfaceTest.testBasicKeyboardInput', 114 'BasicKeyboardInterfaceTest.testBasicKeyboardInput',
109 'BasicKeyboardInterfaceTest.testBasicKeyboardInputOnActiveElement', 115 'BasicKeyboardInterfaceTest.testBasicKeyboardInputOnActiveElement',
110 'BasicKeyboardInterfaceTest.testSendingKeyDownOnly', 116 'BasicKeyboardInterfaceTest.testSendingKeyDownOnly',
111 'BasicKeyboardInterfaceTest.testSendingKeysToActiveElement', 117 'BasicKeyboardInterfaceTest.testSendingKeysToActiveElement',
112 'BasicKeyboardInterfaceTest.testSendingKeysWithShiftPressed', 118 'BasicKeyboardInterfaceTest.testSendingKeysWithShiftPressed',
113 'BasicKeyboardInterfaceTest.testSendingKeyUp', 119 'BasicKeyboardInterfaceTest.testSendingKeyUp',
114 'BasicMouseInterfaceTest.testCannotMoveToANullLocator', 120 'BasicMouseInterfaceTest.testCannotMoveToANullLocator',
(...skipping 108 matching lines...) Expand 10 before | Expand all | Expand 10 after
223 229
224 Returns: 230 Returns:
225 Set of passed test names. 231 Set of passed test names.
226 """ 232 """
227 filters = (_OS_NEGATIVE_FILTER[operating_system] + 233 filters = (_OS_NEGATIVE_FILTER[operating_system] +
228 _REVISION_NEGATIVE_FILTER[chrome_version]) 234 _REVISION_NEGATIVE_FILTER[chrome_version])
229 passed = set(tests) 235 passed = set(tests)
230 for f in filters: 236 for f in filters:
231 passed.difference_update(set(t for t in tests if fnmatch.fnmatch(t, f))) 237 passed.difference_update(set(t for t in tests if fnmatch.fnmatch(t, f)))
232 return passed 238 return passed
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698