Adding pagesets for gmail and plus. Creating endure benchmark

Adding pagesets for gmail and plus.  Creating endure benchmark

** Will need to be updated/rebased when https://codereview.chromium.org/22883011/ lands 
BUG=273827
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=220443

[edmundyan, 2013-08-26 21:31:54]

I've migrated over what chrome endure scenarios I could into Telemetry pagesets
and have blockers on the last two:

testGmailComposeDiscard - Can't click 'Compose', also can't do typing
testDocsAlternatelyClickLists - Can't click 'My Drive'

Non-app related scenarios were ignored.  Dennis, any opinion on whether we
should spend time on those too?

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
File tools/perf/page_sets/gmail_expand_collapse_conversation.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
tools/perf/page_sets/gmail_expand_collapse_conversation.json:8: "url":
"https://mail.google.com/mail/u/0/#inbox/13c6a141fa95ffe0",
How do we ensure we enter a long conversation mail?

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
tools/perf/page_sets/gmail_expand_collapse_conversation.json:14: { "action":
"wait", "condition": "duration", "seconds": 10 }
Need to ensure that all expanded emails have their contents downloaded (this is
more for WPR to correctly record).  There should be a more precise way to know
when all downloading is complete.

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
File tools/perf/page_sets/plus_alt_posts_photos.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
tools/perf/page_sets/plus_alt_posts_photos.json:19: { "action": "wait",
"condition": "duration", "seconds": 5 },
Also a similar issue where we need a bit more time to ensure photo thumbnails
were fully downloaded.  This pageset, in general, seemed a bit flaky, thus the
added waits.

[tonyg, 2013-08-27 01:23:02]

everything else lg2m

https://codereview.chromium.org/23465003/diff/1/tools/perf/benchmarks/endure.py
File tools/perf/benchmarks/endure.py (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/benchmarks/endure....
tools/perf/benchmarks/endure.py:14: 'output_format': 'csv',
Why are we using csv for these?

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
File tools/perf/page_sets/gmail_alt_threadlist_conversation.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_threadlist_conversation.json:21: { "action":
"wait", "condition": "duration", "seconds": 1 },
I thought we got rid of condition: duration

[edmundyan, 2013-08-27 03:24:50]

CSV is the temporary output for chrome endure as we figure out how to get the
graphs we want onto the new perf dashboard.  In the meantime, we are outputting
to CSV and then modifying the data to fit with the old endure graphing system.

Yes, "duration" is deprecated.  I'm waiting on the CL to land before rebasing
this branch.

[tonyg, 2013-08-27 03:43:12]

On 2013/08/27 03:24:50, edmundyan wrote:
> CSV is the temporary output for chrome endure as we figure out how to get the
> graphs we want onto the new perf dashboard.  In the meantime, we are
outputting
> to CSV and then modifying the data to fit with the old endure graphing system.
> 
> Yes, "duration" is deprecated.  I'm waiting on the CL to land before rebasing
> this branch.

lgtm then (after rebase)

[dennis_jeffrey, 2013-08-27 20:02:35]

Mostly nits here.

https://codereview.chromium.org/23465003/diff/1/tools/perf/benchmarks/endure.py
File tools/perf/benchmarks/endure.py (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/benchmarks/endure....
tools/perf/benchmarks/endure.py:28: 
add 1 more blank line here

https://codereview.chromium.org/23465003/diff/1/tools/perf/benchmarks/endure....
tools/perf/benchmarks/endure.py:37: 
add 1 more blank line here

https://codereview.chromium.org/23465003/diff/1/tools/perf/benchmarks/endure....
tools/perf/benchmarks/endure.py:46: 
add 1 more blank line here

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
File tools/perf/page_sets/gmail_alt_threadlist_conversation.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_threadlist_conversation.json:2: "description":
"Chrome Endure test for GMail.",
is there a length limit on the description?  Maybe we should add a phrase here
to indicate what scenario is performed, e.g., "Gmail endurance test that
alternates between the threadlist and conversation views."

If you make this change, do the same for the other JSON files in this CL

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_threadlist_conversation.json:13: { "action":
"wait", "condition": "javascript", "javascript": "window.gmonkey !== undefined
&& document.getElementById('gb') !== null"}
add space before the closing } to keep style consistent in this file

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_threadlist_conversation.json:26: { "action":
"wait", "condition": "href_change"},
add space before the closing } to keep style consistent in this file

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
File tools/perf/page_sets/gmail_alt_two_labels.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_two_labels.json:13: { "action": "wait",
"condition": "javascript", "javascript": "window.gmonkey !== undefined &&
document.getElementById('gb') !== null"}
add space before the closing }

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_two_labels.json:26: { "action": "wait",
"condition": "href_change"},
add space before the closing }

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
File tools/perf/page_sets/gmail_expand_collapse_conversation.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
tools/perf/page_sets/gmail_expand_collapse_conversation.json:8: "url":
"https://mail.google.com/mail/u/0/#inbox/13c6a141fa95ffe0",
On 2013/08/26 21:31:55, edmundyan wrote:
> How do we ensure we enter a long conversation mail?

Right now we have to ensure the account is set up in such a way that a long
conversation e-mails opens up (and the e-mails in the account don't change from
run to run).  WebPageReplay can help when we're replaying things, but in
recording mode, we still have to ensure a long conversation e-mail is used.  In
the future we should think about how to better ensure this.

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
File tools/perf/page_sets/plus_alt_posts_photos.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
tools/perf/page_sets/plus_alt_posts_photos.json:13: { "action": "wait",
"condition": "element", "text": "Barack Obama"},
add space before the closing }

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
tools/perf/page_sets/plus_alt_posts_photos.json:14: { "action": "wait",
"condition": "element", "selector":
"span[guidedhelpid=\"posts_tab_profile\"][class*=\"s6U8x\"]"}
add space before the closing }

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
tools/perf/page_sets/plus_alt_posts_photos.json:17: { "action": "click_element",
"selector": "span[guidedhelpid=\"posts_tab_profile\"]"},
add space before the closing }

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
tools/perf/page_sets/plus_alt_posts_photos.json:18: { "action": "wait",
"condition": "element", "selector":
"span[guidedhelpid=\"posts_tab_profile\"][class*=\"s6U8x\"]"},
add space before the closing }

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
tools/perf/page_sets/plus_alt_posts_photos.json:20: { "action": "click_element",
"selector": "span[guidedhelpid=\"photos_tab_profile\"]"},
add space before the closing }

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
tools/perf/page_sets/plus_alt_posts_photos.json:21: { "action": "wait",
"condition": "element", "selector":
"span[guidedhelpid=\"photos_tab_profile\"][class*=\"s6U8x\"]"},
add space before the closing }

[edmundyan, 2013-08-27 23:01:48]

https://codereview.chromium.org/23465003/diff/1/tools/perf/benchmarks/endure.py
File tools/perf/benchmarks/endure.py (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/benchmarks/endure....
tools/perf/benchmarks/endure.py:28: 
On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> add 1 more blank line here

Done.

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
File tools/perf/page_sets/gmail_alt_threadlist_conversation.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_threadlist_conversation.json:2: "description":
"Chrome Endure test for GMail.",
On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> is there a length limit on the description?  Maybe we should add a phrase here
> to indicate what scenario is performed, e.g., "Gmail endurance test that
> alternates between the threadlist and conversation views."
> 
> If you make this change, do the same for the other JSON files in this CL

I've been putting more detailed descriptions in the "why" for the page (below). 
Should I move that up to description instead?

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_threadlist_conversation.json:13: { "action":
"wait", "condition": "javascript", "javascript": "window.gmonkey !== undefined
&& document.getElementById('gb') !== null"}
On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> add space before the closing } to keep style consistent in this file

Done.

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
File tools/perf/page_sets/gmail_alt_two_labels.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_two_labels.json:13: { "action": "wait",
"condition": "javascript", "javascript": "window.gmonkey !== undefined &&
document.getElementById('gb') !== null"}
On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> add space before the closing }

Done.

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_two_labels.json:26: { "action": "wait",
"condition": "href_change"},
On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> add space before the closing }

Done.

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
File tools/perf/page_sets/gmail_expand_collapse_conversation.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
tools/perf/page_sets/gmail_expand_collapse_conversation.json:8: "url":
"https://mail.google.com/mail/u/0/#inbox/13c6a141fa95ffe0",
On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> On 2013/08/26 21:31:55, edmundyan wrote:
> > How do we ensure we enter a long conversation mail?
> 
> Right now we have to ensure the account is set up in such a way that a long
> conversation e-mails opens up (and the e-mails in the account don't change
from
> run to run).  WebPageReplay can help when we're replaying things, but in
> recording mode, we still have to ensure a long conversation e-mail is used. 
In
> the future we should think about how to better ensure this.

So is the explicit URL acceptable?  This won't work for other accounts.  The old
chrome endure test just clicked the 1st email in the inbox and hoped it would
have replies.

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
File tools/perf/page_sets/plus_alt_posts_photos.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/plus_alt...
tools/perf/page_sets/plus_alt_posts_photos.json:13: { "action": "wait",
"condition": "element", "text": "Barack Obama"},
On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> add space before the closing }

Done.

[dennis_jeffrey, 2013-08-27 23:17:22]

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
File tools/perf/page_sets/gmail_alt_threadlist_conversation.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_al...
tools/perf/page_sets/gmail_alt_threadlist_conversation.json:2: "description":
"Chrome Endure test for GMail.",
On 2013/08/27 23:01:48, edmundyan wrote:
> On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> > is there a length limit on the description?  Maybe we should add a phrase
here
> > to indicate what scenario is performed, e.g., "Gmail endurance test that
> > alternates between the threadlist and conversation views."
> > 
> > If you make this change, do the same for the other JSON files in this CL
> 
> I've been putting more detailed descriptions in the "why" for the page
(below). 
> Should I move that up to description instead?

Oh, didn't realize that.  Maybe it's ok as is.  Currently, we need a separate
pageset file for each endure scenario, but in the future if that's no longer the
case, then the pageset "description" should probably be something more general
anyway.

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
File tools/perf/page_sets/gmail_expand_collapse_conversation.json (right):

https://codereview.chromium.org/23465003/diff/1/tools/perf/page_sets/gmail_ex...
tools/perf/page_sets/gmail_expand_collapse_conversation.json:8: "url":
"https://mail.google.com/mail/u/0/#inbox/13c6a141fa95ffe0",
On 2013/08/27 23:01:48, edmundyan wrote:
> On 2013/08/27 20:02:35, dennis_jeffrey wrote:
> > On 2013/08/26 21:31:55, edmundyan wrote:
> > > How do we ensure we enter a long conversation mail?
> > 
> > Right now we have to ensure the account is set up in such a way that a long
> > conversation e-mails opens up (and the e-mails in the account don't change
> from
> > run to run).  WebPageReplay can help when we're replaying things, but in
> > recording mode, we still have to ensure a long conversation e-mail is used. 
> In
> > the future we should think about how to better ensure this.
> 
> So is the explicit URL acceptable?  This won't work for other accounts.  The
old
> chrome endure test just clicked the 1st email in the inbox and hoped it would
> have replies.

I'd say it's acceptable with a TODO added to the code that we should implement a
better approach.

[commit-bot: I haz the power, 2013-08-29 18:10:55]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/edmundyan@chromium.org/23465003/36001

[commit-bot: I haz the power, 2013-08-29 18:17:57]

Step "update" is always a major failure.
Look at the try server FAQ for more details.
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_cl...

[commit-bot: I haz the power, 2013-08-29 20:29:54]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/edmundyan@chromium.org/23465003/36001

[commit-bot: I haz the power, 2013-08-30 00:04:46]

Change committed as 220443