|
|
Created:
7 years, 3 months ago by deymo Modified:
7 years, 3 months ago CC:
chromium-reviews, Dirk Pranke, cmp-cc_chromium.org, iannucci+depot_tools_chromium.org, ilevy-cc_chromium.org, szager1 Base URL:
https://chromium.googlesource.com/chromium/tools/depot_tools.git@master Visibility:
Public. |
Descriptionmy_activity.py: Use gerrit new REST API.
This patch makes my_activity.py use the new REST API to access the
list of gerrit changes. The client doesn't authenticate with the
server, so only the public changes are shown.
BUG=chromium:281695
TEST=Ran my_activity.py, shows changes&reviews and links work.
Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=222767
Patch Set 1 #
Total comments: 4
Patch Set 2 : use urlencode #Patch Set 3 : support both interfaces. #
Total comments: 1
Patch Set 4 : added error message when marker not found #
Total comments: 4
Patch Set 5 : review_url uses https. #
Total comments: 1
Patch Set 6 : Added note about https for crosreview.com #
Total comments: 18
Patch Set 7 : maruel comments #
Total comments: 2
Patch Set 8 : maruel nit #Messages
Total messages: 23 (0 generated)
cmp@ for OWNER review. cjhopman@ seems to be the last person who modified the gerrit interface, so please review the patch. Thanks for review.
This feels so much better than the ssh api. It seems that Chrome for Android's projects are not on chrome-review.googlesource.com or chrome-internal-review.googlesource.com. I don't know if gerrit-int.chromium.org supports the rest API, but we shouldn't break them if it doesn't. https://codereview.chromium.org/24047003/diff/1/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/1/my_activity.py#newcode388 my_activity.py:388: args = 'q=-age:%ss+%s&o=LABELS&o=MESSAGES' % (str(max_age), user_filter) use urllib.urlencode to encode parameters if possible https://codereview.chromium.org/24047003/diff/1/my_activity.py#newcode398 my_activity.py:398: open('zaraza', 'w').write(stdout) ?
New patch with the comments included. I don't know about the Chrome for Android workflow, but the ChromeOS is currently broken :). Would you suggest to support both the old and new interfaces on this script? https://codereview.chromium.org/24047003/diff/1/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/1/my_activity.py#newcode388 my_activity.py:388: args = 'q=-age:%ss+%s&o=LABELS&o=MESSAGES' % (str(max_age), user_filter) On 2013/09/09 21:50:05, cjhopman wrote: > use urllib.urlencode to encode parameters if possible Done. https://codereview.chromium.org/24047003/diff/1/my_activity.py#newcode398 my_activity.py:398: open('zaraza', 'w').write(stdout) On 2013/09/09 21:50:05, cjhopman wrote: > ? aaagr... I forgot to remove that debug line. Sorry.
Hi! I changed the CL so it now supports changes from both interfaces. I thinks this should be transitional as I'm not sure what happens with the changes that are mirrored in both gerrit servers. Thanks for reviewing.
lgtm https://codereview.chromium.org/24047003/diff/8001/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/8001/my_activity.py#newcode424 my_activity.py:424: if stdout[0:5] != ")]}'\n": maybe print a short error here too?
Ok, added a new log message as pointed by cjhopman@ cmp@, could you please OWNER review this patch? Thanks!
lgtm with two changes +szager fyi https://codereview.chromium.org/24047003/diff/13001/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/13001/my_activity.py#newcode488 my_activity.py:488: ret['review_url'] = 'http://%s/%s' % (instance['url'], issue['_number']) use https for URLs https://codereview.chromium.org/24047003/diff/13001/my_activity.py#newcode490 my_activity.py:490: ret['review_url'] = 'http://%s/%s' % (instance['shorturl'], use https for URLs
Only changed one of the urls to https. The short url doesn't support https. https://codereview.chromium.org/24047003/diff/13001/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/13001/my_activity.py#newcode488 my_activity.py:488: ret['review_url'] = 'http://%s/%s' % (instance['url'], issue['_number']) On 2013/09/10 22:03:56, cmp wrote: > use https for URLs Done. https://codereview.chromium.org/24047003/diff/13001/my_activity.py#newcode490 my_activity.py:490: ret['review_url'] = 'http://%s/%s' % (instance['shorturl'], On 2013/09/10 22:03:56, cmp wrote: > use https for URLs crosreview.com doesn't support HTTPS. Example: * http://crosreview.com/167913 works * https://crosreview.com/167913 gives me a connection error.
Just FYI, the python module chromite/lib/gob_util.py has a lot of useful boilerplate for interacting with the gerrit servers. I intend to move gob_util.py into depot_tools eventually...
lgtm with nit https://codereview.chromium.org/24047003/diff/2001/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/2001/my_activity.py#newcode490 my_activity.py:490: ret['review_url'] = 'http://%s/%s' % (instance['shorturl'], Can you add a documentation note before line 490 that we should switch this to https once crosreview.com supports that?
lgtm with nit (from my @chromium.org acct)
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/deymo@chromium.org/24047003/3001
Presubmit check for 24047003-3001 failed and returned exit status 1. Running presubmit commit checks ... Checking out rietveld... Running save-description-on-failure.sh Running push-basic.sh Running upstream.sh Running submit-from-new-dir.sh Running abandon.sh Running submodule-merge-test.sh Running upload-local-tracking-branch.sh Running hooks.sh Running post-dcommit-hook-test.sh Running upload-stale.sh Running patch.sh Running basic.sh ** Presubmit ERRORS ** tests/checkout_test.py failed Switched to branch 'master' Already on 'master' .EEE................... ====================================================================== ERROR: testException (__main__.GitCheckout) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/checkout_test.py", line 345, in setUp self.enabled = self.FAKE_REPOS.set_up_git() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 355, in set_up_git self.set_up() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 211, in set_up self.cleanup_dirt() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 229, in cleanup_dirt if not self.tear_down_git(): File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 270, in tear_down_git subprocess2.kill_pid(pid) File "/b/commit-queue/workdir/tools/depot_tools/subprocess2.py", line 63, in kill_pid return os.kill(pid, signal.SIGKILL) OSError: [Errno 3] No such process ====================================================================== ERROR: testMove (__main__.GitCheckout) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/checkout_test.py", line 345, in setUp self.enabled = self.FAKE_REPOS.set_up_git() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 358, in set_up_git assert self.git_pid_file == None AssertionError ====================================================================== ERROR: testProcess (__main__.GitCheckout) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/checkout_test.py", line 345, in setUp self.enabled = self.FAKE_REPOS.set_up_git() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 358, in set_up_git assert self.git_pid_file == None AssertionError ---------------------------------------------------------------------- Ran 23 tests in 7.521s FAILED (errors=3) Presubmit checks took 89.3s to calculate.
The CQ failure looks totally unrelated, a few notes on the CL. https://codereview.chromium.org/24047003/diff/3001/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode400 my_activity.py:400: [stdout, _] = subprocess.Popen(gquery_cmd, stdout=subprocess.PIPE, (stdout, _) https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode401 my_activity.py:401: stderr=subprocess.PIPE).communicate() Failure will be hard to handle since you discard both the exit code and stderr. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode402 my_activity.py:402: issues = str(stdout).split('\n')[:-2] splitlines() stdout is already a str(), not need to convert explicitly. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode423 my_activity.py:423: # Check that the returned JSON starts with the right marker. s/starts/ends/ https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode427 my_activity.py:427: return json.loads(stdout[5:]) What are you skipping exactly? https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode445 my_activity.py:445: raise Exception("Invalid gerrit_instances configuration.") resto f the file use single quotes, stay consistent. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode490 my_activity.py:490: # TODO: Move this short link to https once crosreview.com supports it. TODO(deymo) https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode502 my_activity.py:502: ret['reviewers'] = set() (optional) Could be done as a one liner: ret['reviewers'] = set( r['author'] for r in ret['replies'] if r['author'] != ret['author']) or: ret['reviewers'] = set(r['author'] for r in ret['replies']) ret['reviewers'].discard(ret['author']) https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode513 my_activity.py:513: r = {} ret.append( { 'author': reply['author']['email'], ... }) or better and shorter: return [ { 'author': reply['author']['email'], ... } for reply in replies ]
I addressed most of maruel comments. I must point out that the diff here might be a bit confused about what code really changed and what was just moved. Some of the comments were about pre-existing I'm just copying. maruel, can you take another look? https://codereview.chromium.org/24047003/diff/3001/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode400 my_activity.py:400: [stdout, _] = subprocess.Popen(gquery_cmd, stdout=subprocess.PIPE, On 2013/09/11 13:47:26, M-A Ruel wrote: > (stdout, _) Done. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode401 my_activity.py:401: stderr=subprocess.PIPE).communicate() On 2013/09/11 13:47:26, M-A Ruel wrote: > Failure will be hard to handle since you discard both the exit code and stderr. This part of the code is using the old gerrit interface and was like that on the codebase. I think the idea is to ignore errors on my_activity.py (for example, if you don't use gerrit and don't have permission or a user to access it). I'd keep this as it is. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode402 my_activity.py:402: issues = str(stdout).split('\n')[:-2] On 2013/09/11 13:47:26, M-A Ruel wrote: > splitlines() > > stdout is already a str(), not need to convert explicitly. Done. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode423 my_activity.py:423: # Check that the returned JSON starts with the right marker. On 2013/09/11 13:47:26, M-A Ruel wrote: > s/starts/ends/ hmmm... I think that "starts" is the right option here. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode427 my_activity.py:427: return json.loads(stdout[5:]) On 2013/09/11 13:47:26, M-A Ruel wrote: > What are you skipping exactly? I'm skipping the string ")]}'\n" from the beginning of the HTTP response and checking that the skipped part is what I'm expecting to be (in the if statement right before this return). The REST API responds with this 5 chars and then a totally valid JSON string. I don't know why, but that's what the examples on the documentation show (and what the server is actually sending). https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode445 my_activity.py:445: raise Exception("Invalid gerrit_instances configuration.") On 2013/09/11 13:47:26, M-A Ruel wrote: > resto f the file use single quotes, stay consistent. Done. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode490 my_activity.py:490: # TODO: Move this short link to https once crosreview.com supports it. On 2013/09/11 13:47:26, M-A Ruel wrote: > TODO(deymo) Done. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode502 my_activity.py:502: ret['reviewers'] = set() Changed in both functions. On 2013/09/11 13:47:26, M-A Ruel wrote: > (optional) Could be done as a one liner: > ret['reviewers'] = set( > r['author'] for r in ret['replies'] if r['author'] != ret['author']) > > or: > ret['reviewers'] = set(r['author'] for r in ret['replies']) > ret['reviewers'].discard(ret['author']) Done. https://codereview.chromium.org/24047003/diff/3001/my_activity.py#newcode513 my_activity.py:513: r = {} This code is essentially the same as the previously existing process_gerrit_issue_replies. I replaced both, the new and the old function. On 2013/09/11 13:47:26, M-A Ruel wrote: > ret.append( > { > 'author': reply['author']['email'], > ... > }) > > or better and shorter: > return [ > { > 'author': reply['author']['email'], > ... > } > for reply in replies > ]
lgtm https://codereview.chromium.org/24047003/diff/25001/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/25001/my_activity.py#newcode440 my_activity.py:440: for issue in issues] style nit: I prefer everything at +4 or everything at ( or [ in this case.
https://codereview.chromium.org/24047003/diff/25001/my_activity.py File my_activity.py (right): https://codereview.chromium.org/24047003/diff/25001/my_activity.py#newcode440 my_activity.py:440: for issue in issues] On 2013/09/12 00:15:30, M-A Ruel wrote: > style nit: I prefer everything at +4 or everything at ( or [ in this case. Done.
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/deymo@chromium.org/24047003/30001
Presubmit check for 24047003-30001 failed and returned exit status 1. Running presubmit commit checks ... Checking out rietveld... Running save-description-on-failure.sh Running push-basic.sh Running upstream.sh Running submit-from-new-dir.sh Running abandon.sh Running submodule-merge-test.sh Running upload-local-tracking-branch.sh Running hooks.sh Running post-dcommit-hook-test.sh Running upload-stale.sh Running patch.sh Running basic.sh ** Presubmit ERRORS ** tests/checkout_test.py failed Switched to branch 'master' Already on 'master' .EEE................... ====================================================================== ERROR: testException (__main__.GitCheckout) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/checkout_test.py", line 345, in setUp self.enabled = self.FAKE_REPOS.set_up_git() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 355, in set_up_git self.set_up() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 211, in set_up self.cleanup_dirt() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 229, in cleanup_dirt if not self.tear_down_git(): File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 270, in tear_down_git subprocess2.kill_pid(pid) File "/b/commit-queue/workdir/tools/depot_tools/subprocess2.py", line 63, in kill_pid return os.kill(pid, signal.SIGKILL) OSError: [Errno 3] No such process ====================================================================== ERROR: testMove (__main__.GitCheckout) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/checkout_test.py", line 345, in setUp self.enabled = self.FAKE_REPOS.set_up_git() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 358, in set_up_git assert self.git_pid_file == None AssertionError ====================================================================== ERROR: testProcess (__main__.GitCheckout) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/checkout_test.py", line 345, in setUp self.enabled = self.FAKE_REPOS.set_up_git() File "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line 358, in set_up_git assert self.git_pid_file == None AssertionError ---------------------------------------------------------------------- Ran 23 tests in 8.269s FAILED (errors=3) Presubmit checks took 89.7s to calculate. Was the presubmit check useful? Please send feedback & hate mail to maruel@chromium.org!
On 2013/09/12 00:37:27, I haz the power (commit-bot) wrote: > Presubmit check for 24047003-30001 failed and returned exit status 1. > > Running presubmit commit checks ... > Checking out rietveld... > Running save-description-on-failure.sh > Running push-basic.sh > Running upstream.sh > Running submit-from-new-dir.sh > Running abandon.sh > Running submodule-merge-test.sh > Running upload-local-tracking-branch.sh > Running hooks.sh > Running post-dcommit-hook-test.sh > Running upload-stale.sh > Running patch.sh > Running basic.sh > > ** Presubmit ERRORS ** > tests/checkout_test.py failed > Switched to branch 'master' > Already on 'master' > .EEE................... > ====================================================================== > ERROR: testException (__main__.GitCheckout) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "tests/checkout_test.py", line 345, in setUp > self.enabled = self.FAKE_REPOS.set_up_git() > File > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line > 355, in set_up_git > self.set_up() > File > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line > 211, in set_up > self.cleanup_dirt() > File > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line > 229, in cleanup_dirt > if not self.tear_down_git(): > File > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line > 270, in tear_down_git > subprocess2.kill_pid(pid) > File "/b/commit-queue/workdir/tools/depot_tools/subprocess2.py", line 63, in > kill_pid > return os.kill(pid, signal.SIGKILL) > OSError: [Errno 3] No such process > > ====================================================================== > ERROR: testMove (__main__.GitCheckout) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "tests/checkout_test.py", line 345, in setUp > self.enabled = self.FAKE_REPOS.set_up_git() > File > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line > 358, in set_up_git > assert self.git_pid_file == None > AssertionError > > ====================================================================== > ERROR: testProcess (__main__.GitCheckout) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "tests/checkout_test.py", line 345, in setUp > self.enabled = self.FAKE_REPOS.set_up_git() > File > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", line > 358, in set_up_git > assert self.git_pid_file == None > AssertionError > > ---------------------------------------------------------------------- > Ran 23 tests in 8.269s > > FAILED (errors=3) > > > Presubmit checks took 89.7s to calculate. > > Was the presubmit check useful? Please send feedback & hate mail to > maruel@chromium.org! I suspect a recent depot_tools breakage. Ravi, does this sounds a bell?
On 2013/09/12 01:04:19, M-A Ruel wrote: > On 2013/09/12 00:37:27, I haz the power (commit-bot) wrote: > > Presubmit check for 24047003-30001 failed and returned exit status 1. > > > > Running presubmit commit checks ... > > Checking out rietveld... > > Running save-description-on-failure.sh > > Running push-basic.sh > > Running upstream.sh > > Running submit-from-new-dir.sh > > Running abandon.sh > > Running submodule-merge-test.sh > > Running upload-local-tracking-branch.sh > > Running hooks.sh > > Running post-dcommit-hook-test.sh > > Running upload-stale.sh > > Running patch.sh > > Running basic.sh > > > > ** Presubmit ERRORS ** > > tests/checkout_test.py failed > > Switched to branch 'master' > > Already on 'master' > > .EEE................... > > ====================================================================== > > ERROR: testException (__main__.GitCheckout) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "tests/checkout_test.py", line 345, in setUp > > self.enabled = self.FAKE_REPOS.set_up_git() > > File > > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", > line > > 355, in set_up_git > > self.set_up() > > File > > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", > line > > 211, in set_up > > self.cleanup_dirt() > > File > > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", > line > > 229, in cleanup_dirt > > if not self.tear_down_git(): > > File > > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", > line > > 270, in tear_down_git > > subprocess2.kill_pid(pid) > > File "/b/commit-queue/workdir/tools/depot_tools/subprocess2.py", line 63, in > > kill_pid > > return os.kill(pid, signal.SIGKILL) > > OSError: [Errno 3] No such process > > > > ====================================================================== > > ERROR: testMove (__main__.GitCheckout) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "tests/checkout_test.py", line 345, in setUp > > self.enabled = self.FAKE_REPOS.set_up_git() > > File > > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", > line > > 358, in set_up_git > > assert self.git_pid_file == None > > AssertionError > > > > ====================================================================== > > ERROR: testProcess (__main__.GitCheckout) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "tests/checkout_test.py", line 345, in setUp > > self.enabled = self.FAKE_REPOS.set_up_git() > > File > > "/b/commit-queue/workdir/tools/depot_tools/testing_support/fake_repos.py", > line > > 358, in set_up_git > > assert self.git_pid_file == None > > AssertionError > > > > ---------------------------------------------------------------------- > > Ran 23 tests in 8.269s > > > > FAILED (errors=3) > > > > > > Presubmit checks took 89.7s to calculate. > > > > Was the presubmit check useful? Please send feedback & hate mail to > > maruel@chromium.org! > > I suspect a recent depot_tools breakage. Ravi, does this sounds a bell? These are the tests I added as part of https://codereview.chromium.org/22794015/ they worked in the initial checkin but something is obviously wrong now. I will send out a CL tomorrow morning to disable the tests and then will investigate. Sorry about the breakage.
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/deymo@chromium.org/24047003/30001
Message was sent while issue was closed.
Change committed as 222767 |