Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(186)

Unified Diff: tools/telemetry/telemetry/page/cloud_storage.py

Issue 21684002: [telemetry] Automatic hash file creation and upload to Cloud Storage. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Add to unit tests. Created 7 years, 5 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « tools/perf/page_sets/PRESUBMIT.py ('k') | tools/telemetry/telemetry/page/page_set_archive_info.py » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: tools/telemetry/telemetry/page/cloud_storage.py
diff --git a/tools/telemetry/telemetry/page/cloud_storage.py b/tools/telemetry/telemetry/page/cloud_storage.py
index 2ed781db0163204f07558b9dc10cfb3c2f629000..3d4d2e5fb267b68af95fb3488ab1fb95ed979b82 100644
--- a/tools/telemetry/telemetry/page/cloud_storage.py
+++ b/tools/telemetry/telemetry/page/cloud_storage.py
@@ -5,6 +5,7 @@
"""Wrappers for gsutil, for basic interaction with Google Cloud Storage."""
import cStringIO
+import hashlib
import logging
import os
import subprocess
@@ -19,6 +20,10 @@ _GSUTIL_URL = 'http://storage.googleapis.com/pub/gsutil.tar.gz'
_DOWNLOAD_PATH = os.path.join(util.GetTelemetryDir(), 'third_party', 'gsutil')
+class CloudStorageError(Exception):
+ pass
+
+
def _DownloadGsutil():
logging.info('Downloading gsutil')
response = urllib2.urlopen(_GSUTIL_URL)
@@ -56,7 +61,7 @@ def _RunCommand(args):
stdout, stderr = gsutil.communicate()
if gsutil.returncode:
- raise Exception(stderr.splitlines()[-1])
+ raise CloudStorageError(stderr.splitlines()[-1])
return stdout
@@ -82,3 +87,15 @@ def Insert(bucket, remote_path, local_path):
url = 'gs://%s/%s' % (bucket, remote_path)
logging.debug('Uploading %s to %s' % (local_path, url))
_RunCommand(['cp', local_path, url])
+
+
+def GetHash(file_path):
+ sha1 = hashlib.sha1()
+ with open(file_path, 'rb') as f:
+ while True:
+ # Read in 1mb chunks, so it doesn't all have to be loaded into memory.
+ chunk = f.read(1024*1024)
+ if not chunk:
+ break
+ sha1.update(chunk)
+ return sha1.hexdigest()
« no previous file with comments | « tools/perf/page_sets/PRESUBMIT.py ('k') | tools/telemetry/telemetry/page/page_set_archive_info.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698