Monitoring proxy for time series data

appengine/chrome_infra_mon_proxy/common.py

Index: appengine/chrome_infra_mon_proxy/common.py
diff --git a/appengine/chrome_infra_mon_proxy/common.py b/appengine/chrome_infra_mon_proxy/common.py
new file mode 100644
index 0000000000000000000000000000000000000000..5bc79ce6b89d55849cd2f9be6a1ccf6027e4d03e
--- /dev/null
+++ b/appengine/chrome_infra_mon_proxy/common.py
@@ -0,0 +1,55 @@
+# Copyright 2015 The Chromium Authors. All rights reserved.

[ghost stip (do not use), 2015/04/14 00:36:42]

rename to something more meaningful like 'credentials'

[Sergey Berezin (google), 2015/04/15 01:38:09]

It is actually a common set of functions, not just credentials. I'll rename the
key to be more generic, since it's no longer just credentials.

+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+import hashlib
+import logging
+import os
+
+from google.appengine.ext import ndb
+
+
+CREDENTIALS_KEY = 'credentials_key'
+
+
+def is_development_server():

[ghost stip (do not use), 2015/04/14 00:36:42]

there is another implementation of this, meaning it's *not* common :). inline it
back to the place that calls it?

[Sergey Berezin (google), 2015/04/15 01:38:09]

I fixed that, it's now the only (common) implementation. Thanks for catching!

+  return os.environ.get('SERVER_SOFTWARE', '').startswith('Development')
+
+
+class MonAcqData(ndb.Model):

[ghost stip (do not use), 2015/04/14 00:36:42]

MonAcqCredentials

[Sergey Berezin (google), 2015/04/15 01:38:09]

It's not only creds, it's also the URL for the endpoint, the scopes, and the
additional headers. I'm planning to add possibly load percentages for the
modules. So, I think Data is more appropriate.

+  """Store the sensitive endpoint data."""
+  credentials = ndb.JsonProperty()
+  url = ndb.StringProperty()
+  scopes = ndb.StringProperty(repeated=True)
+  headers = ndb.JsonProperty(default={})
+
+
+def get_data():

[ghost stip (do not use), 2015/04/14 00:36:42]

def _get_datastore_credentials()

[Sergey Berezin (google), 2015/04/15 01:38:09]

I'm moving this to vm_module.py, it's the only place it's used.

+  data_entity = MonAcqData.get_by_id(CREDENTIALS_KEY)
+  logging.info('get_data(): entity = %r', data_entity)
+  if not data_entity:
+    return None
+  return data_entity.to_dict()
+
+
+def get_credentials(credentials_dict, scopes):

[ghost stip (do not use), 2015/04/14 00:36:42]

def get_credentials(credentials_dict=None, scopes=None)

[Sergey Berezin (google), 2015/04/16 04:39:06]

Moved to vm_module. I don't see the need for defaults here, since the function
is only used once.

+  """Obtain Aquisition API credentials as Credentials object."""
+  from oauth2client.client import SignedJwtAssertionCredentials
+

[ghost stip (do not use), 2015/04/14 00:36:42]

credentials_dict = credentials_dict or
self._get_datastore_credentials()['credentials']
scopes = scopes or self._get_datastore_credentials()['scopes']

[Sergey Berezin (google), 2015/04/16 04:39:06]

Same as above.

+  # Ideally, we should have loaded credentials with GoogleCredentials.
+  # However, it insists to load only from a file. So, here's a hack.
+  return SignedJwtAssertionCredentials(
+      service_account_name=credentials_dict['client_email'],
+      private_key=credentials_dict['private_key'],
+      scope=scopes,
+      # Extra **kwargs, just in case.
+      service_account_id=credentials_dict['client_id'],
+      private_key_id=credentials_dict['private_key_id'],
+  )
+
+
+def payload_stats(data):
+  md5 = hashlib.md5()
+  md5.update(data)
+  md5hex = md5.hexdigest()
+  return 'type=%s, %d bytes, md5=%s' % (type(data), len(data), md5hex)