Add meta package to datastore.

service/datastore/meta/namespaces.go

+// Copyright 2015 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package meta
+
+import (
+        "github.com/luci/gae/service/datastore"
+        "github.com/luci/luci-go/common/logging"
+        "golang.org/x/net/context"
+)
+
+// Namespaces returns a list of all of the namespaces in the datastore.
+//
+// This is done by issuing a datastore query for kind "__namespace__". The
+// resulting keys will have IDs for the namespaces, namely:
+//      - The empty namespace will have integer ID 1. We ignore this.
+//      - Other namespaces will have string IDs.
+func Namespaces(c context.Context) ([]string, error) {

[dnj, 2016/04/23 02:17:10]

(This is also verbatim transplant from luci-go) w/ method name change.

[iannucci, 2016/04/23 19:15:21]

I'm not comfortable making the default API be GetAll; it will only work with a
relatively small number of namespaces. A function taking `func (ns string)
error`, with the ability to return gae.Stop or a user error to stop, and nil to
continue would be much better, IMO. It could make sense to have a AllNamespaces
which does the allocate+GetAll though.

We'll probably also need to add NamespacesWithPrefix(c, "prefix-", func()) and
NamespacesBetween(c, "start", "end", func() ...). To that end, would it be
better to have a function that just returns the Query?

[dnj, 2016/04/24 00:08:37]

The problem with a query is that you still have to extract the namespaces out of
the keys.

I've restructured this around a callback, adding a collector class to help with
the "all" case. Moving forward, once #49 is resolved, we can add vararg
NamespaceOptions to the call to allow begin/end cursors. We can also update the
prefix function to use a begin cursor.

+        q := datastore.NewQuery("__namespace__").KeysOnly(true)
+
+        // Query our datastore for the full set of namespaces.
+        var namespaceKeys []*datastore.Key
+        if err := datastore.Get(c).GetAll(q, &namespaceKeys); err != nil {

[iannucci, 2016/04/23 19:15:21]

Note: this query will ONLY run in the default (empty) namespace. I would doc
this in the help string for now, since the go SDK is broken and won't allow
setting the namespace "" unconditionally. This query also will not function from
inside a transaction :/

[dnj, 2016/04/24 00:08:37]

So on live Datastore, I can execute a GQL query for namespaces within any
namespace. I haven't tested transactional.

+                logging.WithError(err).Errorf(c, "Failed to execute namespace query.")
+                return nil, err
+        }
+
+        namespaces := make([]string, 0, len(namespaceKeys))
+        for _, nk := range namespaceKeys {
+                if ns := nk.StringID(); ns != "" {
+                        namespaces = append(namespaces, ns)
+                }
+        }
+        return namespaces, nil
+}