Add datastore dumper tool.

Add datastore dumper tool.

This allows you to dump the datastore in whole or part, with or without special
formatting directives.

R=dnj@chromium.org, nodir@chromium.org
BUG=

Committed: https://github.com/luci/gae/commit/062a32372e8f6159980129438a7ff15b5509ddc2

[dnj (Google), 2016-06-02 03:55:57]

dnj@google.com changed reviewers:
+ dnj@google.com

[dnj (Google), 2016-06-02 03:55:58]

https://codereview.chromium.org/2037433002/diff/40001/service/datastore/dumpe...
File service/datastore/dumper/dumper.go (right):

https://codereview.chromium.org/2037433002/diff/40001/service/datastore/dumpe...
service/datastore/dumper/dumper.go:82: prnt := func(format string, args
...interface{}) bool {
I don't think "bool on error, check bool" is any better than "error on error,
check error". IMO just make these all return error, get rid of "runErr", and
have the query forward errors from prnt and prop.

https://codereview.chromium.org/2037433002/diff/40001/service/datastore/dumpe...
service/datastore/dumper/dumper.go:110: return datastore.Stop
Why do we stop on print failure here? Seems like a legitimate error.

https://codereview.chromium.org/2037433002/diff/40001/service/datastore/dumpe...
service/datastore/dumper/dumper.go:130: prop(key.Kind(), k, pm[k])
If one prop fails, then another succeeds, we mask the error. Just "if err :=
prop ... { return err }". None of these should fail.

https://codereview.chromium.org/2037433002/diff/40001/service/datastore/dumpe...
service/datastore/dumper/dumper.go:135: err = runErr
If you return inner error through Run, you won't need this either, which is
nice.

[iannucci, 2016-06-02 22:59:45]

add test and fix nit, PTAL

https://chromiumcodereview.appspot.com/2037433002/diff/40001/service/datastor...
File service/datastore/dumper/dumper.go (right):

https://chromiumcodereview.appspot.com/2037433002/diff/40001/service/datastor...
service/datastore/dumper/dumper.go:110: return datastore.Stop
On 2016/06/02 03:55:58, dnj (Google) wrote:
> Why do we stop on print failure here? Seems like a legitimate error.

the only errors which may occur are IO errors, which are recorded by prnt. I'll
make this more normal.

[dnj, 2016-06-06 14:01:38]

lgtm, have comment, up to you.

https://chromiumcodereview.appspot.com/2037433002/diff/40001/service/datastor...
File service/datastore/dumper/dumper.go (right):

https://chromiumcodereview.appspot.com/2037433002/diff/40001/service/datastor...
service/datastore/dumper/dumper.go:110: return datastore.Stop
On 2016/06/02 22:59:45, iannucci wrote:
> On 2016/06/02 03:55:58, dnj (Google) wrote:
> > Why do we stop on print failure here? Seems like a legitimate error.
> 
> the only errors which may occur are IO errors, which are recorded by prnt.
I'll
> make this more normal.

Acknowledged.

https://chromiumcodereview.appspot.com/2037433002/diff/60001/service/datastor...
File service/datastore/dumper/dumper.go (right):

https://chromiumcodereview.appspot.com/2037433002/diff/60001/service/datastor...
service/datastore/dumper/dumper.go:105: key := datastore.GetMetaDefault(pm,
"key", nil).(*datastore.Key)
nit: I think "err" and "err" are a bit awkward here. I don't see much that is
gained from named output values on the inner function, so I'd prefer you just
"return err" in here.

Don't want to block CL on it though, so up to you.

[iannucci, 2016-06-11 02:18:16]

The CQ bit was checked by iannucci@chromium.org

[iannucci, 2016-06-11 02:18:16]

The patchset sent to the CQ was uploaded after l-g-t-m from dnj@chromium.org
Link to the patchset: https://codereview.chromium.org/2037433002/#ps80001
(title: "fix err")

[commit-bot: I haz the power, 2016-06-11 02:18:19]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2037433002/80001

[iannucci, 2016-06-11 02:18:40]

On 2016/06/06 14:01:38, dnj wrote:
> lgtm, have comment, up to you.
> 
>
https://chromiumcodereview.appspot.com/2037433002/diff/40001/service/datastor...
> File service/datastore/dumper/dumper.go (right):
> 
>
https://chromiumcodereview.appspot.com/2037433002/diff/40001/service/datastor...
> service/datastore/dumper/dumper.go:110: return datastore.Stop
> On 2016/06/02 22:59:45, iannucci wrote:
> > On 2016/06/02 03:55:58, dnj (Google) wrote:
> > > Why do we stop on print failure here? Seems like a legitimate error.
> > 
> > the only errors which may occur are IO errors, which are recorded by prnt.
> I'll
> > make this more normal.
> 
> Acknowledged.
> 
>
https://chromiumcodereview.appspot.com/2037433002/diff/60001/service/datastor...
> File service/datastore/dumper/dumper.go (right):
> 
>
https://chromiumcodereview.appspot.com/2037433002/diff/60001/service/datastor...
> service/datastore/dumper/dumper.go:105: key := datastore.GetMetaDefault(pm,
> "key", nil).(*datastore.Key)
> nit: I think "err" and "err" are a bit awkward here. I don't see much that is
> gained from named output values on the inner function, so I'd prefer you just
> "return err" in here.
> 
> Don't want to block CL on it though, so up to you.

Done

[commit-bot: I haz the power, 2016-06-11 02:28:49]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-06-11 02:28:49]

Try jobs failed on following builders:
  Luci-GAE Presubmit on master.tryserver.infra (JOB_FAILED,
https://build.chromium.org/p/tryserver.infra/builders/Luci-GAE%20Presubmit/bu...)

[iannucci, 2016-06-14 00:05:39]

The CQ bit was checked by iannucci@chromium.org

[iannucci, 2016-06-14 00:05:40]

The patchset sent to the CQ was uploaded after l-g-t-m from dnj@chromium.org
Link to the patchset: https://codereview.chromium.org/2037433002/#ps100001
(title: "fix copyright")

[commit-bot: I haz the power, 2016-06-14 00:05:49]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2037433002/100001

[commit-bot: I haz the power, 2016-06-14 00:11:08]

Description was changed from

==========
Add datastore dumper tool.

This allows you to dump the datastore in whole or part, with or without special
formatting directives.

R=dnj@chromium.org, nodir@chromium.org
BUG=
==========

to

==========
Add datastore dumper tool.

This allows you to dump the datastore in whole or part, with or without special
formatting directives.

R=dnj@chromium.org, nodir@chromium.org
BUG=

Committed:
https://github.com/luci/gae/commit/062a32372e8f6159980129438a7ff15b5509ddc2
==========

[commit-bot: I haz the power, 2016-06-14 00:11:09]

Committed patchset #6 (id:100001) as
https://github.com/luci/gae/commit/062a32372e8f6159980129438a7ff15b5509ddc2