Add an alternate default boto file for bootstrapping source.

Add an alternate default boto file for bootstrapping source.

Able to pull internal tools, but distinct from the source that
those tools may eventually build.

TBR=cmp@chromium.org
BUG=252226
TEST=local buildslave

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=209166

[Peter Mayo, 2013-06-20 00:42:55]

And the point of the previous bug.

[Peter Mayo, 2013-06-20 12:18:42]

There is an active email thread debating other ways to solve the problem at
hand.

We should capture the salient points in a bug before deciding whether to submit,
but any mechanical comments are welcome here.

[Peter Mayo, 2013-06-26 14:59:58]

M-A is on vacation,  let's use cmp@ or iannucci@ for the OWNERS LGTM

[Ryan Tseng, 2013-06-26 17:44:57]

https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
File download_from_google_storage.py (right):

https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
download_from_google_storage.py:262:
default=ifexists(os.path.expanduser('~/.boto.depot_tools')))
Instead of hardcoding ~/.boto.depot_toos, i'd rather it check for
.boto.depot_tools next to where it would've otherwise found the .boto file, so
that we can put the .boto.depot_tools file in /b/build/site_config/ instead of ~

[Peter Mayo, 2013-06-26 18:20:53]

Done.

https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
File download_from_google_storage.py (right):

https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
download_from_google_storage.py:262:
default=ifexists(os.path.expanduser('~/.boto.depot_tools')))
On 2013/06/26 17:44:57, Ryan T. wrote:
> Instead of hardcoding ~/.boto.depot_toos, i'd rather it check for
> .boto.depot_tools next to where it would've otherwise found the .boto file, so
> that we can put the .boto.depot_tools file in /b/build/site_config/ instead of
~

The .netrc and .ssh configs are in the home directory, so I'm not sure why we
want this one floating elsewhere, but meh.

Also, I'm not sure what uses that as the default, perhaps it isn't deployed yet,
because in my testing on the slave I was given it doesn't work.

[agable, 2013-06-26 18:59:03]

Everything in build/ and build-internal/ expects the bot to have its .boto file
at /b/build/site_config/.boto. Putting one in ~ would be new and nonstandard,
unless this is something that CrOS does and we don't know about it.

[Peter Mayo (wrong one), 2013-06-26 19:07:16]

https://sites.google.com/a/google.com/chrome-infrastructure/buildbot/deployin...
:

step 3

It's been documented in the chrome-infrastructure manual for a very long
time.  Just saying.

And if putting the identity that had access to the chrome-goma bucket there
had worked, I probably wouldn't be looking to do this.  I'll spend some
time figuring out why it doesn't.


On Wed, Jun 26, 2013 at 2:59 PM, <agable@chromium.org> wrote:

> Everything in build/ and build-internal/ expects the bot to have its .boto
> file
> at /b/build/site_config/.boto. Putting one in ~ would be new and
> nonstandard,
> unless this is something that CrOS does and we don't know about it.
>
>
https://codereview.chromium.**org/17351008/<https://codereview.chromium.org/1...
>



-- 

Peter Mayo | Waterloo | petermayo@google.com | 519-880-3439

[Ryan Tseng, 2013-06-26 19:24:37]

How are you testing the slave? Are you running commands on the command line, or
through run_slave.py?

Otherwise though the code lgmt

https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
File download_from_google_storage.py (right):

https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
download_from_google_storage.py:262:
default=ifexists(os.path.expanduser('~/.boto.depot_tools')))
On 2013/06/26 18:20:54, Peter Mayo wrote:
> On 2013/06/26 17:44:57, Ryan T. wrote:
> > Instead of hardcoding ~/.boto.depot_toos, i'd rather it check for
> > .boto.depot_tools next to where it would've otherwise found the .boto file,
so
> > that we can put the .boto.depot_tools file in /b/build/site_config/ instead
of
> ~
> 
> The .netrc and .ssh configs are in the home directory, so I'm not sure why we
> want this one floating elsewhere, but meh.
> 
> Also, I'm not sure what uses that as the default, perhaps it isn't deployed
yet,
> because in my testing on the slave I was given it doesn't work.

We have some code in run_slave.py that checks to see if theres a .boto file in
<build_dir>/site_config/.boto and use that file if it exists.  This is currently
how all Chromium test bots are configured right now (eg -m1, -m4, etc).

[Ryan Tseng, 2013-06-26 19:26:31]

Ooop nevermind, found something.  See comment.

Note:
call() is interactive, check_call() blocks until the process finishes.
call() is only used during the gsutil config passthrough
check_call() is used for all other calls to gsutil.

https://codereview.chromium.org/17351008/diff/14001/download_from_google_stor...
File download_from_google_storage.py (right):

https://codereview.chromium.org/17351008/diff/14001/download_from_google_stor...
download_from_google_storage.py:62: env['AWS_CREDENTIAL_FILE'] = self.boto_path
Also need to do it here.

[Peter Mayo, 2013-06-26 19:34:52]

On 2013/06/26 19:26:31, Ryan T. wrote:
> Ooop nevermind, found something.  See comment.
> 
> Note:
> call() is interactive, check_call() blocks until the process finishes.
> call() is only used during the gsutil config passthrough
> check_call() is used for all other calls to gsutil.
> 
>
https://codereview.chromium.org/17351008/diff/14001/download_from_google_stor...
> File download_from_google_storage.py (right):
> 
>
https://codereview.chromium.org/17351008/diff/14001/download_from_google_stor...
> download_from_google_storage.py:62: env['AWS_CREDENTIAL_FILE'] =
self.boto_path
> Also need to do it here.

Addressed, untested.  Back in a bit.

[Peter Mayo, 2013-06-26 19:35:34]

On 2013/06/26 19:24:37, Ryan T. wrote:
> How are you testing the slave? Are you running commands on the command line,
or
> through run_slave.py?

Runtest trying to get a compilation to work, blocked on 254567

> 
> Otherwise though the code lgmt
> 
>
https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
> File download_from_google_storage.py (right):
> 
>
https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
> download_from_google_storage.py:262:
> default=ifexists(os.path.expanduser('~/.boto.depot_tools')))
> On 2013/06/26 18:20:54, Peter Mayo wrote:
> > On 2013/06/26 17:44:57, Ryan T. wrote:
> > > Instead of hardcoding ~/.boto.depot_toos, i'd rather it check for
> > > .boto.depot_tools next to where it would've otherwise found the .boto
file,
> so
> > > that we can put the .boto.depot_tools file in /b/build/site_config/
instead
> of
> > ~
> > 
> > The .netrc and .ssh configs are in the home directory, so I'm not sure why
we
> > want this one floating elsewhere, but meh.
> > 
> > Also, I'm not sure what uses that as the default, perhaps it isn't deployed
> yet,
> > because in my testing on the slave I was given it doesn't work.
> 
> We have some code in run_slave.py that checks to see if theres a .boto file in
> <build_dir>/site_config/.boto and use that file if it exists.  This is
currently
> how all Chromium test bots are configured right now (eg -m1, -m4, etc).

[Peter Mayo, 2013-06-26 19:36:13]

On 2013/06/26 19:35:34, Peter Mayo wrote:
> On 2013/06/26 19:24:37, Ryan T. wrote:
> > How are you testing the slave? Are you running commands on the command line,
> or
> > through run_slave.py?
> 
> Runtest trying to get a compilation to work, blocked on 254567
> 
> > 
> > Otherwise though the code lgmt
> > 
> >
>
https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
> > File download_from_google_storage.py (right):
> > 
> >
>
https://codereview.chromium.org/17351008/diff/8001/download_from_google_stora...
> > download_from_google_storage.py:262:
> > default=ifexists(os.path.expanduser('~/.boto.depot_tools')))
> > On 2013/06/26 18:20:54, Peter Mayo wrote:
> > > On 2013/06/26 17:44:57, Ryan T. wrote:
> > > > Instead of hardcoding ~/.boto.depot_toos, i'd rather it check for
> > > > .boto.depot_tools next to where it would've otherwise found the .boto
> file,
> > so
> > > > that we can put the .boto.depot_tools file in /b/build/site_config/
> instead
> > of
> > > ~
> > > 
> > > The .netrc and .ssh configs are in the home directory, so I'm not sure why
> we
> > > want this one floating elsewhere, but meh.
> > > 
> > > Also, I'm not sure what uses that as the default, perhaps it isn't
deployed
> > yet,
> > > because in my testing on the slave I was given it doesn't work.
> > 
> > We have some code in run_slave.py that checks to see if theres a .boto file
in
> > <build_dir>/site_config/.boto and use that file if it exists.  This is
> currently
> > how all Chromium test bots are configured right now (eg -m1, -m4, etc).

254051

[Ryan Tseng, 2013-06-26 21:47:36]

lgtm.

cmp: Can you do an owners approval since maruel isn't around?

[Peter Mayo, 2013-06-28 17:02:30]

Tested.

[commit-bot: I haz the power, 2013-06-28 17:02:40]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/petermayo@chromium.org/17351008/20001

[commit-bot: I haz the power, 2013-06-28 17:04:46]

Change committed as 209166