Attempting to fix a C:\cygwin\bin\python.exe stack trace during windows deploy.

Attempting to fix a C:\cygwin\bin\python.exe stack trace
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=135377

[johnw1, 2012-04-27 20:49:55]

Hello!

We were seeing the following repeated stack trace when trying to deploy new
windows machines via setup.cmd (Tested on Win7 and Vista). Bev and I were taking
a look...

*****************
2 [main] python 2528 C:\cygwin\bin\python.exe: *** fatal error - unable to
 remap \\?\C:\cygwin\lib\python2.6\lib-dynload\itertools.dll to same address as
parent: 0x370000 != 0x3F0000
Stack trace:
Frame     Function  Args
00288BC8  6102796B  (00288BC8, 00000000, 00000000, 00000000)
00288EB8  6102796B  (6117EC60, 00008000, 00000000, 61180977)
00289EE8  61004F1B  (611A7FAC, 6124730C, 00370000, 003F0000)
End of stack trace
      2 [main] python 416 fork: child 2528 - died waiting for dll loading, errno
 11
Error: Command svn checkout svn://svn.chromium.org/chrome-internal/trunk/tools/b
uild/slave.DEPS /cygdrive/e/b/slave.DEPS --ignore-externals returned non-zero ex
it status 11 in /cygdrive/e/b
Visit http://code.google.com/p/chromium/wiki/CygwinDllRemappingFailure to learn
how to fix this error; you need to rebase your cygwin dlls
bash: build/site_config/.bot_password: No such file or directory
bash: build/slave/info/host: No such file or directory
*****************

Rebase seems to fix this on a case by case basis, however we discovered that by
adding:

PATH=%~dp0;%PATH%

back to gclient.bat, it permanently fixes this error.

Looks like that line was removed here:
https://chromiumcodereview.appspot.com/9751009/diff/33001/gclient.bat

Is it advisable to add this line back? Thanks for any advice!

[gab, 2012-04-30 22:48:54]

@maruel: You originally suggested (if I recall correctly) that I remove this
line as part of the last change I made, do you remember why?

Thanks,
Gab

[M-A Ruel, 2012-04-30 23:43:06]

Sorry I missed it.

On 2012/04/27 20:49:55, johnw1 wrote:
> Hello!
> 
> We were seeing the following repeated stack trace when trying to deploy new
> windows machines via setup.cmd (Tested on Win7 and Vista). Bev and I were
taking
> a look...
> 
> *****************
> 2 [main] python 2528 C:\cygwin\bin\python.exe: *** fatal error - unable to
>  remap \\?\C:\cygwin\lib\python2.6\lib-dynload\itertools.dll to same address
as
> parent: 0x370000 != 0x3F0000
> Stack trace:
> Frame     Function  Args
> 00288BC8  6102796B  (00288BC8, 00000000, 00000000, 00000000)
> 00288EB8  6102796B  (6117EC60, 00008000, 00000000, 61180977)
> 00289EE8  61004F1B  (611A7FAC, 6124730C, 00370000, 003F0000)
> End of stack trace
>       2 [main] python 416 fork: child 2528 - died waiting for dll loading,
errno
>  11
> Error: Command svn checkout
svn://svn.chromium.org/chrome-internal/trunk/tools/b
> uild/slave.DEPS /cygdrive/e/b/slave.DEPS --ignore-externals returned non-zero
ex
> it status 11 in /cygdrive/e/b
> Visit http://code.google.com/p/chromium/wiki/CygwinDllRemappingFailure to
learn
> how to fix this error; you need to rebase your cygwin dlls
> bash: build/site_config/.bot_password: No such file or directory
> bash: build/slave/info/host: No such file or directory
> *****************
> 
> Rebase seems to fix this on a case by case basis, however we discovered that
by
> adding:
> 
> PATH=%~dp0;%PATH%
> 
> back to gclient.bat, it permanently fixes this error.
> 
> Looks like that line was removed here:
> https://chromiumcodereview.appspot.com/9751009/diff/33001/gclient.bat
> 
> Is it advisable to add this line back? Thanks for any advice!

The problem you are seeing is that you are running 'gclient.bat' from within
cygwin. And from within cygwin, you should be running 'gclient'.

In practice the slaves are not using cygwin so you should shell out cmd.exe or
make sure you don't have cygwin executable in the path when shelling out
'gclient.bat'.

M-A

[johnw, 2012-05-04 02:18:19]

On 2012/04/30 23:43:06, Marc-Antoine Ruel wrote:
> Sorry I missed it.
> 
> On 2012/04/27 20:49:55, johnw1 wrote:
> > Hello!
> > 
> > We were seeing the following repeated stack trace when trying to deploy new
> > windows machines via setup.cmd (Tested on Win7 and Vista). Bev and I were
> taking
> > a look...
> > 
> > *****************
> > 2 [main] python 2528 C:\cygwin\bin\python.exe: *** fatal error - unable to
> >  remap \\?\C:\cygwin\lib\python2.6\lib-dynload\itertools.dll to same address
> as
> > parent: 0x370000 != 0x3F0000
> > Stack trace:
> > Frame     Function  Args
> > 00288BC8  6102796B  (00288BC8, 00000000, 00000000, 00000000)
> > 00288EB8  6102796B  (6117EC60, 00008000, 00000000, 61180977)
> > 00289EE8  61004F1B  (611A7FAC, 6124730C, 00370000, 003F0000)
> > End of stack trace
> >       2 [main] python 416 fork: child 2528 - died waiting for dll loading,
> errno
> >  11
> > Error: Command svn checkout
> svn://svn.chromium.org/chrome-internal/trunk/tools/b
> > uild/slave.DEPS /cygdrive/e/b/slave.DEPS --ignore-externals returned
non-zero
> ex
> > it status 11 in /cygdrive/e/b
> > Visit http://code.google.com/p/chromium/wiki/CygwinDllRemappingFailure to
> learn
> > how to fix this error; you need to rebase your cygwin dlls
> > bash: build/site_config/.bot_password: No such file or directory
> > bash: build/slave/info/host: No such file or directory
> > *****************
> > 
> > Rebase seems to fix this on a case by case basis, however we discovered that
> by
> > adding:
> > 
> > PATH=%~dp0;%PATH%
> > 
> > back to gclient.bat, it permanently fixes this error.
> > 
> > Looks like that line was removed here:
> > https://chromiumcodereview.appspot.com/9751009/diff/33001/gclient.bat
> > 
> > Is it advisable to add this line back? Thanks for any advice!
> 
> The problem you are seeing is that you are running 'gclient.bat' from within
> cygwin. And from within cygwin, you should be running 'gclient'.
> 
> In practice the slaves are not using cygwin so you should shell out cmd.exe or
> make sure you don't have cygwin executable in the path when shelling out
> 'gclient.bat'.
> 
> M-A

Hello, 

Thanks for your advice and help with this. I went back an shelled out calls to
gclient.bat, but the problem persisted. I traced the code and it seems to be
failing consistently on line 13 of gclient.bat

python "%~dp0\gclient.py" %*

Question: Shouldn't we be calling python.BAT from within gclient.bat? (instead
of calling plain python)... python.bat seems to correctly set the python_bin
path and references python.exe. I think since we removed "PATH=%~dp0;%PATH%"
from gclient.bat, it may have exposed this problem. Let me know what you think
or if you have any further advice.

Thanks again.

[M-A Ruel, 2012-05-04 02:22:06]

On 2012/05/04 02:18:19, johnw wrote:
> Hello, 
> 
> Thanks for your advice and help with this. I went back an shelled out calls to
> gclient.bat, but the problem persisted. I traced the code and it seems to be
> failing consistently on line 13 of gclient.bat
> 
> python "%~dp0\gclient.py" %*
> 
> Question: Shouldn't we be calling python.BAT from within gclient.bat? (instead
> of calling plain python)... python.bat seems to correctly set the python_bin
> path and references python.exe. I think since we removed "PATH=%~dp0;%PATH%"
> from gclient.bat, it may have exposed this problem. Let me know what you think
> or if you have any further advice.

Well, this is a problem due to the fact that it is being run with a cygwin path.
Why was "PATH=%~dp0;%PATH%" removed? I don't recall. I think this should be
kept. I'm fine with:
%~dp0python "%~dp0\gclient.py" %*

> Thanks again.

np!

[gab, 2012-05-04 02:47:33]

On Thu, May 3, 2012 at 10:22 PM, <maruel@chromium.org> wrote:

> On 2012/05/04 02:18:19, johnw wrote:
>
>> Hello,
>>
>
>  Thanks for your advice and help with this. I went back an shelled out
>> calls to
>> gclient.bat, but the problem persisted. I traced the code and it seems to
>> be
>> failing consistently on line 13 of gclient.bat
>>
>
>  python "%~dp0\gclient.py" %*
>>
>
>  Question: Shouldn't we be calling python.BAT from within gclient.bat?
>> (instead
>> of calling plain python)... python.bat seems to correctly set the
>> python_bin
>> path and references python.exe. I think since we removed
>> "PATH=%~dp0;%PATH%"
>> from gclient.bat, it may have exposed this problem. Let me know what you
>> think
>> or if you have any further advice.
>>
>
> Well, this is a problem due to the fact that it is being run with a cygwin
> path.
> Why was "PATH=%~dp0;%PATH%" removed? I don't recall. I think this should be
> kept. I'm fine with:
> %~dp0python "%~dp0\gclient.py" %*
>

It was removed in my patch (https://chromiumcodereview.appspot.com/9751009/)
as I think you told me it was no longer needed.

Cheers,
Gab


>
>  Thanks again.
>>
>
> np!
>
>
https://chromiumcodereview.**appspot.com/10206030/<https://chromiumcodereview...
>

[johnw, 2012-05-04 03:18:45]

Hi,

%~dp0python "%~dp0\gclient.py" %* works fine on it's own. I've updated the
CL.

Let me know if you think we need both changes in this CL or if just fixing
the python path with "%~dp0python" is enough. I'm ok with either or both,
as long as it stops the stack trace. ;)

Thanks again,

-john-


On Thu, May 3, 2012 at 7:47 PM, Gabriel Charette <gab@chromium.org> wrote:

>
>
> On Thu, May 3, 2012 at 10:22 PM, <maruel@chromium.org> wrote:
>
>> On 2012/05/04 02:18:19, johnw wrote:
>>
>>> Hello,
>>>
>>
>>  Thanks for your advice and help with this. I went back an shelled out
>>> calls to
>>> gclient.bat, but the problem persisted. I traced the code and it seems
>>> to be
>>> failing consistently on line 13 of gclient.bat
>>>
>>
>>  python "%~dp0\gclient.py" %*
>>>
>>
>>  Question: Shouldn't we be calling python.BAT from within gclient.bat?
>>> (instead
>>> of calling plain python)... python.bat seems to correctly set the
>>> python_bin
>>> path and references python.exe. I think since we removed
>>> "PATH=%~dp0;%PATH%"
>>> from gclient.bat, it may have exposed this problem. Let me know what you
>>> think
>>> or if you have any further advice.
>>>
>>
>> Well, this is a problem due to the fact that it is being run with a
>> cygwin path.
>> Why was "PATH=%~dp0;%PATH%" removed? I don't recall. I think this should
>> be
>> kept. I'm fine with:
>> %~dp0python "%~dp0\gclient.py" %*
>>
>
> It was removed in my patch (
> https://chromiumcodereview.appspot.com/9751009/) as I think you told me
> it was no longer needed.
>
> Cheers,
> Gab
>
>
>>
>>  Thanks again.
>>>
>>
>> np!
>>
>>
https://chromiumcodereview.**appspot.com/10206030/<https://chromiumcodereview...
>>
>
>

[M-A Ruel, 2012-05-04 13:00:56]

lgtm, I'm stupid. We should probably fix git-cl.bat, gcl.bat and friends.

[commit-bot: I haz the power, 2012-05-04 13:19:26]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/johnw@chromium.org/10206030/7002

[commit-bot: I haz the power, 2012-05-04 13:19:35]

Can't apply patch for file depot_tools/gclient.bat.
While running patch -p0 --forward --force;
patching file depot_tools/gclient.bat
Hunk #1 FAILED at 3.
1 out of 1 hunk FAILED -- saving rejects to file depot_tools/gclient.bat.rej

[M-A Ruel, 2012-05-04 13:22:01]

On 2012/05/04 13:19:35, I haz the power (commit-bot) wrote:
> Can't apply patch for file depot_tools/gclient.bat.
> While running patch -p0 --forward --force;
> patching file depot_tools/gclient.bat
> Hunk #1 FAILED at 3.
> 1 out of 1 hunk FAILED -- saving rejects to file depot_tools/gclient.bat.rej

Please commit manually.