Fix quoting shell args on Windows

pylib/gyp/generator/ninja.py

 # Copyright (c) 2012 Google Inc. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 import copy
 import gyp
 import gyp.common
 import gyp.system_test
 import gyp.xcode_emulation
 import os.path
@@ skipping 37 matching lines @@
 # - how to decide what the host compiler is (should not just be $cc).
 # - need ld_host as well.
 generator_supports_multiple_toolsets = False
 
 
 def StripPrefix(arg, prefix):
   if arg.startswith(prefix):
     return arg[len(prefix):]
   return arg
 
+windows_quoter_regex = re.compile(r'(\\*)"')

[Nico, 2012/02/20 18:33:22]

grit is very slow because it has all these module-scope 're.compile' calls which
get executed at module load time. grit imports all its modules but uses only a
few of these regexen on every run depending on various flags. So as a result
grit has slow startup because of its 'global' re.compile statements. Maybe we
can prevent this somehow?

[scottmg, 2012/02/20 22:30:48]

I guess it depends if we end up with a lot, for some definition of a lot.
Quoting shell arguments happens lots, otherwise we could just not bother
precompiling the regex at all I suppose. I guess I could make it a lazily
initialized global if you think adding global re's is a bad idea (so it'd just
be a test per shell arg, rather than a compile).

 
 def QuoteShellArgument(arg):
   """Quote a string such that it will be interpreted as a single argument
   by the shell."""
   # Rather than attempting to enumerate the bad shell characters, just
   # whitelist common OK ones and quote anything else.
   if re.match(r'^[a-zA-Z0-9_=-]+$', arg):
     return arg  # No quoting necessary.
-  return "'" + arg.replace("'", "'" + '"\'"' + "'")  + "'"
+  if sys.platform == 'win32':

[Nico, 2012/02/20 18:33:22]

It's probably better to key this off 'flavor' (which you could pass in)? Then
msvc projects could be created on linux (GYP_GENERATORS=ninja-win
build/gyp_chromium)

[scottmg, 2012/02/20 22:30:48]

Done.

+    # See http://goo.gl/tRtOd and http://goo.gl/dhPnp including the comment

[Nico, 2012/02/20 18:33:22]

I get "The resource you are looking for has been removed, had its name changed,
or is temporarily unavailable." for the first link.

[scottmg, 2012/02/20 22:30:48]

Fixed.

+    # threads. This is actually the quoting rules for CommandLineToArgvW, not
+    # for the shell, because the shell doesn't do anything in Windows. This
+    # works more or less because most programs (including the compiler, etc.)
+    # use that function to handle command line arguments.
+    #
+    # For a literal quote, CommandLineToArgvW requires 2n+1 backslashes
+    # preceding it, and results in n backslashes + the quote. So we
+    # substitute in 2* what we match, +1 more, plus the quote.
+    tmp = windows_quoter_regex.sub(lambda mo: 2 * mo.group(1) + '\\"', arg)
+
+    # Now, we need to escape some things that are actually for the shell. %'s
+    # need to be doubled, and various other characters need to be ^ escaped.
+    tmp = re.sub(r'([&|^])', r'^\1', tmp)
+    # Also make sure to escape the % so that they're passed literally through
+    # escaping so they can be singled to just the original %. Otherwise,
+    # trying to pass the literal representation that looks like an environment
+    # variable to the shell (e.g. %PATH%) would fail.
+    tmp = tmp.replace('%', '^%^%')
+
+    # Finally, wrap the whole thing in quotes so that the above quote rule
+    # applies and whitespace isn't a word break.
+    return '"' + tmp + '"'
+  else:

[Nico, 2012/02/20 18:33:22]

Can you put this branch first? Then the common case ( :-) ) comes first, and you
can dedent the whole windows block.

More generally, do you think you'll do MSVC option emulation eventually? If so,
maybe you want to put the windows bits here into a new function into a new 'msvc
emulation' module (maybe pylib/gyp/msvc_emulation.py to parallel
xcode_emulation.py?), then this function could just look like

  ...
  if sys.platform == 'win32':
    return msvc_emulation.QuoteCmdExeArgument(arg)
  return "'" + arg.replace...

[scottmg, 2012/02/20 22:30:48]

Yeah, options emulation is the next thing to do after the basic tests all pass.
I'm not sure cmd.exe has anything to do with msvs (vs. bash implementation
details belonging in the main module), but it's a convenient place to stuff it
for now.

+    return "'" + arg.replace("'", "'" + '"\'"' + "'")  + "'"
 
 
 def InvertRelativePath(path):
   """Given a relative path like foo/bar, return the inverse relative path:
   the path from the relative path back to the origin dir.
 
   E.g. os.path.normpath(os.path.join(path, InvertRelativePath(path)))
   should always produce the empty string."""
 
   if not path:
@@ skipping 1170 matching lines @@
 
   user_config = params.get('generator_flags', {}).get('config', None)
   if user_config:
     GenerateOutputForConfig(target_list, target_dicts, data, params,
                             user_config)
   else:
     config_names = target_dicts[target_list[0]]['configurations'].keys()
     for config_name in config_names:
       GenerateOutputForConfig(target_list, target_dicts, data, params,
                               config_name)