Create a gyp generator for Eclipse CDT settings

pylib/gyp/generator/eclipse.py

Index: pylib/gyp/generator/eclipse.py
diff --git a/pylib/gyp/generator/eclipse.py b/pylib/gyp/generator/eclipse.py
new file mode 100644
index 0000000000000000000000000000000000000000..78faa5446ad62ae343b7a4d9e8175e9edbf4e1a7
--- /dev/null
+++ b/pylib/gyp/generator/eclipse.py
@@ -0,0 +1,227 @@
+# Copyright (c) 2012 Google Inc. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+"""GYP backend that generates Eclipse CDT settings files.
+
+This backend DOES NOT generate Eclipse CDT projects. Instead, it generates XML
+files that can be imported into an Eclipse CDT project. The XML file contains a
+list of include paths and symbols (i.e. defines).
+"""

[Nico, 2012/04/16 17:21:52]

Since you don't intend this to be generally useful, the docstring should
probably list that intention, and the specific caveats this generator has.

[Jesse Greenwald, 2012/04/27 17:12:29]

Done.

+
+import os.path
+import subprocess
+import gyp
+import gyp.common
+import shlex
+
+generator_wants_static_library_dependencies_adjusted = False
+
+generator_default_variables = {
+}
+
+for dirname in ['INTERMEDIATE_DIR', 'PRODUCT_DIR', 'LIB_DIR', 'SHARED_LIB_DIR']:
+  # Some gyp steps fail if these are empty(!).
+  generator_default_variables[dirname] = 'dir'
+
+for unused in ['RULE_INPUT_PATH', 'RULE_INPUT_ROOT', 'RULE_INPUT_NAME',
+               'RULE_INPUT_DIRNAME', 'RULE_INPUT_EXT',
+               'EXECUTABLE_PREFIX', 'EXECUTABLE_SUFFIX',
+               'STATIC_LIB_PREFIX', 'STATIC_LIB_SUFFIX',
+               'SHARED_LIB_PREFIX', 'SHARED_LIB_SUFFIX']:
+  generator_default_variables[unused] = ''
+
+# Include dirs will occasionaly use the SHARED_INTERMEDIATE_DIR variable as
+# part of the path when dealing with generated headers.
+# TODO: Make this work for all gyp generators (possible?).  This currently
+# matches what make gyp generator does.
+generator_default_variables['SHARED_INTERMEDIATE_DIR'] = \
+    os.getcwd() + '/out/Release/obj/gen'

[Nico, 2012/04/16 17:21:52]

If this supports just one configuration, maybe you want to take the
configuration as a generator variable instead of hardcoding 'Release' (see the
ninja generator as an example)

[Jesse Greenwald, 2012/04/27 17:12:29]

I've updated it to match the ninja generator behavior.

+
+
+def CalculateVariables(default_variables, params):
+  generator_flags = params.get('generator_flags', {})
+  for key, val in generator_flags.items():
+    default_variables.setdefault(key, val)
+  default_variables.setdefault('OS', gyp.common.GetFlavor(params))
+
+
+def CalculateGeneratorInputInfo(params):
+  """Calculate the generator specific info that gets fed to input (called by
+  gyp)."""
+  generator_flags = params.get('generator_flags', {})
+  if generator_flags.get('adjust_static_libraries', False):
+    global generator_wants_static_library_dependencies_adjusted
+    generator_wants_static_library_dependencies_adjusted = True
+
+
+def GetAllIncludeDirs(target_list, target_dicts):
+  """Calculate the set of include dirs to be used.
+
+  Returns:
+    A list including all the include_dir's specified for every target followed
+    by any incude dirs that were added as cflag compiler options.

[Nico, 2012/04/16 16:12:12]

typo incude

[Jesse Greenwald, 2012/04/16 17:00:55]

Done.

+  """
+
+  gyp_includes = []
+  compiler_includes = []
+
+  for target_name in target_list:
+    target = target_dicts[target_name]
+
+    for _, config in target['configurations'].iteritems():
+
+      # Look for any include dirs that were explicitly added via cflags. This
+      # may be done in gyp files to force certain includes to come at the end.
+      cflags = config['cflags']

[Nico, 2012/04/16 16:12:12]

'cflags' is linux-specific. It shouldn't be used to add include directories,
that's what the cross-OS cross-generator 'include_dirs' is for. If there are
concerns about the ordering of directories in include_dir, that should be
addressed in another way.

[Jesse Greenwald, 2012/04/16 17:00:55]

That makes sense.. but unfortunately this is being done in build/common.gypi:

  'cflags': [
    '-I<(android_ndk_root)/sources/cxx-stl/stlport/stlport',
  ],

Without this hack in the eclipse generator, the stlport includes won't be picked
up.

[Nico, 2012/04/16 17:21:52]

build/common.gypi could be changed though, right?

[Jesse Greenwald, 2012/04/27 17:12:29]

It could probably be fixed at some point, although I'm not sure how this should
be done.

+      for cflag in cflags:
+        include_dir = ''
+        if cflag.startswith('-I'):
+          include_dir = cflag[2:]
+        if include_dir and not include_dir in compiler_includes:
+          compiler_includes.append(include_dir)
+
+      # Find standard gyp include dirs

[Nico, 2012/04/16 16:12:12]

add trailing .

[Jesse Greenwald, 2012/04/16 17:00:55]

Done.

+      if config.has_key('include_dirs'):
+        include_dirs = config['include_dirs']
+        for include_dir in include_dirs:
+          if not os.path.isabs(include_dir):
+            base_dir = os.path.dirname(target_name)
+
+            include_dir = base_dir + '/' + include_dir
+            include_dir = os.path.abspath(include_dir)
+
+          if not include_dir in gyp_includes:
+            gyp_includes.append(include_dir)
+
+  gyp_includes.sort()
+
+  # Generate a list that has all the include dirs
+  all_includes = list(gyp_includes)
+  for compiler_include in compiler_includes:
+    if not compiler_include in all_includes:
+      all_includes.append(compiler_include)
+
+  # All done
+  return all_includes
+
+
+def GetCompilerPath(target_list, target_dicts, data):
+  """Determine a command that can be used to invoke the compiler.
+
+  Returns:
+    If this is a gyp project that has explicit make settings, try to determine
+    the compiler from that.  Otherwise, see if a compiler was specified via the
+    CC_target environment variable.
+  """
+
+  # First, see if the compiler is configured in make's settings
+  build_file, _, _ = gyp.common.ParseQualifiedTarget(target_list[0])
+  make_global_settings_dict = data[build_file].get('make_global_settings', {})
+  for key, value in make_global_settings_dict:
+    if key in ['CC', 'CXX']:
+      return value
+
+  # Check to see if the compiler was specified as an environment variable
+  cc_target = os.environ.get('CC_target')
+  if cc_target:
+    return cc_target
+
+  # Default
+  return 'gcc'
+
+
+def GetAllDefines(target_list, target_dicts, data):
+  """Calculate the defines for a project.
+
+  Returns:
+    A dict that includes explict defines declared in gyp files along with all of
+    the default defines that the compiler uses.
+  """
+
+  # Get defines declared in the gyp files
+  all_defines = {}
+  for target_name in target_list:
+    target = target_dicts[target_name]
+
+    for _, config in target['configurations'].iteritems():
+      for define in config['defines']:
+        split_define = define.split('=', 1)
+        if len(split_define) == 1:
+          split_define.append('1')
+        if split_define[0].strip() in all_defines:
+          # Already defined
+          continue
+
+        all_defines[split_define[0].strip()] = split_define[1].strip()
+
+  # Get default compiler defines (if possible)
+  cc_target = GetCompilerPath(target_list, target_dicts, data)
+  if cc_target:
+    command = shlex.split(cc_target)
+    command.extend(['-E', '-dM', '-'])
+    cpp_proc = subprocess.Popen(args=command, cwd='.',
+                                stdin=subprocess.PIPE, stdout=subprocess.PIPE)
+    cpp_output = cpp_proc.communicate()[0]
+    cpp_lines = cpp_output.split('\n')
+    for cpp_line in cpp_lines:
+      if not cpp_line.strip():
+        continue
+      cpp_line_parts = cpp_line.split(' ', 2)
+      key = cpp_line_parts[1]
+      if len(cpp_line_parts) >= 3:
+        val = cpp_line_parts[2]
+      else:
+        val = '1'
+      all_defines[key] = val
+
+  return all_defines
+
+
+def WriteIncludePaths(out, eclipse_langs, include_dirs):
+  """Write the includes section of a CDT settings export file."""
+
+  out.write('  <section name="org.eclipse.cdt.internal.ui.wizards.' \
+            'settingswizards.IncludePaths">\n')
+  out.write('    <language name="holder for library settings"></language>\n')
+  for lang in eclipse_langs:
+    out.write('    <language name="%s">\n' % lang)
+    for include_dir in include_dirs:
+      out.write('      <includepath workspace_path="false">%s</includepath>\n' %
+                include_dir)
+    out.write('    </language>\n')
+  out.write('  </section>\n')
+
+
+def WriteMacros(out, eclipse_langs, defines):
+  """Write the macros section of a CDT settings export file."""
+
+  out.write('  <section name="org.eclipse.cdt.internal.ui.wizards.' \
+            'settingswizards.Macros">\n')
+  out.write('    <language name="holder for library settings"></language>\n')
+  for lang in eclipse_langs:
+    out.write('    <language name="%s">\n' % lang)
+    for key in sorted(defines.iterkeys()):
+      out.write('      <macro><name>%s</name><value>%s</value></macro>\n' %
+                (key, defines[key]))
+    out.write('    </language>\n')
+  out.write('  </section>\n')
+
+
+def GenerateOutput(target_list, target_dicts, data, params):
+  """Generate an XML settings file that can be imported into a CDT project."""
+

[Nico, 2012/04/16 17:21:52]

You might want to print a warning or error if any of --generator-output
(controls where generator output such as generated makefiles go), -Goutput_dir
(controls where the build output goes), --target_dir (controls the source root
location) are set, or are set to unexpected values.

[Jesse Greenwald, 2012/04/27 17:12:29]

Done.

+  out = open('eclipse-cdt-settings.xml', 'w')
+
+  out.write('<?xml version="1.0" encoding="UTF-8"?>\n')
+  out.write('<cdtprojectproperties>\n')
+
+  eclipse_langs = ['C++ Source File', 'C Source File', 'Assembly Source File',
+                   'GNU C++', 'GNU C', 'Assembly']
+  WriteIncludePaths(out, eclipse_langs,
+                    GetAllIncludeDirs(target_list, target_dicts))
+  WriteMacros(out, eclipse_langs, GetAllDefines(target_list, target_dicts,

[Nico, 2012/04/16 16:12:12]

This looks like it sets all defines from all targets for each target. That's not
what other generators do: usually, the 'defines' in one target are only set for
that target, unless they are also in 'export_dependent_settings'

[Jesse Greenwald, 2012/04/16 17:00:55]

The settings.xml file that gets generated isn't perfect, it's more of a close
approximation.  This is because you can put target specific properties into a
CDT settings file.

If an entire Eclipse CDT project was being generated, we could do a better job
because it allows include paths and symbols to be defined on a per-path basis.

The Eclipse CDT project format is fairly complicated, though.  The generated
settings.xml file that gets generated seems to be a good compromise between
complexity and correctness.  I've found that nearly everything is resolving
correctly in CDT when using one of these generated settings.xml files.

+              data))
+
+  out.write('</cdtprojectproperties>\n')
+  out.close()