Ignore Godot Engine source from Godot projects by default

[Xrayez]

Jump To Workaround

When Godot Engine is added as a submodule of a game project, various engine's build related obj files are falsely treated as assets, leading to a spam of errors related to importing.

Rationale: adding Godot as a submodule allows to keep track the exact Godot version used by the project at the moment of development, and allows to symlink any custom C++ modules associated with the project for compiling.

[akien-mga]

I don't think that's a common use case to add the engine code inside the project folder.

For your use case, you can just have this structure:

• My project
  • Godot
  • Game (with project.godot)
  • Design docs
  • Assets sources
  • Etc.

[Xrayez]

• My project
  • Godot
  • Game (with project.godot)

Hmm yes this is what I've been doing so far, the problem with such structure is that I have to sync the Godot version (commit hash) with the project. So far I've done this via prepare-commit-msg git hook:

#!/bin/sh
#
# Appends Godot Engine version in each commit
#

GODOT_VERSION=$(godot --version)
BUILD_NAME="Xrayez"

# Ensure that version contains build name
if [[ $GODOT_VERSION = *"$BUILD_NAME"* ]]; then
    # Append only if not already present in the commit message
    grep -qs $BUILD_NAME "$1" || printf "\n$GODOT_VERSION" >> "$1"
else
    # Build name is not configured
    exit -1
fi

So that each game project commit "keeps track" of what Godot version was used during developing:

commit 0fc135ec410b4592427e610b17c3cbf7a0e10752
...
Date:   Mon Apr 8 19:14:32 2019 +0300

    Transfer testing code to implementation

    3.2.dev.Xrayez.5095c28

But now I've figured that I could actually add the whole godot as a submodule, and this should already track the commit hash that way, plus there's a bonus to easily link custom modules to the engine inside game project.

Plus adding a simple and empty .gdignore for such specific use case would not hurt anything I suppose.

[Xrayez]

Checking out other commits within a game project would also force to checkout associated Godot version, and this even can be done automatically depending on the git configuration.

[akien-mga]

Your git repository can be initialized in the parent folder in the structure I exemplified, so you can have access to the commit hash from the Godot submodule.

[Xrayez]

Ok if you say that's possible I'll try that... but there's just another layer of usability with this... I was thinking that maybe a game project could have it's own SConstruct with the contents like so:

env = Environment()

# Compile Godot (with custom modules)
env.SConscript(['godot/SConstruct'])

# Compile GDNative plugins/modules
env.SConscript('modules/gdnative/a/SConstruct')
env.SConscript('modules/gdnative/b/SConstruct')
env.SConscript('modules/gdnative/c/SConstruct')

See Scons Hierarchical builds.

Would you say that I also need to keep GDNative modules outside game project source? Doesn't seem intuitive to me. And with setup like this it would be really, really convenient to compile it all from within a game project root folder with a regular scons command:

project/game> scons platform=windows target=release_debug bits=64 debug_symbols=no -j4

Btw, I haven't managed compiling godot that way yet, seems like some paths are absolute there in Godot's Sconstruct or something, but that's already possible for GDNative as reported by some user from Discord.

[Xrayez]

Your git repository can be initialized in the parent folder in the structure I exemplified, so you can have access to the commit hash from the Godot submodule.

I think I misread your message, I thought you suggested that having an ability to initialize Godot as a submodule outside the project is possible (which wouldn't make sense though):

git submodule add -f https://github.com/godotengine/godot ../godot
fatal: ../godot: '../godot' is outside repository

Having a simple direct access to commit hash in the parent directory is very similar to what I've been doing so far, so this defeats the purpose for me.

Anyways I think I've come up with another structure that would allow the editor to ignore Godot's build files while still benefiting from having godot as a submodule:

• My project
  • Game (with project.godot)
    • Godot (folder)
      • Engine (submodule)
      • Templates (.gitignored, contains custom export templates)
      • Tools (.gitignored, contains custom built editor binaries)
      • .gdignore (ignore the Engine indirectly...)

I still think such a simple enhancement wouldn't hurt anything. This is probably the smallest PR in Godot with such a lengthy discussion. 😄

[Xrayez]

Ok, the workaround is the following (assuming the above structure and within game project's root):

git submodule add --name godot https://github.com/godotengine/godot godot/engine`

Notice --name is godot as godot/engine path would be also become a name otherwise (by default).
Also you must not attempt to create the path structure godot/engine beforehand as it will complain that the path already exists on trying to add godot as a submodule.

Now, all you have to do is to create .gdignore within godot folder (which doesn't contain the engine itself directly as it's located at godot/engine). From there you can safely symlink existing custom C++ modules (that could be located at project/modules as submodules just fine). I use the following script to "push" the modules to godot engine for compiling:

#!/usr/bin/env python3

import os

MODULES_DIR = "modules/"
TARGET_DIR = "godot/engine/modules/"

for module in os.listdir(MODULES_DIR):
    # Relative path to module(s)
    path_to_module = os.path.relpath(os.path.join(MODULES_DIR, module), TARGET_DIR)
    path_to_target = os.path.join(TARGET_DIR, module)

    os.symlink(path_to_module, path_to_target, True)

Benefits:

1. Independent structure of custom modules (as well as GDNative stuff).
2. Self-contained. All code related stuff can be reproduced at different versions. Docs, conceptual designs may remain outside code structure.
3. Keep track of Godot version used to develop the project seamlessly, no need to worry about that something may break and you have to recover specific Godot version when using tools like git bisect for finding bugs.
4. Possible integration with scons to build Godot custom templates, tools, GDNative, and exporting project by using the same build system alone.

I feel like this could be part of documentation. Closing this as I've found the solution to my problem and so far it works, thanks for reading. 😛

[follower]

Your git repository can be initialized in the parent folder in the structure I exemplified, so you can have access to the commit hash from the Godot submodule.

I think I misread your message, I thought you suggested that having an ability to initialize Godot as a submodule outside the project is possible (which wouldn't make sense though):

git submodule add -f https://github.com/godotengine/godot ../godot
fatal: ../godot: '../godot' is outside repository

FWIW, I interpreted @akien-mga's structure suggestion as:

• My Project [directory] [git repository root]
  • Godot [directory] [git repository included as a submodule]
  • Game (with project.godot) [directory]
  • Design docs [directory]
  • Assets sources [directory]
  • Etc.

(i.e. the Game directory--with project.godot in it--is not a separate repository nor the root of the repository.)

[Xrayez]

@follower yeah this could work too, yet with the Game folder being already initialized as git repository this would involve moving files around and could turn out a lot of work to fix a simple nuance. 😅

[Xrayez]

Ok, I got this structure now, given all the suggestions:

So all files related to the actual build process are at the root, this allowed me to clean up the project itself and get rid of many .gdignore files. I guess I got a bit confused at first because I already had the main repository folder named as project which I renamed to source now, and moved actual project-related files to a new source/project. Parent folder (outside git repo) is dedicated to things not particularly related, like temporary files, sketch concepts, website links etc.

There's one inconvenience with this, namely if you symlink the modules from modules to engine/modules, git will constantly report that the godot repository has untracked content. Just add this line to locally ignore those modules:

# .git/modules/godot/info/exclude:
modules/