Please explain why you consider the existing solution of using relative paths to be "not great".

Relative paths themselves are not the problem (surely they are used in many projects out there) but rather the fact that only the src/ directory can be used as the reference for those relative paths.

The goal is to be able to include other open-source projects in Arduino libraries without modification - thereby greatly lightening the burden of maintenance.

Lets imagine that there are two great projects that should be included in an Arduino library. Their structure is not flat and header files may be located in one or more sub-levels. Each project might have a structure similar to this:

- projN
	- subA
		- headA.h
	- subB
		- headB.h
	- projN.h		(#include "headA.h" and #include "headB.h")
	- projN.cpp

AFAIK there is no way to use this project without modifying it - even if you placed the contents of projN under the library's src/ directory the Arduino build system could not find headA.h or headB.h. The source code of the (third-party) project would need to be modified to include headers via relative paths.

Maybe it is best practice for a project like projN to use relative paths internally so that only one include path is needed in the build system but unfortunately we can't enforce that on third-party projects. So the alternative is to maintain a modified fork/copy or to allow additional include paths to be added to the build system.

Is there another way that I am overlooking?

Hi @oclyke ,
I indeed like the idea of allowing a library to specify additional include paths; it should also be totally safe (in fact, we are only recursively compiling inside the src folder, not outside).
My proposal:

• we can't use include= because it's too similar to includes= (https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#libraryproperties-file-format) ; maybe additional_include_paths= is a bit better?
• the additional include paths shouldn't be used during library discovery to avoid polluting the search results but only later in the process.

I'll try to prototype something and let you know 😉

Wow fantastic! Can't wait to test drive. I've never worked on the IDE source code but let me know if there's anything I can do to help.

@oclyke sorry for the delay. I'm attaching the builds of arduino-builder for the most common platforms to #502

the additional include paths shouldn't be used during library discovery to avoid polluting the search results but only later in the process.

What does this mean exactly? The environment for include detection and compilation should be as identical as possible, otherwise you might end up with different results and missing includes during include detection?

Do you maybe mean that these additional include paths should only be used when compiling source files from the library itself? If so, the include path should be added only for this library's source files, when compiling and when doing include detection I think. However, I'm not so sure this would really work: If the sketch includes a library header file that wants to include a header file from the additional include path, this would break. Because of this, in practice you pretty much always need a single, global, list of include paths used in all compilations.

The file ossdl.c will most likely include the open-source portable interface with a line such as: #include "ossdl.h" -- unfortunately this will not work in Arduino 1.8.10

@oclyke Are you sure this will not work in Arduino? When you include with "" rather than <>, the compiler should resolve the include relative to the current file, so you should be able to include from the current directory.

This does not solve the more complex problem when you have two immutable sublibraries of which one needs to include the other, so the additional include paths can still be useful.

On a related note: Maybe it would be interesting to also have an option to not put the src directory in the include path at all. This would allow libraries with a separate include directory for their public interface, and still allow "private" headers in their src/ directory that can be included with relative includes (with "").

With avoid polluting the search space I was thinking about this usecase (let's call it "Arduino style explicit library request" )

#include "foo.h"
#include "bar.h"
void setup() {}

Then, foo.h is discovered in Foo library with the following structure

.
|-- keywords.txt
|-- library.properties
`-- src
    |-- foo.cpp
    |-- foo.h
    `-- libbar
        |-- include
        |   |-- another_include.h
        |   `-- bar.h
        `-- src
            `-- bar.c

Foo library adds libbar/include to additional_include_paths and this stops the library discovery there. Since the user wanted to add bar.h from Bar library (in sketchbook) the compilation is now broken.

Now, there are a couple of real life things to keep in mind:

• if I had to write a library wrapper I'd love keeping the library I'm wrapping as-is (to allow git submodules or similar)
• if all libraries were written with relative includes, this would work out of the box.
• if I want to wrap an existing library I'm going to provide to the user higher level APIs and provide them in the "first level" search directory.

Hope it's clearer now, but I'd love this thing to be ready soon and having a larger audience can help in finding the ideal solution.

@facchinm I'm a bit confused with your example. The sketch includes "bar.h", the quotes indicate that this file exists in the sketch? If so, there is no conflict there, as the quotes make sure that the current directory is searched first, so the file from the sketch is found. If they wanted to include bar.h from the Bar library, it would have been better to write #include <bar.h> instead. In any case, if the sketch contains no bar.h, then either one has the same result.

Furthermore, you seem to suggest a conflict between bar.h in the Bar library and the include in libbar, but in your example you only have a libbar.h, so again, no conflict there?

Assuming that you intended no bar.h in the sketch, and intended libbar.h to be named bar.h, then indeed, there is a conflict between libbar/include/bar.h and Bar/bar.h, but I don't think there is a way to solve this.

Your suggestion seems to be to only put libbar/include on the include path when compiling foo.cpp and libbar/src/bar.cpp? If so, this would break if the sketch includes foo.h, and that does #include <bar.h> (intending to include libbar/include/bar.h). This could of course be solved by doing a relative include (#include "libbar/include/bar.h") instead (which is a good idea in any case, since it disambiguates), but does not solve everything. Consider the following library:

.
|-- keywords.txt
|-- library.properties
`-- src
    |-- foo.cpp
    |-- foo.h
    `-- libbar
        |-- include
        |   `-- bar.h
        `-- src
            `-- bar.c
    `-- libbaz
        |-- include
        |   `-- baz.h
        `-- src
            `-- baz.c

Now, suppose that libbar depends on libbaz and libbar/include/bar.h contains #include <baz.h>. If the sketch is compiled without additional library paths, and it includes foo.h which includes libbar/include/bar.h which tries to include <baz.h>, which is not in the include path and fails. Since libbar/include/bar.h is immutable, we cannot fix this using a relative include.

I guess this means the perfect solution does not exist (the problem, really, is that "includes" in C/C++ inhabit a global namespace and can cause conflicts). The traditional way to solve this is to do have a single (or a few) include dir instead of one per library and doing things like #include <libbar/bar.h> where the libbar part provides some namespacing, but that seems infeasible with the way Arduino works right now (in fact, it is actively prevented, since library autodetection does not work on files in subdirectories, so you cannot even emulate this behaviour with a subdirectory inside src currently).

Given there will be some limitations either way, I think I would prefer to keep the include namespace simple and flat, and let people deal with duplicate header files (which then need to do already, if multiple libraries provide a bar.h file, they might run into trouble), rather than letting them deal with inconsistent results for inclusion ("it works when I include file from my library .c file but not from the .h file, huh?") and not supporting multiple interdependent sublibraries.