Library configuration file management (issue #1734)

[GDV0]

When an application uses a library supporting configuration:

• the library configuration file is stored in the sketch folder as a
  project ressource.
• a sketch tab is created in Arduino IDE to edit the library
  configuration file

This commit manages following actions:

• create configuration file tab when opening a sketch which uses a
  Library configuration file
• copy the configuration file in the sketch folder and create a
  configuration file tab when importing a library which uses a library
  configuration file
• add the library configuration file (from sketch) as include file when
  compiling the library only

Added as an example, Morse library supporting configuration file

[bkconrad]

Hi, could someone revisit this PR? This functionality is necessary for anyone who wants to provide a library configurable through compiler definitions (e.g. to turn off features for the sake of code size), and this PR seems like a good framework for per-sketch configuration in general.

[ArduinoBot]

Can one of the admins verify this patch?

[Diaoul]

+1

[ricardojlrufino]

+1

[ffissore]

Sorry for the late reply. Would you please rebase your PR on top of current master and provide the example Morse library as a zip file or separate repo, without including it in the PR? Otherwise we had to remove it right after merge

[ffissore]

On second thought, we probably don't want such a feature at the moment. Having a separate configuration file may be confusing and lead library developers rely too much on #defines instead of providing a clearer API. In case of the Morse library, you could provide a richer constructor or (better) several overloaded constructors. For example

Morse(int pin); //would set variables as in https://github.com/arduino/Arduino/pull/1808/files#diff-5b136a0d870efacf6b332258c2258b1d
Morse(int pin, uint8_t morseUnit); //would set just morseUnit and use default multipliers for the other values
Morse(int pin, uint8_t morseUnit, uint8_t morseDuration, uint8_t dashDuration, etc...); //for a complete configuration

[cmaglie]

Isn't the compiler able to remove the unused methods from the final output? In your Morse library I guess that you can remove the define LIB_MORSE_ALPHABET_AVAILABLE and leave the methods always defined.

[tzapu]

@ffissore Will this be added in 1.6.5? planned release date for 1.6.5?

Cheers

[svatoun]

Well, the reasoning about clean API is good - but for example now, the lack of #defines made me to fork the standard Keypad library, since I'm memory constrained, and the Keypad does not use memory effectively at all. #define could allow to limit size of the object to the actually used number of switches (16 instead of 160), reducing memory consumption about ~40 (sorry, correction) bytes. Alternatives are quite bad: using new operator (= malloc library = +200bytes for sketch, fragmentation) or user-supplied buffers (error prone). Or templatized Keypad - very ugly and not understandable code.
Second, optimization requires some backward incompatibilities - again the user is not able to choose the tradeoff - speed (program code) over memory without #define -- this time without any possible alternative.

[siara-cc]

I strongly recommend re-opening this discussion, as I am facing this issue with my Sqlite arduino library, which is only a wrapper to the original sqlite code.

As a library developer, I wouldn't want to be the one to decide which sqlite feature should be included or omitted. A per sketch configuration would allow the user of my library to turn on/off whatever feature he wants.

I am pasting below my config_ext.h (which I am having to supply with the library now) to illustrate the issue.

#define YYSTACKDEPTH                        20
#define SQLITE_SMALL_STACK                   1
#define SQLITE_DEFAULT_PAGE_SIZE          4096
#define SQLITE_MAX_EXPR_DEPTH                0

#undef SQLITE_OMIT_ALTERTABLE
#undef SQLITE_OMIT_ANALYZE
#undef SQLITE_OMIT_ATTACH
#define SQLITE_OMIT_AUTHORIZATION            1
#undef SQLITE_OMIT_AUTOINCREMENT
#define SQLITE_OMIT_AUTOINIT                 1
#define SQLITE_OMIT_AUTOMATIC_INDEX          1
#define SQLITE_OMIT_AUTORESET                1
#define SQLITE_OMIT_AUTOVACUUM               1
#undef SQLITE_OMIT_BETWEEN_OPTIMIZATION
#define SQLITE_OMIT_BLOB_LITERAL             1
#define SQLITE_OMIT_BTREECOUNT               1
#define SQLITE_OMIT_BUILTIN_TEST             1
#define SQLITE_OMIT_CAST                     1
#define SQLITE_OMIT_CHECK                    1
#define SQLITE_OMIT_COMPILEOPTION_DIAGS      1
#define SQLITE_OMIT_COMPOUND_SELECT          1
#define SQLITE_OMIT_CONFLICT_CLAUSE          1
#undef SQLITE_OMIT_CTE
#define SQLITE_OMIT_DECLTYPE                 1
#define SQLITE_OMIT_DEPRECATED               1
#undef SQLITE_OMIT_DISKIO
#define SQLITE_OMIT_EXPLAIN                  1
#define SQLITE_OMIT_FLAG_PRAGMAS             1
#define SQLITE_OMIT_FOREIGN_KEY              1
#define SQLITE_OMIT_GET_TABLE                1
#define SQLITE_OMIT_INCRBLOB                 1
#define SQLITE_OMIT_INTEGRITY_CHECK          1
#undef SQLITE_OMIT_LIKE_OPTIMIZATION
#undef SQLITE_OMIT_LOCALTIME
#define SQLITE_OMIT_LOOKASIDE                1
#undef SQLITE_OMIT_MEMORYDB
#undef SQLITE_OMIT_OR_OPTIMIZATION
#undef SQLITE_OMIT_PAGER_PRAGMAS
#define SQLITE_OMIT_PARSER_TRACE             1
#undef SQLITE_OMIT_PRAGMA
#define SQLITE_OMIT_PROGRESS_CALLBACK        1
#define SQLITE_OMIT_QUICKBALANCE             1
#undef SQLITE_OMIT_REINDEX
#define SQLITE_OMIT_SCHEMA_PRAGMAS           1
#define SQLITE_OMIT_SCHEMA_VERSION_PRAGMAS   1
#define SQLITE_OMIT_SHARED_CACHE             1
#define SQLITE_OMIT_TCL_VARIABLE             1
#define SQLITE_OMIT_TEMPDB                   1
#define SQLITE_OMIT_TRACE                    1
#undef SQLITE_OMIT_TRIGGER
#define SQLITE_OMIT_TRUNCATE_OPTIMIZATION    1
#define SQLITE_OMIT_UTF16                    1
#undef SQLITE_OMIT_VACUUM
#undef SQLITE_OMIT_VIEW
#undef SQLITE_OMIT_VIRTUALTABLE
#undef SQLITE_OMIT_WSD
#define SQLITE_OMIT_XFER_OPT                 1
#define SQLITE_PERFORMANCE_TRACE             1
#define SQLITE_OMIT_COMPLETE              1
#define SQLITE_OMIT_SUBQUERY              1
#define SQLITE_OMIT_DATETIME_FUNCS        1
#define SQLITE_OMIT_FLOATING_POINT        1
#define SQLITE_COUNTOFVIEW_OPTIMIZATION      0

@ffissore @svatoun @cmaglie @tobozo

[ricardojlrufino]

I think it should be possible to create new header file (e.g. MyRunningMedian.h) in your project, containing just your changed definitions:

#ifndef MEDIAN_MAX_SIZE
#define MEDIAN_MAX_SIZE     30
#endif

Then, you need to pass this header file to preprocessor, to include it as a first #include for the RunningMedian. You can do this by using -include directive, see https://gcc.gnu.org/onlinedocs/gcc/Preprocessor-Options.html.

E.g. in PlatformIO you can do it in platformio.ini this way:

[common]
build_flags = 
    -include "include/MyRunningMedian.h"

from: https://stackoverflow.com/questions/42151118/how-to-override-define-in-arduino-library-header

[tobozo]

unfortunately this isn't a platformio problem and Arduino IDE has no such build_flags mechanism that can be used from the project folder itself

[ricardojlrufino]

unfortunately this isn't a platformio problem and Arduino IDE has no such build_flags mechanism that can be used from the project folder itself

sorry, i forgot to mention it was just for reference