Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sprintf for floats on Arduino Mega? #293

Open
tofrnr opened this issue Jun 12, 2016 · 57 comments
Open

sprintf for floats on Arduino Mega? #293

tofrnr opened this issue Jun 12, 2016 · 57 comments

Comments

@tofrnr
Copy link

tofrnr commented Jun 12, 2016

could we finally have sprintf for floats on Arduino Mega?
enabled perhaps by a #include or by a #pragma config

@Chris--A
Copy link

To keep things consistent, this would probably have to be added to the Uno and other small variants. Otherwise code will fail to work correctly between them. This has been discussed before I'm pretty sure, and I think the overhead would be undesirable for the smaller boards. There are already floating point conversions in both the Print class and String class. If both are used, then there is a substantial code size increase.

If there is a pragma for this option, you cuold simply add it to your sketch when you need it.

If you're interested, I have a library PrintEx in the library manager which has sprintf and a printf for Print objects. It also supports repition, reading from PROGMEM, and EEPROM.

Using a GString object you can also use print/printf functions and streaming to write to RAM variables. Which is far more efficient that grabbing in the whole sprintf function (depending on how much of its functionality you use).
.

@tofrnr
Copy link
Author

tofrnr commented Jun 13, 2016

my question was about to be optionally (!!) enabled perhaps by a #include or by a #pragma config.

Having to deal currently with the mutilated AVR-sprintf-crippel stuff and those unconscionable ftoa functions is an impertinence, so every programmer should be empowered to decide by his own if he wants the full ANSI C /stdio.h functionality or not (like on the Due, and the Mega is powerful enough for that purpose, too). And finally the original stdio.h-sprintf version is already existing and available, so why not to be simply - optionally - enabled for Sketch?

@matthijskooijman
Copy link
Collaborator

Enabling float support for printf is currently only possible by passing some compiler flags to the linker, which is not currently possible from within a sketch. This feature is disabled by default to save space.

@tofrnr
Copy link
Author

tofrnr commented Jun 13, 2016

re: "This feature is disabled by default to save space."

Of course I know that, and that's already been said.
Workarounds like mentioned or e.g., patching all those lib.c files is also no option on the duration.

So my post was meant to be a request for a new additional feature.

@tofrnr
Copy link
Author

tofrnr commented Jul 20, 2016

now what about making possible that sprintf() can be optionally (!!) enabled perhaps by a #include or by a #pragma config ??

@matthijskooijman
Copy link
Collaborator

now what about making possible that sprintf() can be optionally (!!) enabled perhaps by a #include or by a #pragma config ??

I cannot think of any way to do this, without having to make significant changes to gcc and/or avr-libc...

@tofrnr
Copy link
Author

tofrnr commented Jul 21, 2016

Perhaps have just two sets of different lib.c folders?
and then choose manually in the proprietary which set the user wishes to be #included.

either by #include an extended lib manually:

#include <libfloat.h> // switches from lib.c ctandard to libcfloat (extended)

or., e.g., set a #define:

#define libcfloat // switches the libs for the linker automatically

or., e.g., set a #pragma:

#pragma config  libcfloat

which does the same automatically by preprocessor, compiler and linker.

Finally when patching all lib.c files it is actually already possible to enable sprintf also for floats for all AVRs!.
But then it's irreversible for small Unos, Minis, Micros and Nanos which is also unreasonable.

@oqibidipo
Copy link

oqibidipo commented Jul 21, 2016

Add this to boards.txt and you can select printf and scanf version from the Tools menu:

##############################################################

menu.printf=AVR printf Version
menu.scanf=AVR scanf Version

uno.menu.printf.default=Default printf
uno.menu.printf.default.avr_printf_flags=
uno.menu.printf.full=Full printf
uno.menu.printf.full.avr_printf_flags=-Wl,-u,vfprintf -lprintf_flt
uno.menu.printf.minimal=Minimal printf
uno.menu.printf.minimal.avr_printf_flags=-Wl,-u,vfprintf -lprintf_min

uno.menu.scanf.default=Default scanf
uno.menu.scanf.default.avr_scanf_flags=
uno.menu.scanf.full=Full scanf
uno.menu.scanf.full.avr_scanf_flags=-Wl,-u,vfscanf -lscanf_flt
uno.menu.scanf.minimal=Minimal scanf
uno.menu.scanf.minimal.avr_scanf_flags=-Wl,-u,vfscanf -lscanf_min

uno.compiler.c.elf.extra_flags={avr_printf_flags} {avr_scanf_flags}

mega.menu.printf.default=Default printf
mega.menu.printf.default.avr_printf_flags=
mega.menu.printf.full=Full printf
mega.menu.printf.full.avr_printf_flags=-Wl,-u,vfprintf -lprintf_flt
mega.menu.printf.minimal=Minimal printf
mega.menu.printf.minimal.avr_printf_flags=-Wl,-u,vfprintf -lprintf_min

mega.menu.scanf.default=Default scanf
mega.menu.scanf.default.avr_scanf_flags=
mega.menu.scanf.full=Full scanf
mega.menu.scanf.full.avr_scanf_flags=-Wl,-u,vfscanf -lscanf_flt
mega.menu.scanf.minimal=Minimal scanf
mega.menu.scanf.minimal.avr_scanf_flags=-Wl,-u,vfscanf -lscanf_min

mega.compiler.c.elf.extra_flags={avr_printf_flags} {avr_scanf_flags}

# ... repeat for each board

(Full version attached.)

avr_printf_scanf_2.txt

@tofrnr
Copy link
Author

tofrnr commented Jul 21, 2016

I just need the functionality for sprintf and possibly vsprintf and sscanf,
i.e. just for formatting different value types to a char* (string).

  • not for read/write from stdin/stdout or from/to files (printf, scanf, fprintf, fscanf,...)

But your list does not mention sprintf and either string formatting function so I'm curious if it will solve the sprintf issue?

(BTW, for the DUE sprintf and vsprintf work fine even for floats, just not for AVRs)

@oqibidipo
Copy link

It covers the whole printf (and scanf) family of functions.
vfprintf and vfscanf are the functions that do the actual work, others are more or less just wrappers for them.

@tofrnr
Copy link
Author

tofrnr commented Jul 21, 2016

ok,
once having done that for 1 of my projects, how can I switch back and forth to include/exclude this functionality if desirable?

@oqibidipo
Copy link

Just select Default printf / Default scanf if you don't need floating point support.

@tofrnr
Copy link
Author

tofrnr commented Jul 21, 2016

how can I do it in a couple of programs and do the other thing in another couple of programs from inside my code?

as I wrote in my TOP:
e.g., enabled perhaps by a #include or by a #pragma config

  • or even by a different approach ?

one time I must be able to exclude the full sprintf functionality optionally because of memory reasons, and the other time I must be able to have this sprintf feat. floats formatting optionally when I need this functionality !

@bperrybap
Copy link

@tofrnr
You don't seem to be understanding that the way this is controlled is through compiler and linker flags.
That means that there is nothing you can do inside a source code module to control this.

Could it done differently, perhaps. It might be possible to have played some games with some weak symbols to affect link ordering to trick the linker to select different libraries by using a dummy symbol in the application code. However, that is now how the AVR libC is implemented and it is very unlikely something that Arduino development team would ever want to take on.

So basically there are a hand full of approaches.

  • Create custom board types that control the compiler/linker options
  • Modify the build recipes to permanently enable/disable AVR floating point support
  • Modify the IDE to control the compiler/linker options through some sort of GUI preference option similat to the way verbose mode is handled.

None of these will give you control from the application source code.

As an alternative you might want to look at the PrintEx library.
It offers most of the xxprintf() capability as well as some extensions and works with Arduino Print class objects as well.
Currently, I'm in the process of working with the Author to simplify how the object initialization works to avoid creating a separate object so you will be able to do things like:
objname.printf(......)
and get what you want for any object that uses the Print class without having to use a separate PrintEx object.
It will essentially transparently extend the objects methods.
(see the github issues list for the PrintEx library for that discussion)

It is actually a much nicer solution than using the xxprintf() routines and will work consistently across all platforms with a smaller footprint than using the libC xxpprintf() routines, particularly on some of the other (non AVR) platforms.

@tofrnr
Copy link
Author

tofrnr commented Jul 23, 2016

re: "You don't seem to be understanding that the way this is controlled is through compiler and linker flags."
I know that it's the way it's done currently, but that should be changed for the future:

perhaps by a #pragma config which changes the compiler and linker flags or by doing different magic things, or a
#include of a 2nd library like the already existing lib.c float patch lib for example: then one could switch between the default lib.c libs and the float-patched lib.c libs.
This lib.c float patch finally is also already existing - you know that lib.c float patch, I suppose?

Anyway, oqibidipo's proposal sounded as if perhaps he might have found a workaround nevertheless.

So my wish is still the same, i.e. to change the current implementation into a more suitable one:

"could we finally have sprintf for floats on Arduino Mega?
enabled perhaps by a #include or by a #pragma config"

@bperrybap
Copy link

bperrybap commented Jul 23, 2016

@tomFr,
It appears that you still may have some misunderstands how compiling, linking, and linking against libraries works and what is possible through using #pragma as you keep suggesting the same things over and over that simply cannot be used to do what you wanting.

I don't know of a way to do what you are asking for without having to modify the gcc tools which is the same comment @matthijskooijman made earlier.

Also, consider that using the floating point xxprintf() libraries over the reduced libraries is a cost of about 1.8k of flash space when floating point formatting is not used.
It isn't that much.
I'd suggest that you consider turning on the floating point support all the time and not look back particularly since you are talking about using this on the Mega and are likely to have the extra 2k of flash to use.

I think it is reasonable to request that there be a way to enable xxprintf() floating point support from the IDE. - Perhaps through an option just like the verbose mode. Or perhaps through a free form option text box that can be passed through to the compiler and/or linker.

But I believe to ask for the capability for this to be controlled by an individual application source code module is a step too far.

Think about what you are asking. You are asking for the ability of an individual source code module to affect the compiler and/or linker options used to compile that module (as well as potentially affecting options used for other modules) and to control the linker options that are used when linking together all the modules and libraries.
That capability doesn't exist today in gcc tools and I'm not sure if could technically ever work.
It would require putting linker flag information in the .o file so that the flags could be "remembered" and used at "link" time since the compiler tools and the linker tools are all separate programs.
There are many issues with that type of control, including what would happen if there were conflicting linker options in multiple objects?
And how could that ever work when doing partial or staged linking?

There are are already multiple pre-built libraries for the xxprintf() support. They come with the avr libC libraries that come from the avr gcc package included in the IDE but is not maintained by the arduino development team. The way the various xxprintf() libraries are selected is through linker flags.
The @oqibidipo solution is in the list of approaches I suggested earlier. It uses custom board types to control the compiler/linker flags. While that works, it doesn't scale very well since it requires creating new board types for every single board and it does not give per application control since that simply is not possible as selecting which libraries to link against is outside the control of the application source code.

Like I said earlier it might be possible to play some games using weak symbols or symbol renaming pragmas to trick the linker into pulling in a specific xxprintf() library. But then you would have to go in and modify the avr libC code for those libraries, rebuild all the libraries to create a new avr libC package to put into a new avr-gcc package and update all the associated documentation. And you would have to convince Atmel or whoever is maintaining the AVR libC libraries and avr-gcc packages to do it since I doubt arduino developers would want to take on that task (assuming it is even possible).

If you want to get a flavor of the technical possibility of doing what you are wanting, or to see if there is any other interest in it, I suggest that you venture over to the AVRfreaks site and start a thread over there. That is where the hardcore AVR tool folks and avr-gcc developers hang out.

@tofrnr
Copy link
Author

tofrnr commented Jul 23, 2016

well, actually I have no idea HOW it could work, I'm just interested in THAT it does work how I stated:

to optionally enable sprintf() for float formatting also on AVRs, as it always works on the Due, and especially just that for the Mega.

As sprintf for floats is the standard in C (stdio.h), it would work always that way on either platform when stdio.h once was #included.
Thus #including stdio.h would provide full stdio.h functionality to either stdio.h function, especially also for sprintf , and especially for float formatting, whoever wants it,, and whenever he wants it.

OTOH, whoever needs a stripped-down, crippled formatting function on AVrs using less memory space, might #include a different stripped-down lib in his program, perhaps one could create a "avrio.h" lib for that purpose.

Just in case, one was not enable to automatize that by #defines or #ifdefs or #pragmas or whatever by the wirng libs like

#ifdef __AVR_WHATEVER_FLAG__
#include <avrio.h>
#elif
#include <stdio.h>
#endif 

But in doubt, stdio.h is the standard and so sprintf should then work always like the standard, also for AVRs, not other way round.

@bperrybap
Copy link

But you are asking for more than having float formatting in xxprintf() functions on AVRs optionally work.
You are asking for floating point formatting support to be selectively enabled & disabled from a application source module.
No libC library on any target platform that I have seen and worked with in over 35 years, including Due offers that.
On all other platforms floating point format support is enabled all the time or you link against separate libraries just like the way the AVR libC works.

You are also asking for a capability that is not compatible with the way code is compiled and linked.
It is kind of like the many requests people make for the ability to put a define inside a source module and control the compilation of a library.
There are certain things that are not possible given how compilation units and libraries work.
And that is the problem.
You want something that is WAY outside the norm and is likely technically impossible to do.

@tofrnr
Copy link
Author

tofrnr commented Jul 23, 2016

then AVR programmers with a lack of RAM should #include a different lib without sprintf,
and whoever wants full sprintf for floats should #include stdio.h providing automatically all the standard sprintf features.

Just in case no genius invents another more sophisticated method.

@bperrybap
Copy link

@tofrnr
Huh? Your most recent comment doesn't make sense.

This is not a RAM issue.
The only difference between the xxprintf() library that includes float support and the one that does is code size which is in flash not RAM. The floating point formatting version is quite a bit larger and pulls in several floating point library functions.

This is not an issue using a library with or without sprintf() or even a sprintf() issue.
sprintf() is as simple wrapper function for the underlying vfprintf() code and exists in every version of the various AVR libC libraries.
The vfprintf code is compiled in different ways and used to create different libraries.
You link against the different libraries to get which ever version of the pre-compiled code you want.

Here is a link to the SVN repository so you can look at the xxprint routines for yourself:
http://svn.savannah.nongnu.org/viewvc/trunk/avr-libc/libc/stdio/?root=avr-libc

@tofrnr
Copy link
Author

tofrnr commented Jul 23, 2016

" the overhead would be undesirable for the smaller boards"
" This feature is disabled by default to save space."

@tofrnr
Copy link
Author

tofrnr commented Jul 23, 2016

sprintf() is a stdio.h function and must work as the default, i.e. providing floats formatting features.

If one wants a crippled version for AVRs to save space, he must be able to use a different function with a minor functionality - either explicitely or automated by flags or #pragmas or #includes or whatever.

But if one uses sprintf() he must be able to expect and to have (at least optionally) ALWAYS the full functionality, i.e. like described here:
http://www.cplusplus.com/reference/cstdio/sprintf/

format
C string that contains a format string that follows the same specifications as format in printf (see printf for details).
http://www.cplusplus.com/reference/cstdio/printf/

@Chris--A
Copy link

But if one uses sprintf() he must be able to expect and to have (at least optionally) ALWAYS the full functionality, i.e. like described here: http://www.cplusplus.com/reference/cstdio/sprintf/

But on AVR-GCC you are using this: http://www.nongnu.org/avr-libc/user-manual/group__avr__stdio.html#gaa3b98c0d17b35642c0f3e4649092b9f1

And all its variants/limitations are listed there. AVR-GCC is a non standard implementation because it does not support/contain many things such as a complete 'C++ standard library' or exceptions. It implies from the beginning that everything inside it might not be to standard, including the expectation for some difference in the things that are provided.

@bperrybap
Copy link

" the overhead would be undesirable for the smaller boards"
" This feature is disabled by default to save space."

But like I said earlier the space saved is in flash not ram as you said:

then AVR programmers with a lack of RAM should #include a different lib without sprintf,

And you don't #include a library. That isn't how libraries and linking works.
A #include like stdio.h merely declares things like various functions; it does not actually define them.
http://www.cprogramming.com/declare_vs_define.html
The actual code is off in other separately precompiled modules and in this case there is more than one pre-compiled image that can be pulled in at link time depending on the compiler and linker flags.

In terms of how reduced functionality in a function works:

If one wants a crippled version for AVRs to save space, he must be able to use a different function with a minor functionality - either explicitely or automated by flags or #pragmas or #includes or whatever.

That isn't how reduced functionality works.
The function name is the same, the feature set & capabilities of the function is what changes not the name of the function.
i.e.the application code will always call the same sprintf() function regardless of whether the xxprintf() code has floating point support or not.

And the capability to use different versions of the xxprintf() code already exists. Compiler & linker flags are used to control which version of the xxprintf() code you get. There are different pre-compiled versions of the code and you use the compiler & linker flags to select which version you get. It is not possible to select which library object is used from a source module.

But if one uses sprintf() he must be able to expect and to have (at least optionally) ALWAYS the full functionality, i.e. like described here:
http://www.cplusplus.com/reference/cstdio/sprintf/
Like @Chris--A said, you can't get all the standard xxprintf() formatting functionality because the AVR version doesn't have everything. But the larger versions for the code have almost everything and definitely has floating point support.

So being able to get nearly all the xxprintf() formatting capabilities ALL the time is already possible. You don't seem to like the way it works.
Remember there are pre-compiled versions of this code with different capabilities.
All you can do is tell the linker which one you want . This selection is made at link time.
That is WAY after any compilation is done.

The default version of the xxprintf() code selected at link time happens to be a reduced feature set with without floating point formatting because the vast majority of the users of this code have wanted it this way as it saves flash space.
You can select which version of the xxprintf() code you get based on the flags passed into the compiler & linker.
When using the Arduino IDE, you can set those flags by creating custom board types or by modifying the build recipes.

If you want to ALWAYS have floating point formatting capabilities go in and modify your build recipes for AVR builds.
Just be aware that you will pay a just under 2k price in flash even if not using floating point formatting since the xxprintf() code to support floating point formatting will be there regardless of whether you use it.
But it is one time thing to make this change and after that, you can get the xxprintf() code you want ALL the time.

@tofrnr
Copy link
Author

tofrnr commented Jul 24, 2016

I don't feel like reading your posts as long enough to sink a ship.

sprintf() is a standard C command provided by stdio.h, providing well-defined features

OTOH:
How currently sprintf is implemeneted by Arduino Sketches is illogical and inadequate.

Whoever wants sprintf in Sketches must be able to have it, with full funcionality, even and especially with float formatting support, at any time. Finally this is standard C syntax.

If it would be needed to #include stdio.h for that purpose, and no other magic automated way, ok, provide it by an explicite #include < stdio.h >

Who might not want sprintf() because of memory reasons is not forced to #include stdio.h, then he can (and must ) do without sprintf and may save RAM.
Perhaps he may use special shrinked Arduino-AVR-Libs, BUT NOT sprintf() ! .
Perhaps for those purposes one could automatically or manually #include a avrio.h lib providing a different command, e.g. avrprintf() or sformat() or whatever.

Anyway, sprintf() is expected to always being able to work standard-C-like, without compromising, at any time, for all processors, e.g. also for the MEGA, period.

@Chris--A
Copy link

@tofrnr

could we finally have sprintf for floats on Arduino Mega?
enabled perhaps by a #include

I have made an update to my library. sprintf now works with floats.

  1. Download PrintEx from the library manager (Make sure you get version 1.2.0 of the library).

  2. Use sprintf:

    #include <PrintEx.h>
    
    char buffer[100];
    
    void setup(){
      sprintf( buffer, "This is a float: %07.3f", 23.45f );
    
      Serial.begin(9600);
      Serial.println(buffer);
    }
    
    void loop(){}
    

Prints:

This is a float: 023.450

This solves the problem using a single #include.

@tofrnr
Copy link
Author

tofrnr commented Jul 24, 2016

no, definitely: no extra lib! perhaps #include < stdio.h >, yes!

sprintf from stdio.h is perfect, there is no need to change that.
One just must be able to have it's complete functionality when ever one wants it!
On either platform!

But what perhaps might have to be changed is:
Provide another downstripped sformat program tool for AVRs (for those who do not want float formatting).

But don't ever mutilate the original sprintf() from stdio.h !!!
Let sprintf() ALWAYS have ALL it's capabilities, i.e. also everything for float formatting, also for AVRs and especially for the MEGA !!

(and BTW, I never use the library manager, it always muddles everything up. I just copy libs manually to the Arduino libs folder)

@bperrybap
Copy link

@tofrnr
Well, I'll make one final post.
You seem to have many misunderstandings about many things including:

  • how code is compiled and linked in the gcc environment (which is the same as most development systems.)
  • how the IDE works to build sketches
  • how libraries work
  • the difference between Arduino "libraries" which are not real libraries and real libraries like avr libC that come with the gcc toolset.
  • RAM vs FLASH usage

For example, if you were not using the Arduino IDE, and using makefiles instead, then changing the compiler and linker flags would be trivial as you could go in and modify the makefile for that sketch.

And you keep shifting between what you want.
In some cases you say you want full formatting support in sprintf() all the time:

Anyway, sprintf() is expected to always being able to work standard-C-like, without compromising, at any time, for all processors, e.g. also for the MEGA, period.

and

But don't ever mutilate the original sprintf() from stdio.h !!!
Let sprintf() ALWAYS have ALL it's capabilities, i.e. also everything for float formatting, also for AVRs and especially for the MEGA !!

and in other cases you say you want to be able to control the capabilities of what is in the sprintf() code based on flags inside the application module:

could we finally have sprintf for floats on Arduino Mega?
enabled perhaps by a #include or by a #pragma config

and

... to be optionally (!!) enabled perhaps by a #include or by a #pragma config.

You can't have it both ways and because of this, it is very difficult to have a technical conversion.

The reality is that the way compilers and linkers work and the way the avr-gcc package is put together, it includes different pre-compiled versions of the xxprintf() code and you tell the linker which one you want. The default happens to be a reduced functionality version but that can easily be changed by using compiler & linker flags.

The Arduino IDE currently has no way to set those compiler & linker flags from the GUI as a configuration option similar to the way it can set the compiler warning flags through a "verbose" mode option.
If this issue was a feature request for that capability to be added to the GUI, then that would make sense and could become reality since it would not be difficult to do.

In the absence of having a IDE configuration option to set these flags from the GUI,
people have told you multiple ways of how to change things to give you the full formatting capabilities in the xxprintf() code including with sprintf() so that support for floating point formatting is enabled all the time or could be changed using custom board types.
You didnt' like it.

And then when @Chris--A offered a simple solution that offered EXACTLY what you asked for in your original post,

... enabled perhaps by a #include ...

to give you full sprintf() support by simply including a header file.
You didn't like that either, saying you didn't want to install another library.

You don't like the way things currently work, but you don't seem to want to change anything to get the capabilities you want.

Well, the reality is that unless you install a different or additional library or specify some command line options you can't get a different version of sprintf() than the default which is current the reduced capability sprintf() without floating point formatting support.

You seem to have some desires to do things in ways that are not supported by the existing tool sets.
The good news is that the gcc tools and the avr libC code is all open source so you can go in an modify the tools to your liking.

But I think it would be much simpler and whole lot less work, to simply reconfigure a few small things to get the desired capabilities.

If you feel strongly enough about it, then by means go over to AVRfreaks site:
http://www.avrfreaks.net/
and start a thread and try to convince the people that wrote and maintain the avr gcc tools that what they have been doing for years is wrong and that they should change to doing things some other way.

@tofrnr
Copy link
Author

tofrnr commented Jul 25, 2016

I didn't read your novel, it's simply too many words - just this:

The reaseon why I don't like the idea of printEx which mimics stdio.h is that I am extremely skeptical if it will really objectively mimic all and everything of the stdio.h features then too, additionally, like listed here:
http://www.cplusplus.com/reference/cstdio/printf/

But at long last, Arduino simply must not mutilate standard C functions and then keep it in that crippled version.

i.e., stdio.h functions must provide all stdio.h functionalities!

If Arduino wants to provide mutilated functions, then Arduino must call them differently or build in a compiler switch that one is able to switch between the full and the mutilated version.

In that case it means:

  • sprintf MUST work like in standard C
  • sprintf MUST match stdio.h and so MUST have float formatting features (beyond all the other things)
  • if AVR users want a stripped-down mutilted non-compliant string formatting function there must be another, additional command, if not switchable.

@bperrybap
Copy link

bperrybap commented Jul 25, 2016

@tomfmr
Clearly you don't understand how this stuff works and the real world constraints when working within an embedded environment vs a system with a full blown operating system.
stdio is a HUGE i/o system and could NEVER be fully supported on the AVR nor on most embedded platforms.
All the sprintf() capabilities you want already exist.
You are complaining about how it is implemented and what the defaults are and how to change it from the defaults.
avr libC which includes sprintf() is not written by or maintained by arduino.cc so
if you want the defaults changed or how it works changed, then like I said before go over to the avrfreaks site and try to convince them that the way they have done it is wrong and that they should be doing it the way you want - which I'm still not sure what that is.

And now I'm out as this discussion is not advancing and is simply spinning in circles.

@bperrybap
Copy link

As I mentioned earlier, it would be possible to set the needed compiler/linker options from the GUI just like there is way to set the compiler options to turn on more verbose warnings. It is a small change and should be fairly isolated.
To me, that would be a legitimate feature request.
But we have to keep in mind that this is just an issue for the AVR cores.
Other cores don't have this issue so that makes it a bit tricky.

The GUI could also be changed to have a text box where the user could optionally type in additional desired command line arguments. Perhaps a couple of different boxes for the different tools (pre-processor, compiler, linker, etc...) This method could be used for other things besides just the selection of which xxprintf() routines to use.
Yes this information should be in the recipe files but this would be a convenient way to change the flags from the GUI.
Maybe the GUI could have an option to modify certain fields in the platform file?

@tofrnr
Copy link
Author

tofrnr commented Jul 25, 2016

yes, that sounds promising!
Remember that this is only a AVR issue as the ARM powered Arduinos (e.g., DUE) already always provide the full sprintf float capability.

@bperrybap
Copy link

@tofrnr
What sounds promising?

@tofrnr
Copy link
Author

tofrnr commented Jul 25, 2016

a way to perhaps resolve this issue from out of the GUI / IDE.

@bperrybap
Copy link

I suggested that earlier in my first response in this issue/thread (a toggle to enable/disable the floating point formatting on AVR processors) but you rejected it because it didn't (couldn't) provide a mechanism to do the selection from the source code modules.

@oqibidipo
Copy link

"How to do the impossible" a.k.a. #pragma AVR printf float, proof of concept:

  1. Add a hook to scan the sketch for #pragma AVR printf and save corresponding options to options_file.
    More on hooks introduced in IDE 1.6.5.
  2. The compiler, linker and other tools support reading command line arguments from a file by using @filename (e.g. gcc @options -c ...)

platform.local.txt (IDE 1.6.6 or later):

recipe.hooks.linking.prelink.1.pattern=perl /Users/rjl/perl/arduino_avr_printf_scanf_magic.pl {build.path}/preproc/ctags_target_for_gcc_minus_e.cpp {build.path}/{build.project_name}.avr_magic_linker_options
# just in case the above fails
recipe.hooks.linking.prelink.2.pattern=touch {build.path}/{build.project_name}.avr_magic_linker_options

recipe.c.combine.pattern="{compiler.path}{compiler.c.elf.cmd}" {compiler.c.elf.flags} -mmcu={build.mcu} {compiler.c.elf.extra_flags} @{build.path}/{build.project_name}.avr_magic_linker_options -o "{build.path}/{build.project_name}.elf" {object_files} "{build.path}/{archive_file}" "-L{build.path}" -lm

arduino_avr_printf_scanf_magic.pl.txt

@tofrnr
Copy link
Author

tofrnr commented Jul 25, 2016

bperrybap, I am a hobbyist and simply could not imagine to do that configuration/switching from out of the IDE unless then doing it out of the source code. Anyway, I'm gladly looking forward!

@bperrybap
Copy link

@oqibidipo
Clearly the IDE can go in and lots of things as it pre-process the .ino sketch file when turning it into a .cpp file; however, those types of methodologies will make the code that uses it tied to using the new versions arduino.cc IDE that have this capability which can cause code to break when being built with other build environments.
In general I'm not a fan of these types of approaches.

It seems like a lot of work when there are other alternatives that already work with existing IDE versions that could offer what is needed (possibly not all of what is wanted) at a much lower development and support cost that don't wonk up the IDE code with new Arduino specific pre-processing extensions.

I think a better option (and probably the best at the current time) for the majority of users would be to simply include something like the board type menu extensions you proposed earlier in the stock AVR boards.txt file that comes with the IDE. This would give the user build control over which xxprintf() code to use using menus in the IDE GUI when building the sketch on all the AVR based arduino boards that arduino.cc includes in the IDE "out of the box".
This would cover most users since most users don't install other AVR cores and use the AVR stock IDE board types that come with the IDE.

Using the menu capability in board types requires no changes to any IDE code and offers the user the ability to select the desired xxprintf code to use from a simple GUI selection menu when he selects the target board for the build.
I have no idea why @tofrnr hated it - other than it doesn't offer the ability to control the option from the source code module.
It is actually quite nice and very easy to use.
I'm going to go talk to Paul and see about getting this added to his Teensyduino package so it at least these xxprintf menu options will be in the Teensy AVR based board types.

In the mean time, users could add it to their boards.txt file themselves. it takes just a minute or so and would be a simple cut and paste operation.
To help users get it installed into existing IDEs, all that would be necessary would be a playground page that has the instructions for how to add it and then a pointer to the actual file (with support for all the arduino.cc boards) to insert into their boards.txt file.

Baring that, using the PrintEx library offers and easy way to get better sprintf() or even a printf() method within a device library class on a per sketch basis.

@Sonophoto
Copy link

Just for the record, #PRAGMA is exactly the way you control compiler flags from a source file but it is up to the compiler to support it. HOWEVER, IMHO it is probably best to control it from a build system of some sort not the source file.

@oqibidipo Thank you very much for this solution Risto, it appears to work great on Arduino IDE 1.8.2 / Linux Mint / Mate / JDK 8

@bperrybap
Copy link

@Sonophoto
To change which xxprintf() code is used, requires changing a linker flag not a compiler flag.
Which takes us back to my original comments. This is all already doable. If you are not using the IDE, then just change your makefile and use whatever linker flags you want. If using the IDE, I offered several solutions that can be done today with existing IDEs which allow selecting the flags from the IDE GUI.

@Sonophoto
Copy link

@bperrybap -L -l and -Wl<,linker,options> are gcc compiler flags that control the linker.

You did not offer the solution (!?) not sure where you came up with that idea. @oqibidipo offered the correct solution for the Arduino IDE and it was posted a day before you entered the conversation.

@bperrybap
Copy link

technically "gcc" is really a wrapper not the actual compiler. gcc typically invokes the pre-processor, the compiler and the linker.
And I did talk about solutions. 3 to be exact. 2 of them don't require changing the IDE.
one was creating custom board types that could even use menus to set gcc flags.
The other was to go modify the recipe file to permanently change the type of xxprintf() code used and linked in.

@Sonophoto
Copy link

@bperrybap You just have to keep fucking arguing don't you? Giving everyone a hard time, trying to act like you have some unique technical knowledge the rest of us don't have or that the rest of us don't really know what is going on.

@tofrnr
Copy link
Author

tofrnr commented Mar 11, 2018

as to me, patching system files are not very much appreciated, because for every IDE update everything had to be patched again.
Instead, a way to have the sprint float thing arbitrarily on demand and by a very simple IDE add-in or setting in the preferences would be much more appreciated.
But tbh, I don't see such a way so far.

@bperrybap
Copy link

@tomFr,
The bottom line given today's state of Arduino and AVR gcc toolset s/w, is that if you want to use floating point output on the AVR processors in Arduino sketches, you will have to make some kind of changes to the environment since the AVR gcc toolset ships with a default of linking in a reduced xxprintf() to save space - because AVRs don't have the code space that many of the newer microcontrollers have and the Arduino IDE developers have decided not to provide a way to change this from the GUI.
Yes you can create custom board types to alter the flags, to allow the GUI to adjust the link options, but you didn't seem to want to do that.
If you always have the spare code space (about 2k) and want the full xxprintf() with floating point support all the time, the modification to the AVR platform file is quite minimal. It would only need to be done one time after the IDE is installed/updated - but you didn't seem to want to that way either.

I think an easy alternative solution that works quite well with Arduino and the existing Arduino IDEs is to use the PrintEx library as originally suggested quite some time ago.
Using PrintEx does not require using pragmas which are very platform specific, or making changes to the IDE or any of the IDE bundled libraries, or boards.txt file or to the platform.txt file
It is an easy way to be able to have the floating point formatting capabilities available based on the source code of the sketch,
PrintEx adds a xxrintf() methods like printf() to the library object which fits right into the Arduino i/o model.
i.e. you can use mydevice.printf() to output formatted text.
No need or use of sprintf() or extra buffers, and it can be used to extend existing objects.
Also, because PrintEx was designed for Arduino, and takes advantage of some of the built in formatting capabilities of the Print class, it will uses less code than using the full blown AVR libC xxprintf() code.
Since PrintEx is an Arduino library, it is also portable across all Arduino cores so it works not only for AVR but all the other cores.
While PrintEx is not a 100% xxprintf() implementation, it is fairly complete and robust and useful for most situations.
This would seem to give you what you are wanting.
Is there some specific reason why this solution seems to have been rejected?

@tofrnr
Copy link
Author

tofrnr commented Mar 14, 2018

it' because I write code which is intended to work both for the Due and for the Mega and for esp8266 (and perhaps in future also for the M0 pro), using float-to-string formatting by sprintf as already provided for the Due. The Due does not require PrintEx, the Mega works faulty for floats, for the esp8266 it's probably not fixed yet and I'm no quite sure about the M0 pro.
So the goal is: to have identical, universal stdio.h-like xxprintf libs which works for all, just like on the Due, nothing additional, and no patched libs which would vanish for either future or reinstalled Arduino IDE version.

@bperrybap
Copy link

bperrybap commented Mar 14, 2018

The AVR sprintf() code is not faulty for floats. The avr-gcc AVRlibc supports configuring which xxprintf() code is linked in. By default it links in a smaller reduced functionality version because many of the AVR chips are very code limited and the majority of users wanted it that way. While it is configurable, you can't configure it from the Arduino IDE.
I don't like the way AVRlibc did it either. There are other ways it could have been handled like using a different stdio include header file to manipulate which symbols were generated to control which library code was used/linked in.
I tried to make this happen years ago with the avr-gcc maintainers but could not get any traction.
Given how easy it is to specify linker options when using real Makefiles the effort to change AVRlibc to be smarter to allow selection from the source code was deemed not worth the effort.

We have to keep in mind that avr-gcc is outside the control of Arduino.

While we can ask for some kind of support to be added to the Arduino IDE GUI to configure AVR xxprintf() support, that may be a long time coming.

Until there is something else, you will have do something to get floating point support.
Several options were put forward

  • create board types
  • use pragmas
  • use a library like PrintEx (which is cross platform),
  • patch patch the platform.txt file to permanently enable full xxprintf() code

Given your new stated goal/requirements of always wanting full xxprintf() support, I'd recommend that you go in and patch the AVR platform.txt file to default to the full xxprintf() code and be done with it.
(I've done this on several of my development environment environments)
If you are always building for mega and/or don't care about a few extra k of code being used even if floats are not used by the xxprintf() formatting, this will be the simplest solution.
Tweaking platform.txt can turn on full xxprintf() support for the AVR.
No other core has a reduced xxprintf() implementation so it is only something that has to be done for AVR and it only has to be done once when the IDE is installed or updated.
It would literally take like 30 seconds to a 1 minute to do, if that long, to go in and edit the platform.txt to make the needed changes.
After that, you will always get full xxprintf() support just like on all the other cores/platforms.
Yeah it is a pain to do each time the IDE is updated, but it isn't that bad since it only has to be once for each install/update.

@tofrnr
Copy link
Author

tofrnr commented Mar 14, 2018

as already stated, I do not want to patch system files or create board types, that is too cumbersome.
xxprintf will have to work for floats out of the box, without external libs like PrintEx - just the AVR xxprintf restrictions must be able to be dropped arbitrarily (or reactivated for an Uno or Micro if there will be a need to do it).
I actually would appreciate something which works out of the IDE, e.g. by a predefined #pragma, or by a checkbox, or a radio button in settings/preferences, or in preferences.txt.

@per1234
Copy link
Contributor

per1234 commented Mar 14, 2018

It seems like this discussion is getting really redundant (and some of it was already gone over years ago in arduino/Arduino#1719). Please keep it productive or I'll need to lock the thread. Saying the same things over and over is just a waste of time.

@bperrybap
Copy link

@pert234 I totally agree.
I think the main issue has been that main things desired and being asked for (using pragma or and include) are simply not possible given the way the tools a libraries work.
And any changes to make either of those work, is in gcc itself, or AVRlibC which is WAY outside the scope and control of Arduino.

@tofrnr
Copy link
Author

tofrnr commented Mar 15, 2018

I don't see why it shouldn't be possible to choose the libs I want to use by simple #includes:
if I need the C functions of xxprintf, I normally #include <stdio.h>,
If I need cin or cout then I normally #include <iostream>,
If I need atof or atoi then I normally #include <stdlib.h>,
if I need numeric limits then I #include <limits>
if I need digitalRead then I #include <wiringPi.h> (on Raspbian)

so if there is low space at a certain cpu (e.g., Uno) it should be possible to have a restricted stdio lib version, and then just
#include <stdio_restricted.h>,
otherwise (like for the Mega or already for the ARM cpus), the standard enhanced version <stdio.h> has to be #included.

If I do that by my gcc for Linux, then I do not even need a linker flag for those standard lib functionalities, I just use
g++ -o file file.c

But even if a linkerflag was needed additionally like for wiringPi.h in my example above, why couldn't that flag (e.g., -lwiringPi) be added then by the IDE either automatically or by a setting in preferences, depending on the required lib name ? That that would be "WAY outside the scope and control of Arduino" is honestly WAY beyond my scope of understanding.

@facchinm
Copy link
Member

@tofrnr includes are not linker flags; if you want a constructive discussion, please keep quiet.
The latest builder, included in the Beta release, already contains 99% of what you are requesting.
I'm attaching a sample library that simply enables printf_flt when included; the only modification needed at core level is

diff --git a/platform.txt b/platform.txt
index 16c25b1..729b426 100644
--- a/platform.txt
+++ b/platform.txt
@@ -65,7 +65,7 @@ archive_file_path={build.path}/{archive_file}
 recipe.ar.pattern="{compiler.path}{compiler.ar.cmd}" {compiler.ar.flags} {compiler.ar.extra_flags} "{archive_file_path}" "{object_file}"
 
 ## Combine gc-sections, archives, and objects
-recipe.c.combine.pattern="{compiler.path}{compiler.c.elf.cmd}" {compiler.c.elf.flags} -mmcu={build.mcu} {compiler.c.elf.extra_flags} -o "{build.path}/{build.project_name}.elf" {object_files} "{build.path}/{archive_file}" "-L{build.path}" -lm
+recipe.c.combine.pattern="{compiler.path}{compiler.c.elf.cmd}" {compiler.c.elf.flags} -mmcu={build.mcu} {compiler.c.elf.extra_flags} -o "{build.path}/{build.project_name}.elf" {object_files} "{build.path}/{archive_file}" "-L{build.path}" {compiler.ldflags} -lm
 
 ## Create output files (.eep and .hex)
 recipe.objcopy.eep.pattern="{compiler.path}{compiler.objcopy.cmd}" {compiler.objcopy.eep.flags} {compiler.objcopy.eep.extra_flags} "{build.path}/{build.project_name}.elf" "{build.path}/{build.project_name}.eep"

Unfortunately, the current ldflags patch has some limitations, especially on how the property name was chosen. You can keep track of the discussion here arduino/arduino-builder#256 and here arduino/ArduinoCore-samd#278 .

If you want to push for the patch to enter mainline, test it, report your findings and contribute to the discussion, thanks.
PrintfCompleteAVR.zip

@tofrnr
Copy link
Author

tofrnr commented Mar 15, 2018

@tofrnr includes are not linker flags; if you want a constructive discussion, please keep quiet.

What??
1st, I was the one who opened this issue, and 2nd, I never wrote that "#includes are linker flags"!!

All I said was that some #includes of libs work without any additional linker flag by gcc/g++ (e.g., stdio.h or iostream) , while some other #includes of libs do need additional linker flags (like for the wiringPi lib example) in order to locate the libs , to link them and "make" the executable.

And I said that choosing a arbitrary lib functionality is normally possible by
a) #include <libname> and
b) perhaps additionally have to add some unique explicit -lxxxxx a/o -Iyyyyy linker flags for make/makefile

@matthijskooijman
Copy link
Collaborator

It would be good if you could choose this using a #include, but as explained a few times already, this is not currently possible due to the way avr-libc has implemented floats in printf.

If Arduino starts supporting custom, per-sketch compiler options (which it should at some point, but is non-trivial, and not the topic of this issue), enabling float support could be made convenient. Until then, some workarounds have been shown in this issue, feel free to use any of them.

As @per1234 indicated, this issue is repeating arguments and not progressing anymore. Since @tofrnr is coninuing to repeat things again, I'll go ahead and lock this issue from further discussion.

@arduino arduino locked and limited conversation to collaborators Mar 15, 2018
@matthijskooijman
Copy link
Collaborator

@sandeepmistry, I think this issue should be moved to ArduinoCore-avr.

@arduino arduino unlocked this conversation Oct 1, 2019
@sandeepmistry sandeepmistry transferred this issue from arduino/Arduino Oct 1, 2019
@arduino arduino locked and limited conversation to collaborators Oct 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

8 participants