Refactoring looks nice and more clear. But what about back compatibility?

+1 looking nice!

Thank you @anakod! I tried hard not to break things and keep backward compatibility. However, only rboot roms are supported so far and much more testing will be required.

Hi @slaff, I like the idea of creating roms without bootloader! However, I'm not familiar with that option. Is there a sample linker file (e. g. for Basic_Blink)? Is there anything else that needs to be changed beside the linker file? I will try to include that option, too.

However, I do not know a replacement for this line
$(PERL) -ple "s{(^\s*irom0_0_seg *: *).*}{\\1org = $(IROM0_ORG0), len = $(IROM0_SIZE)}" $< >$@

If you explain what this perl code does then someone may be able to offer a python alternative. There are fewer developers who know both perl and python than know just one!

@riban-bw: This line is just a simple search and replace with a regular expression, something a *nix user probably would use sed for. However, I wanted to minimize the number of required tools for windows builds and used perl instead.

something a *nix user probably would use sed for.

Then, please, replace it with sed. Sed is also part of cygwin/mingw for Windows.

I removed the dependency on Perl and changed the directory handling to allow projects with multiple directories with duplicate names for source and header files. A new option EXTRA_SRC was added.

Now we have 3 main makefiles:

1. Makefile (to build Sming itself)
2. Makefile-project.mk (to build classic projects)
3. Makefile-rboot.mk (to build rboot projects)

• we have 4 additional system related Makefile (Win, Linux, MacOS, FreeBSD)

I don't think what we should add additional 4th Makefile. It should replace some of previous files instead.
You plan to implement only rboot build or classic projects also? Because I'm sure will be right to have single universal makefile instead of two independend for both project types.

Hi @anakod, thank you for your feedback. You are right, we should not have more makefiles but less. I did not plan to include support for the Espressif bootloader (classic projects) unless there is a real need (see discussion #971).

However, I really like @slaff's idea of builds without any bootloader and would like to include that as an option. Perhaps someone can give me a hint where to find the correct linker and flash settings for this?

Of course, this PR may also just serve as an inspiration for further development of the existing makefiles.

However, I really like @slaff's idea of builds without any bootloader and would like to include that as an option. Perhaps someone can give me a hint where to find the correct linker and flash settings for this?

@tius2000 If you compile the current Basic_Blink sample, or any other non-rboot projects, you will actually create a standalone application. The application comes with two ROM files, the one starting at 0x000 contains the .text .data .rodata sections (IRAM and DRAM) and the other one contains libraries/SDK code (SPI Flash )

esptool2 -quiet -bin -boot0 out/build/app.out out/firmware/0x00000.bin .text .data .rodata
# --boot0 above is for single, non-bootloaded apps.
esptool2 -quiet -lib out/build/app.out out/firmware/0x0a000.bin 

I did not plan to include support for the Espressif bootloader

That's right, we decide remove support for old Espressif bootloader.

(classic projects)

Classic project - I mean project without bootloader - single application splitted to two binaries as @slaff described above (or both binaries can be joined to single).

ANOTHER OPTION
Also we can drop support for applications without rboot bootloader at all. We can consider this option but:
1, We should provide solution compatible with old small 512 KB flash also
2. rBoot add additional delays in startup process. It can be really critical for battery powered deveices
3. In my opinion better to keep both options, it give us more flexibility

I don't think what current solution with two different makefiles is good, even if we skip question about code quality\comments.We have too many related issues during last month (just for example after moving debugf to irom and other).

WITHOUT causing "pain" to our users.

Yes, it's first priority.

Some thoughts on the refactoring:

I) We have to give a chance to the users to adapt to the new makefile re-organization.
Therefore the following could help:

• At the moment old and new makefiles are present. Please, replace the content of Makefile with the content from sming.mk and delete sming.mk.
• Rename Linux.mk to Makefile-Linux.mk and remove the old Makefile-linux.mk file. Apply the same to the other OS makefiles.
• For Makefile-rboot.mk and Makefile-project.mk should be present but having the following content:

1.) Info message $(info "...") saying that the Makefile-{rboot|project}.mk will be removed in the next versions of Sming
2.) Internally include the new makefile.

II) Issues

• Assember .S files are included always.

AS system/irq_check.s
AS gdb/gdbstub-entry.s

While this is true for irq_check.s gdbstub-entry.s should be included ONLY when ENABLE_GDB=1 is given.

• Compilation of the library with make -f sming.mk finishes with error:

sed -r "s/(^\s*irom0_0_seg *: *).*/\\1org = 0x4020A000, len = 0x0F6000/" /home/slavey/dev/esp8266.dev.box/dev/Sming/Sming//compiler/ld/rboot.rom0.ld >out/build/rom0.ld
LD out/build/app_0.out
out/build/app_app.a(user_main.o):(.text.user_init+0x4): undefined reference to `init()'
out/build/app_app.a(user_main.o): In function `user_init':
/home/slavey/dev/esp8266.dev.box/dev/Sming/Sming/appinit/user_main.cpp:70: undefined reference to `init()'
collect2: error: ld returned 1 exit status
make: *** [out/build/app_0.out] Error 1

That has to be fixed.

III) Makefile not in synch with the latest changes. I will write shortly comments next to the line numbers.

Dear @slaff, thank you very much for reviewing! I agree, there is still a long way before that should be merge.

However, I was wondering, whether marking the new makefile as "experimental" in the beginning might be an option. That way we will not break things, but users willing to try the new options can give it try. You could also place a hint with $(info) within the old makefiles to motivate the user to give it a try.

I'm prefer to replace old Makefiles with your version, but please take attention to II) and III) points in @slaff comment above.

Well, IMHO top priority should be not to break any existing projects. As I do not think that we can fix any possible incompatibility for every existing project before the first release, it might be a good idea to move to the new makefile in four phases.

phase 1:
Use the new name and mark the new makefile as "experimental". Include a message in the old makefiles to tell the users that there is something new.

phase 2:
Fix and improve the new makefile based on user feedback.

phase 3:
Mark the old makefiles as "deprecated" and inform the users, that they should switch to the new make process.

phase 4:
Remove the old makefiles or replace them with a stub pointing to the new makefile.

This way we do not break things and our users keep complete control of there projects. What do you think?

II) Issues

Assember .S files are included always.

AS system/irq_check.s
AS gdb/gdbstub-entry.s

While this is true for irq_check.s gdbstub-entry.s should be included ONLY when ENABLE_GDB=1 is given.

The whole gdb directory structure looks very confusing to me. makefile-rboot.mk references $(THIRD_PARTY_DIR)/gdbstub and I kept this. However, this directory does not exist on my system. I see copies of gdbstub-entry.s in Sming/gdb and Sming/third-party/esp-gdbstub. This is why the compilation goes wrong.

Which copy should be removed? Which directory should be compiled when ENABLE_GDB is set?

Compilation of the library with make -f sming.mk finishes with error

The new makefile is meant as a replacement for makefile-rboot.mk and _makefile-project.mk. It is not meant also to replace Makefile at the moment. However, this might be a task for a later phase.

The new makefile is meant as a replacement for makefile-rboot.mk and _makefile-project.mk

Perfect. Make sure to add an info message guiding our users.

Which directory should be compiled when ENABLE_GDB is set?

If you compile the library / or a project with ENABLE_GDB=1 then

1. there are new compile flags appended to the current ones.
2. The $(THIRD_PARTY_DIR)/gdbstub directory is added to the directories that include header files
3. The $(THIRD_PARTY_DIR)/gdbstub directory is added to the directories that contain source code and all (*.c, *.cpp AND *.s) files in it should be compiled.

else ifeq ($(ENABLE_GDB), 1)
	CFLAGS += -Og -ggdb -DGDBSTUB_FREERTOS=0 -DENABLE_GDB=1
	MODULES		 += $(THIRD_PARTY_DIR)/gdbstub
	EXTRA_INCDIR += $(THIRD_PARTY_DIR)/gdbstub
	STRIP := @true
else

If you compile the library / or a project with ENABLE_GDB=1 then

there are new compile flags appended to the current ones.
The $(THIRD_PARTY_DIR)/gdbstub directory is added to the directories that include header files
The $(THIRD_PARTY_DIR)/gdbstub directory is added to the directories that contain source code and all (*.c, *.cpp AND *.s) files in it should be compiled.

I kept this from makefile-rboot.mk. However, the path seems not to be correct. Is there something wrong with my setup?

So far in the latest develop we have the following implemented:

• roms with and without bootloader
• automatic flash address calculation + automatic linker script adjustment per application
• dependency tracking for all header files included in cpp and c sources ( that includes dependency tracking for spiffs sources )
• memanalyzer for rboot builds added

What is still missing is that one:

• automatic patch of esp_init_data_default.bin for system_get_vdd33()
  @tius2000 Is that still needed ? Can you refresh my memory and tell me why was that added and do we need to have it for all applications?

Hi @slaff, if you want to use system_get_vdd33(), you need to patch esp_init_data_default.bin, otherwise you will get wrong values.

you need to patch esp_init_data_default.bin, otherwise you will get wrong values.

We can think about patching the physical params by wrapping around register_chipv6_phy. This way we will have better control over the phy data and be better prepared for changes in the phy params in new SDKs.