Updated SDK with enhancements

[iseries1]

Can hide unit from programmer by use of web page.

[PropGit]

There are a lot of changes in this PR. @dbetz can you help review this? The comment seems like a sensible enhancement and most of the changes are including a new/update SDK. I don't know if this has ramifications on memory or existing features.

[dbetz]

It looks like most of the changes are related to moving to a new SDK. Can the author verify that? I'm actually not sure why we checked in the SDK as part of the repository anyway. Did I do that?

[PropGit]

@dbetz - yes; I found (at least) this commit from 2016

Add SDK .bin files that were left out of the original push because
.bin is in .gitignore.

Modify the SDK path in release/Makefile.

The .bins must be necessary for this project?

[dbetz]

Ugh. It would have better if I had created another project with all of the SDK files. Anyway, I'll merge your changes later tonight after work. Thanks for doing this!

[VonSzarvas]

Has anyone downloaded this from scratch and tried compiling ?
Seems to be a bunch of duplicate headers and a couple undefined references.

espmissingincludes.h can be fixed by commenting out these items:

#if 0 // Remove headers already included in latest SDK

void ets_install_putc1(void *routine);
void ets_isr_attach(int intr, void *handler, void *arg);
size_t ets_strlen(const char *s);
int ets_strncmp(const char *s1, const char *s2, int len);
void ets_timer_arm_new(os_timer_t *a, int b, int c, int isMstimer);
void uart_div_modify(int no, unsigned int freq);
void ets_delay_us(int ms);
#endif

#if 0 // Remove headers already included in latest SDK
void *pvPortMalloc(size_t xWantedSize, const char *file, int line);
void *pvPortZalloc(size_t, const char *file, int line);
void vPortFree(void *ptr, const char *file, int line);
void pvPortFree(void *ptr, const char *file, int line);
#endif

[dbetz]

I haven’t had time to try it. I thought it had been tested already and I didn’t see any problems in a quick glance over the source changes. Does it build after your changes?

…

On Sep 2, 2020, at 2:15 AM, Michael ***@***.***> wrote: Has anyone downloaded this from scratch and tried compiling ? Seems to be a bunch of duplicate headers and a couple undefined references. espmissingincludes.h can be fixed by commenting out these items: #if 0 // Remove headers already included in latest SDK void ets_install_putc1(void *routine); void ets_isr_attach(int intr, void *handler, void *arg); size_t ets_strlen(const char *s); int ets_strncmp(const char *s1, const char *s2, int len); void ets_timer_arm_new(os_timer_t *a, int b, int c, int isMstimer); void uart_div_modify(int no, unsigned int freq); void ets_delay_us(int ms); #endif #if 0 // Remove headers already included in latest SDK void *pvPortMalloc(size_t xWantedSize, const char *file, int line); void *pvPortZalloc(size_t, const char *file, int line); void vPortFree(void *ptr, const char *file, int line); void pvPortFree(void *ptr, const char *file, int line); #endif — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFL5VWIC63SDS2GBZD327LSDXPGTANCNFSM4QNC3MPQ>.

[VonSzarvas]

Thanks David, yeah it looks clean. I'm guessing it's a library version mis-match.

The latest...:
From the command line (and with the mentioned edits) the current Parallax-ESP seems to build, except when opening the wifi.html page in the esp8266 web browser, it crashes the module (it reboots- I can see it in the debug output). The other pages open OK and seem responsive. Obviously the wifi module connects to my AP, such that I can connect to it, so wifi "kinda" works.

Also, this "error" crept back in:
libesphttpd/mkupgimg.c
~ln 93
printf("Header: %d bytes, user1: %d bytes, user2: %d bytes.\n", sizeof(hdr), (int)l1, (int)l2);
should be:
printf("Header: %ld bytes, user1: %d bytes, user2: %d bytes.\n", sizeof(hdr), (int)l1, (int)l2);

Maybe the esp sdk version changed from what iSeries has locally, to what Git is pulling today, on a full fresh install.

Another annoying thing... now my NetBeans won't compile. I get this:
make[1]: xtensa-lx106-elf-gcc: Command not found
Makefile:219: recipe for target 'build/espfs/espfs.o' failed
make[1]: *** [build/espfs/espfs.o] Error 127

xtensa-lx106-elf-gcc is in the OS path, and in netbeans toolchain (although make overrides that anyway). Probably I changed something in make file to make that work before, but have forgotten what that was.

The XTENSA_TOOLS_ROOT and CC values are correctly set in makefile, so not that. And make runs from the command line anyhow, so seems to be a netbeans thing. Argh! I'm so glad the wifi Zoom demo starts in about 8 hours :) Best I revert back to an older version I think!

[dbetz]

One thing that worried me a little about the changes is that one of the buffer sizes was increased from 1k to 4k and I seem to recall that RAM is at a premium on the ESP8266.

…

On Sep 2, 2020, at 6:05 AM, Michael ***@***.***> wrote: Thanks David, yeah it looks clean. I'm guessing it's a library version mis-match. The latest...: From the command line (and with the mentioned edits) the current Parallax-ESP seems to build, except when opening the wifi.html page in the esp8266 web browser, it crashes the module (it reboots- I can see it in the debug output). The other pages open OK and seem responsive. Obviously the wifi module connects to my AP, such that I can connect to it, so wifi "kinda" works. Also, this "error" crept back in: libesphttpd/mkupgimg.c ~ln 93 printf("Header: %d bytes, user1: %d bytes, user2: %d bytes.\n", sizeof(hdr), (int)l1, (int)l2); should be: printf("Header: %ld bytes, user1: %d bytes, user2: %d bytes.\n", sizeof(hdr), (int)l1, (int)l2); Maybe the esp sdk version changed from what iSeries has locally, to what Git is pulling today, on a full fresh install. Another annoying thing... now my NetBeans won't compile. I get this: make[1]: xtensa-lx106-elf-gcc: Command not found Makefile:219: recipe for target 'build/espfs/espfs.o' failed make[1]: *** [build/espfs/espfs.o] Error 127 xtensa-lx106-elf-gcc is in the OS path, and in netbeans toolchain (although make overrides that anyway). Probably I changed something in make file to make that work before, but have forgotten what that was. The XTENSA_TOOLS_ROOT and CC values are correctly set in makefile, so not that. And make runs from the command line anyhow, so seems to be a netbeans thing. Argh! I'm so glad the wifi Zoom demo starts in about 8 hours :) Best I revert back to an older version I think! — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFL5VUTPVRR3FHRZA62WZLSDYKFJANCNFSM4QNC3MPQ>.

[PropGit]

Has anyone downloaded this from scratch and tried compiling ?
Seems to be a bunch of duplicate headers and a couple undefined references.

Oh no! @VonSzarvas were you able to step back to pre-PR and build from there for the work you're doing? I should have postponed the merge of this PR but simply didn't think of the timing of your work in this case.

[VonSzarvas]

No worries @PropGit :)

Yes, I could step back a version, and all is working fine.

[dbetz]

Should we revert this PR? Is that even possible?

…

On Sep 2, 2020, at 9:20 AM, Michael ***@***.***> wrote: No worries @PropGit <https://github.com/PropGit> :) Yes, I could step back a version, and all is working fine. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFL5VXCJM5QMX2WVYUZBFTSDZBBBANCNFSM4QNC3MPQ>.

[VonSzarvas]

maybe, although @iseries1 may have suggestions to resolve just as quick.

Couple other things... the included SDK is now >v3, so a new function is needed to be added to user-main.c

something like this:

#if ESP_SDK_VERSION >= 030000
// user_pre_init is required from SDK v3.0.0 onwards
// It is used to register the parition map with the SDK, primarily to allow
// the app to use the SDK's OTA capability.  We don't make use of that in 
// otb-iot and therefore the only info we provide is the mandatory stuff:
// - RF calibration data
// - Physical data
// - System parameter
// The location and length of these are from the 2A SDK getting started guide
void ICACHE_FLASH_ATTR user_pre_init(void)
{
  bool rc = false;
  static const partition_item_t part_table[] = 
  {
    {SYSTEM_PARTITION_RF_CAL,
     0x3fb000,
     0x1000},
    {SYSTEM_PARTITION_PHY_DATA,
     0x3fc000,
     0x1000},
    {SYSTEM_PARTITION_SYSTEM_PARAMETER,
     0x3fd000,
     0x3000},
  };

  // This isn't an ideal approach but there's not much point moving on unless
  // or until this has succeeded cos otherwise the SDK will just barf and 
  // refuse to call user_init()
  while (!rc)
  {
    rc = system_partition_table_regist(part_table,
				       sizeof(part_table)/sizeof(part_table[0]),
                                       4);
  }

  return;
}
#endif // ESP_SDK_VERSION >= 030000

I've not checked the memory values are appropriate for the Parallax modules.
Or revert to the <v3 version.... if that would work, I don't know.

I'd think it could wait a few days- see if Mike has some advice.

[PropGit]

@dbetz - We can revert it, but let's wait on that for the time being as we're not hindered by it currently. If we can't resolve it later this week, we can always resort to reversion and revision.

[dbetz]

Is this a problem with using the current SDK vs the one I used ages ago when I wrote the original code? If so, we should probably resolve this issue and all move to the new SDK. Is there a reason not to?

…

On Sep 2, 2020, at 9:32 AM, Parallax Git Administrator ***@***.***> wrote: @dbetz <https://github.com/dbetz> - We can revert it, but let's wait on that for the time being as we're not hindered by it currently. If we can't resolve it later this week, we can always resort to reversion and revision. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFL5VX5R2QT23XJP6UNX4TSDZCP3ANCNFSM4QNC3MPQ>.

[iseries1]

I didn't use the current SDK because of all the changes that would be needed to get it to compile. I stuck with version 2.2 which was easer to get too.
I built a virtual server and built the code with no error after removing several lines from 'espmissingincludes.h'. I didn't check in libraries 'libesphttpd' and 'ESP8266_NONOS_SDK' as I didn't want to corrupt those libraries.
I fixed the WiFi router list bug and added some setting for hiding the ESP unit from the programmer.
I increased the buffer size for received data because I had a webservice call that returned lots of data and the 1024 buffer size was way to small. I also changed the receive function to allow the data to be returned in chunks so that I could process the data from the webservice.
I also added the UDP functions and calls.
I did notice that sometimes the WiFi page did not display. Since I didn't touch those function I figured it did that on the old version as well. I think it's a timing issue where the page times out before the data is returned. Anyway the page displays most of the time.
I have not tried to do a clean build so I may have missed something. I do have it building on a virtual server in the Azure cloud and have been using the firmware on inhouse modules for sometime now with no issues.

[VonSzarvas]

seems the answer is simple then.... to update Parallax-ESP github with those versions that iseries1 is using, for 'libesphttpd' and 'ESP8266_NONOS_SDK', and fix 'espmissingincludes.h' as I noted above.

@iseries1 Could you share the version numbers? Or are they the ones in your github copy ?

[iseries1]

I don't want to get into version numbers. I would leave it to your discretion.

[VonSzarvas]

ok, I'll step out of this conversation.

Ultimately this repo has to build when downloaded.
At the moment it doesn't. If Parallax want this repo to work, I guess they'll either have to reverse it to where it was, or figure out what versions work with iseries1's updates.

[dbetz]

I don’t understand. Why don’t you want us to know what version you’re using?

…

On Sep 2, 2020, at 12:21 PM, Mike ***@***.***> wrote: I don't want to get into version numbers. I would leave it to your discretion. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFL5VVTW5DLCNJZVD6Z7ODSDZWG3ANCNFSM4QNC3MPQ>.

[iseries1]

I was talking about what version of the firmware you will call it. Otherwise the only version I am using is with the SDK which is version 2.2.1 just before using 3.0 which requires some work to make it work on the ESP8266 unit. Partition table stuff.
This version included a number of include files that were missing from the old SDK that was used. So to make the compiler happy I removed those duplicates.

[dbetz]

Okay, my mistake. We should probably use the same version of the SDK as you’re using. I’ll admit that I haven’t built this in quite some time. However, I do have a reason to do it again soon as I have an application in mind for a variation of this firmware.

…

On Sep 2, 2020, at 1:05 PM, Mike ***@***.***> wrote: I was talking about what version of the firmware you will call it. Otherwise the only version I am using is with the SDK which is version 2.2.1 just before using 3.0 which requires some work to make it work on the ESP8266 unit. Partition table stuff. This version included a number of include files that were missing from the old SDK that was used. So to make the compiler happy I removed those duplicates. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFL5VV5IFECOTU5FQNX3MDSDZ3MJANCNFSM4QNC3MPQ>.