Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare for release v1.8.7 #2735

Closed
dpgeorge opened this issue Dec 28, 2016 · 19 comments
Closed

Prepare for release v1.8.7 #2735

dpgeorge opened this issue Dec 28, 2016 · 19 comments

Comments

@dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Dec 28, 2016

The December release was skipped, but there have been enough changes to warrant a new release ASAP and it would be good to do it in the first week of January 2017. The biggest change was the update of the HAL for stmhal, and bugs introduced by that have now been fixed (all the ones I know of).

Since there were no big user-facing changes going up a sub-minor version to v1.8.7 makes sense.

@pfalcon

This comment has been minimized.

Copy link
Contributor

@pfalcon pfalcon commented Dec 29, 2016

Sounds good. I can think of 2 things worthy to fit in the release:

  • Make cc3200 deployable with cc3200tool #2627
  • Perhaps, make unix uselect changes as discussed in micropython/micropython-lib#141 (comment), as it's getting harder and harder to sync micropython and micropython-lib package releases. (50%)
@dpgeorge

This comment has been minimized.

Copy link
Member Author

@dpgeorge dpgeorge commented Dec 30, 2016

Perhaps, make unix uselect changes as discussed in

It would be ideal to use extmod/moduselect.c for the unix port, but that would come at the expense of efficiency because the poll/select call becomes essentially a busy-wait loop.

Also, #2672 (esp native code location) would be good to have done.

@pfalcon

This comment has been minimized.

Copy link
Contributor

@pfalcon pfalcon commented Dec 30, 2016

It would be ideal to use extmod/moduselect.c for the unix port, but that would come at the expense of efficiency because the poll/select call becomes essentially a busy-wait loop.

I'm writing a long update to #1550, so long, that I couldn't even finish it in one night, and my current idea (still vague) is that vice-versa, it would be better for extmod/moduselect.c to adopt a design of unix', i.e. use an adhoc table to hold poll descriptors instead of mp_map_t. Anyway, I'll post it in some time, and we can consider what's the next logical step.

Also, #2672 (esp native code location) would be good to have done.

Well, all infra was done for that. Feel free to modify flashbdev.py to include a setting to offset start of FS (I suggest using a generic name like RESERVED_SECS), then just add a docs para that native code cache area starts at esp.flash_user_start() and of size 4K. (Maybe we should rename esp.flash_user_start() to something shorter, I had esp.fw_end() in mind).

@dpgeorge

This comment has been minimized.

Copy link
Member Author

@dpgeorge dpgeorge commented Jan 4, 2017

Feel free to modify flashbdev.py to include a setting to offset start of FS

Done in bae7798.

add a docs para that native code cache area starts at esp.flash_user_start() and of size 4K

The esp.set_native_code_location() func is documented by c3f70c6.

Maybe we should rename esp.flash_user_start() to something shorter, I had esp.fw_end() in mind

I kinda got used to flash_user_start. It's self documenting in that it tells you you can start using the flash from this offset.

@pfalcon

This comment has been minimized.

Copy link
Contributor

@pfalcon pfalcon commented Jan 5, 2017

Is there specific ETA for the release? It would be nice to get #2747 in and fix improperly working hwapi examples.

@dpgeorge

This comment has been minimized.

Copy link
Member Author

@dpgeorge dpgeorge commented Jan 5, 2017

@pfalcon there's #2743, and also what do you think of bae7798#commitcomment-20367399

@pfalcon

This comment has been minimized.

Copy link
Contributor

@pfalcon pfalcon commented Jan 5, 2017

Will respond/act on that, wonder how much more beyond that would fit.

@dpgeorge

This comment has been minimized.

Copy link
Member Author

@dpgeorge dpgeorge commented Jan 5, 2017

wonder how much more beyond that would fit.

Not much. I have the release almost ready to go, just want to make sure flashbdev changes to esp8266 are ok and then release should happen tomorrow.

@dpgeorge

This comment has been minimized.

Copy link
Member Author

@dpgeorge dpgeorge commented Jan 6, 2017

With all the latest additions since the last release (biggest was probably xtensa native support) the esp-extra-scripts branch no longer fits into the irom0 segment (being 0x87000 bytes in size). It overflows by about 2k.

The quick-fix is to move all the scripts from scripts/ to modules/, because frozen bytecode is smaller (about 60% of) plain frozen text. Actually, not all need to be moved to make enough room, just the symlinked scripts that are added as part of esp-extra-scripts.

But there won't be enough room to reduce the firmware size down by a sector (to make room for native code) as suggested here bae7798#commitcomment-20371419

For the release I suggest moving all symlinks from scripts/ to modules/ to make enough room for the firmware. Regarding RESERVED_SECS, it would be good not to require users to re-build their filesystem, so maybe revert this change (just do RESERVED_SECS=0) for this release...

@pfalcon thoughts?

dpgeorge referenced this issue Jan 6, 2017
Starting at esp.flash_user_start(), the reserved sectors are for general
purpose use, for example for native code generation.  There is currently
one sector reserved as such.
@pfalcon

This comment has been minimized.

Copy link
Contributor

@pfalcon pfalcon commented Jan 6, 2017

For the release I suggest moving all symlinks from scripts/ to modules/ to make enough room for the firmware. Regarding RESERVED_SECS, it would be good not to require users to re-build their filesystem, so maybe revert this change (just do RESERVED_SECS=0) for this release...

Well, I guess it's good enough compromise, due to echo effect, it will take some time before people will start to play with native code...

@dpgeorge

This comment has been minimized.

Copy link
Member Author

@dpgeorge dpgeorge commented Jan 6, 2017

Ok, I did as said above: change RESERVED_SECS to 0, updated docs, and moved all script symlinks to modules/ for esp8266 port (remaining flash space in esp-extra-scripts branch build is about 3k). esp-extra-scripts branch has been rebased and force pushed.

Release notes have been written. It just remains to do a bit of testing on esp8266 to make sure the last-minute changes didn't break anything.

@mzdaniel

This comment has been minimized.

Copy link
Contributor

@mzdaniel mzdaniel commented Jan 7, 2017

MicroPython v1.8.6-302-gef1bbad-dirty on 2017-01-07; ESP module with ESP8266

Compilation went well (just a few warnings here and there). In the gist https://gist.github.com/mzdaniel/f7784768054300b2ab885e2ed32c9fda are the results of running the tests in micropython unix and esp8266. For testing in esp8266, I run in the repl micropython-lib/uasyncio/example_http_server.py (modifying the server socket to 0.0.0.0:8081). With that config, esp8266 pull 1000 queries with no concurrency, and 10 queries with 2 concurrent requests.

Will be nice to streamline testing in the esp8266. Happy to help.

@pfalcon

This comment has been minimized.

Copy link
Contributor

@pfalcon pfalcon commented Jan 7, 2017

@mzdaniel : Standard way to test micropython (any port of) is using tests/run-tests script. See http://forum.micropython.org/viewtopic.php?f=16&t=1731 for more info.

@mzdaniel

This comment has been minimized.

Copy link
Contributor

@mzdaniel mzdaniel commented Jan 7, 2017

Fantastic! Added esp8266 run-tests to the gist. 473 passed, 35 skipped, 3 failed: viper_binop_arith viper_binop_multi_comp viper_misc.

@pfalcon

This comment has been minimized.

Copy link
Contributor

@pfalcon pfalcon commented Jan 7, 2017

Ack, I get the same results, with 3 viper tests failing. @dpgeorge : I'm not sure if you plan to fix those, but I'd say as that's new and niche feature, ok to make a release with those failures.

@dpgeorge

This comment has been minimized.

Copy link
Member Author

@dpgeorge dpgeorge commented Jan 8, 2017

I'm not sure if you plan to fix those, but I'd say as that's new and niche feature, ok to make a release with those failures.

They fail only because they run out of iram1 memory to store the native code. Using flash to store native code, and these viper tests all pass (at least they did the last time I checked). So there's not much to do to fix them right now.

Otherwise, testing some more things on esp8266 I found an issue with import of frozen packages, now fixed in b528e9a.

BTW, @mzdaniel , thanks for testing!

@dpgeorge

This comment has been minimized.

Copy link
Member Author

@dpgeorge dpgeorge commented Jan 8, 2017

v1.8.7 was released and tagged in 5653e3c

@dpgeorge dpgeorge closed this Jan 8, 2017
@pfalcon

This comment has been minimized.

Copy link
Contributor

@pfalcon pfalcon commented Jan 8, 2017

Thanks!

@mkarliner

This comment has been minimized.

Copy link

@mkarliner mkarliner commented Jan 8, 2017

Congratulations!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.