Create Release and publish uf2 asset upon version tag creation. #198

awonak · 2022-11-27T20:43:29Z

Here is a successful run of the workflow:
https://github.com/awonak/EuroPi/actions/runs/3560476183/jobs/5980521188

The result of this workflow will create a new release with uploaded uf2 file:
https://github.com/awonak/EuroPi/releases/tag/v0.1.3

mjaskula

This looks really good to me. I just have a few questions/discussions to be resolved before approval, but overall really nicely done.

mjaskula · 2022-11-29T19:48:53Z

.github/workflows/compile_to_uf2.yml

+
+jobs:
+  compile-uf2:
+    runs-on: ubuntu-latest


[discussion] My preference in build scripts is to avoid unversioned and 'latest' dependencies. It is possible that a new version of a dependency will break our release. In that case, we'd rather deal with it when we are upgrading dependencies than when we are trying to create a release.

I would also apply this to:

apt-get packages

ssd1306

marvinpinto/action-automatic-releases

Spoken like a true SRE. I've locked action-automatic-releases and left the other two as-is. The ssd1306 repo doesn't have any tags and hasn't been updated in 4 years, and I don't have any concerns about apt-get build tools throwing surprises.

.github/workflows/compile_to_uf2.yml

mjaskula · 2022-11-29T19:55:34Z

.github/workflows/compile_to_uf2.yml

+          BootloaderMenu(EUROPI_SCRIPT_CLASSES).main()
+          EOF
+
+      - run: tree micropython/ports/rp2/modules


[question] Is this line some debug output?

Yep, verify what was written.

mjaskula · 2022-11-29T19:56:46Z

.github/workflows/compile_to_uf2.yml

+          cp -r europi/software/contrib/*.py micropython/ports/rp2/modules/contrib
+
+      - run: wget https://raw.githubusercontent.com/stlehmann/micropython-ssd1306/master/ssd1306.py -O micropython/ports/rp2/modules/ssd1306.py
+      - run: sed -i 's/progsize=256)/progsize=1024)/g' micropython/ports/rp2/modules/_boot.py


[question] What's this line about? It should probably include a comment or clarifying name.

@redoxcode can you help shed more light on why we needed to increase the progsize?

To be honest I am not 100% sure how the different memory types of the pico work. However without this and without calling the garbage collector before the menu.py the software would freeze somewhere between the lines 15 and 34 of main.py. That's where all the other contib scripts get imported. After trying different import orders without luck I concluded, that it's not a singe module that causes the freeze, but some type of memory reaching it's limit. So yeah, there is a little try and error involved. Maybe we should test again, if changing the progsize is required or if just the garbage collection alone fixed it...

This worries me. I've done some digging and can find any explanation as to why a frozen script would behave differently than an unfrozen one. That is, why this main.py needs a call to gc and the change in progsize, but we don't needed it in the 'normal' situation. It doesn't make sense to me. I'll keep digging.

.github/workflows/compile_to_uf2.yml

…es version

mjaskula · 2022-11-30T15:12:38Z

I missed the documentation PR (#199) that went through just before this, I just caught up on that. I noticed two things:

[required] I think that, for consistency, that documentation and this workflow should agree, and that should be in the scope of this PR. Particularly if our script diverges from our own documentation we should say why.

To be fair, I haven't looked through closely to find any differences, but I suspect that you know if there are any. So this ask may be a noop.

[optional] I think that we should be freezing the included modules. The previous PR included a discussion about obfuscating code being the main benefit, but the documentation says this (emphasis mine):

The benefit of using a manifest is that frozen modules are more efficient: they are faster to import and take up less RAM once imported.

Since we've already taken steps to address the menu startup time and we know that it is caused by the many imports, this looks like it could be an easy win.

redoxcode · 2022-11-30T15:34:54Z

[required] I think that, for consistency, that documentation and this workflow should agree, and that should be in the scope of this PR. Particularly if our script diverges from our own documentation we should say why.

To be fair, I haven't looked through closely to find any differences, but I suspect that you know if there are any. So this ask may be a noop.

I think currently there are no differences, but I agree, if we do something different here than in #199 that should be documented as well. But again, no differences yet.

[optional] I think that we should be freezing the included modules. The previous PR included a discussion about obfuscating code being the main benefit, but the documentation says this (emphasis mine):

So I think you are confusing freezing with bytecode here. Freezing is the process of baking any code into the firmware. The question is if you want to freeze python source code or pre-compiled bytecode. Bytecode is used for performance benefits. The obfuscation is just a side effect (we don't much care about). But as already discussed in #199 we'd have to see if the impact on performance is noticeable at all in our application, since I didn't do any speed comparison (I got modules in my case that take longer to boot up, so I don't care that much about boot time).

Hope this has answered some of your concerns. Keep in mind that there is always the option to improve and optimize on this with future PRs once we get things running.

mjaskula · 2022-11-30T15:56:14Z

So I think you are confusing freezing with bytecode here.

Sorry, yeah, I'm probably getting the terms mixed up here, but my point is that there's a nice potential to improve the wait time when starting up and switching between scripts, which could be huge for non-dev users.

Keep in mind that there is always the option to improve and optimize on this with future PRs once we get things running.

Yess, good point, we can improve import perf in future iterations.

mjaskula · 2022-11-30T16:29:36Z

You two have convinced me that the 'weirdness' in the main.py isn't a good reason to hold this PR, we can improve that in the future as we learn more. Great work.

… compile-uf2

…e the releases firmware asset.

awonak · 2022-12-01T04:56:45Z

I ended up adding the documentation updated to this PR. Please take a look and confirm the new documentation is clear. I also think that merging this PR should not happen until we are ready to cut a release to ensure the updated documentation matches the latest release.

mjaskula · 2022-12-01T18:43:19Z

software/programming_instructions.md

+
+ The quickest way to get your EuroPi flashed with the latest firmware is to head over to the [releases](https://github.com/Allen-Synthesis/EuroPi/releases) page and download the latest `europi-vX.Y.Z.uf2` file. Then follow the 'BOOTSEL' instructions above to flash the EuroPi firmware to your pico. You can now skip the **Setting Up** section and proceed to the [Next Steps](#next-steps) section.
+
+> **_NOTE:_**  The EuroPi firmware and contrib scripts will be installed and accessible via Thonny for writing custom scripts on the pico, however Thonny will not show those firmware and contrib files in the pico filesystem. If you want to have access to the EuroPi firmware source code on your pico, follow the **Setting Up** instructions in the next section.


[question] Have we tested that this is correct? Does a main.py located in the standard location override the one delivered with the firmware? It makes sense that it does, but I'd like to be sure.

This opens up a bunch of other questions for me about how things will work, but I guess we'll need to document them as we find out exactly how people want to use this new distribution.

I can double check, but this follows the import priority documentation @redoxcode provided.

The testing I did was done using REPL, I didn't test adding a main.py and rebooting, so I'll test that to be sure.

@mjaskula Good catch! My assumption was incorrect. I have updated the docs to reflect that main.py will not override the entry point.

We essentially now have 3 firmware options. 1) uf2 quick start drag and drop, 2) pypi distributable source, and 3) download source from GitHub.

I think the updates I've included here are good for now, but we should probably revisit the programming instructions at some point.

mjaskula

The new doc looks good to me. Just one question on what was tested. Thanks.

…mware. Also we should default to generating a prerelase that we can edit. Also updated Release Process instructions.

mjaskula · 2022-12-02T19:30:39Z

.github/workflows/README.md

@@ -34,14 +34,14 @@ Pushes to the main [pypi.org](https://pypi.org/project/micropython-europi/#histo
 ### Release process

 1. Decide which version number to release.
-2. Update [`software/firmware/version.py`](/software/firmware/version.py) with the new version number.
-3. Update [`changelog.md`](CHANGELOG.md) with a description of the release.


[Question] Is there a reason we're removing this step form the release process? Seems unrelated to the new uf2 file

Oh, I see the "Update the generated Github Release" as the second to last step, I guess this replaces the release notes?. I don't know what this is. Should there be a link to it? or a description on how to publish?

Good call. I've added a link to the releases page. The rest is self-explanatory. Edit the release notes as described in the docs and the select either Publish or Save Draft to continue editing.

awonak added 17 commits November 27, 2022 14:42

Create Release and publish uf2 asset upon version tag creation.

b6debc1

typos and syntax

917a237

testing on push

1c1e3e1

separate run commands

878ce96

testing

c786127

clone to path

7d9109d

missed leading folder

cab252f

provide working dir for final make

a05fa6e

install missing build packages

f4a9e25

do not use env vars

ca32b98

add asset type

7c2537e

replace release action

7952958

add automatic_release_tag

d963d19

fix experimental typo

b88e4d7

use correct trigger. remove auto tag latest.

b0f4b89

create release, not draft

603e17b

not a prerelease

3e3ef83

awonak mentioned this pull request Nov 27, 2022

[Project Suggestion] Create custom firmware image that enables drag and drop setup #194

Closed

awonak requested review from mjaskula and roryjamesallen November 27, 2022 23:49

redoxcode mentioned this pull request Nov 28, 2022

Add documentation for creation of custom firmware uf2 files #199

Merged

mjaskula reviewed Nov 29, 2022

View reviewed changes

awonak added 3 commits November 29, 2022 21:40

lock ubuntu version

979d65d

Added more names for documentation and locked action-automatic-releas…

d7e13cc

…es version

revert testing change

c287010

mjaskula approved these changes Nov 30, 2022

View reviewed changes

awonak added 5 commits November 30, 2022 22:08

Merge branch 'main' of https://github.com/Allen-Synthesis/EuroPi into…

e856d46

… compile-uf2

Add workflow documentation.

68958a5

Added a quick start section to programming instructions that referenc…

628d9af

…e the releases firmware asset.

use absolute url for releases

181b5d3

grammar and stuff

987b15f

awonak requested a review from mjaskula December 1, 2022 04:54

mjaskula reviewed Dec 1, 2022

View reviewed changes

mjaskula approved these changes Dec 1, 2022

View reviewed changes

Correct documentation to state uf2 cannot be used to write custom fir…

b03d836

…mware. Also we should default to generating a prerelase that we can edit. Also updated Release Process instructions.

mjaskula reviewed Dec 2, 2022

View reviewed changes

Add link to github release page in release workflow docs.

1f2e676

awonak merged commit d40f44b into Allen-Synthesis:main Dec 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create Release and publish uf2 asset upon version tag creation. #198

Create Release and publish uf2 asset upon version tag creation. #198

awonak commented Nov 27, 2022 •

edited

Loading

mjaskula left a comment

mjaskula Nov 29, 2022

mjaskula Nov 29, 2022

awonak Nov 30, 2022

mjaskula Nov 29, 2022

awonak Nov 30, 2022

mjaskula Nov 29, 2022

awonak Nov 30, 2022

redoxcode Nov 30, 2022

mjaskula Nov 30, 2022

mjaskula commented Nov 30, 2022

redoxcode commented Nov 30, 2022

mjaskula commented Nov 30, 2022

mjaskula commented Nov 30, 2022

awonak commented Dec 1, 2022

mjaskula Dec 1, 2022

awonak Dec 1, 2022

awonak Dec 1, 2022

awonak Dec 2, 2022

mjaskula left a comment

mjaskula Dec 2, 2022 •

edited

Loading

mjaskula Dec 2, 2022

awonak Dec 4, 2022


		The quickest way to get your EuroPi flashed with the latest firmware is to head over to the [releases](https://github.com/Allen-Synthesis/EuroPi/releases) page and download the latest `europi-vX.Y.Z.uf2` file. Then follow the 'BOOTSEL' instructions above to flash the EuroPi firmware to your pico. You can now skip the Setting Up section and proceed to the [Next Steps](#next-steps) section.

		> _NOTE:_ The EuroPi firmware and contrib scripts will be installed and accessible via Thonny for writing custom scripts on the pico, however Thonny will not show those firmware and contrib files in the pico filesystem. If you want to have access to the EuroPi firmware source code on your pico, follow the Setting Up instructions in the next section.

Create Release and publish uf2 asset upon version tag creation. #198

Create Release and publish uf2 asset upon version tag creation. #198

Conversation

awonak commented Nov 27, 2022 • edited Loading

mjaskula left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjaskula commented Nov 30, 2022

redoxcode commented Nov 30, 2022

mjaskula commented Nov 30, 2022

mjaskula commented Nov 30, 2022

awonak commented Dec 1, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjaskula left a comment

Choose a reason for hiding this comment

mjaskula Dec 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

awonak commented Nov 27, 2022 •

edited

Loading

mjaskula Dec 2, 2022 •

edited

Loading