Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Micropython support #728

Open
askvictor opened this issue Jul 25, 2016 · 54 comments
Open

Micropython support #728

askvictor opened this issue Jul 25, 2016 · 54 comments
Labels
feature framework platform

Comments

@askvictor
Copy link

askvictor commented Jul 25, 2016 •
edited by ivankravets

Add support for Micropython for compatible boards (e.g. esp8266 family)
@ivankravets ivankravets changed the title Feature request: Micropython support Micropython support Jul 26, 2016
@ivankravets ivankravets mentioned this issue Aug 9, 2016
Closed
@TravisJoe
Copy link

I would also like to see this.

For any boards that you can connect and get a REPL, one of the key Micropython developers built a tool that is useful and could be used as part of the interface or as a example on how to integrate Micropython into PlatformIO
https://github.com/dhylands/rshell

I am also using the Pycom modules which has a fork of Micropthon. These boards include wifi, bluetooth 4.0, and soon cellular. All their newer boards are based on the ESP32 modules.

@ivankravets ivankravets mentioned this issue Feb 1, 2018
Closed
@gjlawran
Copy link

gjlawran commented Feb 5, 2018

There is also an Atom package called pymakr that can provide access to both Serial and Web - REPL. It is supposed to syncronize (upload/download) files too - but this feature does not appear to be working in the latest release

@gjlawran
Copy link

gjlawran commented Feb 5, 2018

Another useful source to consider in building Micropython suppport into https://github.com/nickzoic/mpy-utils

@idesignstuff
Copy link

As I have been moving more toward Python lately, I miss the integration that Platformio/Atom had achieved. Is there anyone out there working on integrating micropython tools into Platformio?

@TravisJoe
Copy link

Probably the most universal and straightforward way of doing this is by integrating, or doing something similar to rshell. This relies on the REPL which is a pretty universal method on connecting to Micropython devices.
https://github.com/dhylands/rshell

Pycom does something similar with their Atom plugin, but they are focused solely on their products, so even if other devices work, they are not truly supported.
https://atom.io/packages/pymakr

@ShakataGaNai
Copy link

I do a bunch of Micropython on Wemos D1 units (ESP8266) so I really love the idea of support for this. Since it hasn't been linked elsewhere in the thread: MP ESP8266 Docs & Firmware Download

@ivankravets
Copy link
Member

Some feature requests: https://www.reddit.com/r/arduino/comments/9bk3mk/massive_release_of_platformio_ide_for_vscode/e53vu14/

@Niko-La
Copy link

Niko-La commented Nov 6, 2018

Is there a timeline for micropython on platformio?

@TravisJoe
Copy link

Based on how I have used Micropython and VScode what would cover most use cases is

  • Sync to mounted drive (like rsync)
  • Sync using serial (like rshell)
  • Sync using ftp? (like webREPL)

Then being able to open open a reply using one of the methods as well.

For now I write a small script that uses rsync and a file to list what needs to be copied over. I use the standard pyboard based implementations (USB) most of the time so it is mounted as a VCP and a mountable drive through the same connection. I have a pretty quick edit and deploy processes now.

Uploading lower level firmware is so uncommon for me that I don't think that needs to be initially supported.

@Niko-La
Copy link

Niko-La commented Nov 15, 2018

@TravisJoe rshell be used to get the devices repl from remote network over the internet? I would like debug my micropython script using something like that.

@TravisJoe
Copy link

@Niko-La
rshell is designed for serial or virtual com port interactions with a micropython device. For connecting over a network you will want something more like this
https://github.com/micropython/webrepl

I think Adafuits tool also can do over network
https://learn.adafruit.com/micropython-basics-load-files-and-run-code/install-ampy

@Tylersuard
Copy link

Micropython plz. I like Atom.

@Niko-La
Copy link

Niko-La commented Apr 2, 2019

+1

@SantiPerti
Copy link

Platform.io + Micropython. Pretty please!!

@anderl80
Copy link

anderl80 commented May 4, 2019

+1

@DjianR
Copy link

DjianR commented Jun 30, 2019

+1

1 similar comment
@ysazak
Copy link

ysazak commented Jul 12, 2019

+1

@pingf
Copy link

pingf commented Jul 17, 2019

+1 for vscode

@iBobik
Copy link

iBobik commented Jul 17, 2019 via email

@SRGDamia1 SRGDamia1 mentioned this issue Aug 6, 2019
Closed
@ivankravets
Copy link
Member

Hi all,

Could you explain about classic workflow how you do programming using Mycropython? To be honest, I don't understand what should be done from our side.

I would be thankful if you explain what you expect from PlatformIO.

Thanks!

@cselti
Copy link

cselti commented Aug 30, 2019 •
edited

Dear Ivan!
My opinion, that the micropython (mp) support doesn't differ from the other supported platform framework . The plaformio downloads the mp's code, builds it with the mp's tool, flashes the firmware and upload the app specific code. The mp has a package handling tool called upip (after the Python pip). The plarformio IDE should support the mp REPL - it can be usable on the connected microcontroller flashed with mp. I think the debugging could be solved with mp also - but I haven't used the debugging yet, because the REPL very usable tool for testing codes directly on the microcontroller.
I'm only little experienced with mp, but the micropython developers can provide more detailed information and maybe can help in the implementation. The mp forum (https://forum.micropython.org/) and/or github repo (https://github.com/micropython/micropython) could be easiest way to contact with the mp gurus.

I hope I can help.
Regards,
Adrian

@iBobik
Copy link

iBobik commented Aug 30, 2019 via email

@cselti
Copy link

cselti commented Aug 30, 2019 •
edited

@iBobik - As far as I know, the platformio works the following in an application project (RTOS, Arduino)

  1. downloads the platform (FreeRTOS, Arduino) core and compilers, tools etc
  2. downloads the board specific (Atmel, Arm cortex-M, ESP, ..) compiler, tools
  3. builds the the app project with 1+2 compilers
  4. and flash the board with created binary
  5. connects the boards to UART interface and shows the output of the running app

In case of micropython (because python's dynamic nature) the 3 usually necessary once and upload (which is simplified flashing) only the app specific micropython files. Except when the project has native code, that is interfaced to micropython, the 3. steps should repeat in every new project build+upload.

My point of view, that the difference is maybe between in your project's workflow and the platformio worklow - but I think your workflow can be divided to the 1,2,3,4 steps also too.

And in case of micropython (because the Python's REPL) the platformio IDE should support to connect the REPL through UART separately.

But the Pycom (creates ESP32 boards with micropython support) created an VS Code based IDE - the Pymakr (https://github.com/pycom/pymakr-vsc/). The Pymakr supports their boards only, it's doesn't works different ESP32 board, but it can be useful example how it's upload the applications files and connects the REPL on UART.

@ivankravets
Copy link
Member

ivankravets commented Aug 30, 2019 •
edited

Thanks all for the useful comments! I also did some research. I found 2 popular micro Python frameworks (virtual machines):

Do you use other/own VMs?

What I found

  1. All exiting docs and manuals force users to flash the target device with VM before. It is clear.
  2. User personally manages *.py files and external modules/packages
  3. User has a few options on how to "flash/deploy" code to a target device.

My proposal

  1. We keep the same PlatformIO's philosophy with a "framework" option. You will be able to control which VM and version to use
  2. No need to bother with VMs because we know everything about your hardware board =... and we can cross-compile true VM for the specific device directly on the user's host machine. And, yes, we can flash this device with Python VM
  3. I see there is a tool called mpy-cross. I think we can use it to compile raw Python code to bytecode *.mpy, similar to *.pyc on desktop machines.
  4. We can do uploading of *.mpy files. Yes, the *.mpy files should be compiled with the same VM which resides on the target device.

I think we can start with the official MicroPython. It covers a lot of hardware https://github.com/micropython/micropython/tree/master/ports

Libraries

I found a list of useful libraries/packages by @adafruit => https://github.com/adafruit/Adafruit_CircuitPython_Bundle/tree/master/libraries

It would be great to have them in our registry and do the same what we do with C/C++ libraries. So, extend PlatformIO Library Dependency Finder automatically deploy dependent libraries via cross-compilation depending on a VM type.

Example of workflow

  1. Create a new PlatformIO project:
pio init -b esp32dev --project-option="framework=python"

Why python? There a lot of types: MicroPython, CircuitPython, Zerynth

  1. Place project Python source code to src folder

  2. Build a project or deploy firmware

# first time it will build VM 
pio run 
# flash
pio run -t upload

And yes, no need to install any tools, compilers, dockers, etc. Just "upload" and see how board blinks.

Comments are welcome.

@iBobik
Copy link

iBobik commented Aug 30, 2019 •
edited

There is also fork of MP's original author @pfalcon: https://github.com/pfalcon/pycopy

@TravisJoe
Copy link

My personal thought is that the generic development cycle is much more important for user experience than the firmware deployment cycle (which is much less frequent).

My assumption, and based on my experience, most people start with a development board that already has a version of firmware on it. Or they use a pre-built stable firmware for a specific board. The only reason I started building my own firmware instead of using the stable builds is because I make custom boards with custom board definitions.

An efficient "write code, deploy code, test code" cycle is what is missing for me and I have written scripts to aid in this. I would guess most users both experienced and not would benefit from the focus on this as well.

Cross compiling is rarely a requirement because of how the VM is setup. Only once my projects are tested do I cross compile or freeze my code. Having the files in .py format aid in debugging as tracebacks can literally say which line number is causing the issue.

My personal suggestion is keep it simple for now. I rarely need to build new firmware, maybe every couple of months, and I use Micropython nearly every day for work and projects.

@ivankravets
Copy link
Member

What to do with 3rd party packages/libraries?

Do you use them? Are they compatible between all these micro Python implementations?

@TravisJoe
Copy link

I have used some libraries, but few. And to make it more complex is that some libraries are port specific and some are not. I would say that external libraries (non-standard) are pretty use-case specific and most of the time I am writing my own or using the standard libraries.

I do agree that having a unified library and dependency system would help. Micropython has been working on and improving their "Machine" library to abstract more interfaces to help with making code and libraries cross port. But overall this is pretty rare for me. Once I get the library into my source folder (I cannot or do not use upip on my projects)

Micropython is close enough to python 3.5 some desktop libraries even work without editing.

For micropython's list of libraries look here as well
https://github.com/micropython/micropython-lib

@ladyada
Copy link

ladyada commented Aug 30, 2019

Here is some info that may be useful!

All CircuitPython libraries are compatible with all CircuitPython boards, the CircuitPython hardware API is identical across all devices. The CircuitPython hardware API is not compatible with MicroPython (which as mentioned above a different API, sometimes varies for each board)

All available libraries are submodule'd either here (in the main supported bundle) or in the community bundle.

Version management is handled by release tags on github and are published automatically to PyPI as well. The requirements.txt has the dependencies.

@cselti
Copy link

cselti commented Aug 30, 2019

Can the "pio run" be extended with a new "sync" target to syncing only the project's python files?
This target can accept parameters for different type of sync (USB, serial, network).

@stefan-sherwood
Copy link

stefan-sherwood commented Oct 1, 2019 •
edited

I created a VS Code keyboard shortcut (Ctrl+Alt+U) that uploads the current file to my network-connected Micropython device. Maybe something in here will help with this effort:

(Cited file locations are for Windows, obvs)

Script that does the transfer

The file webrepl_cli.py from https://github.com/micropython/webrepl somewhere on your computer. That location specified in the VS Code settings:

settings.json
located in %UserProfile%\Appdata\Roaming\Code\User\
(other platforms)

{
	a bunch of stuff that was already there,
	"webrepl_cli": "C:/code/esp/webrepl/webrepl_cli.py"
}
+ Replace the above path with wherever you put the file.

Micropython device running Webrepl

The device (ESP8266 or ESP32 in my case but can be any Micropython device) is connected to the network and running Webrepl:

boot.py
located in root directory on the Micropython device

import webrepl
webrepl.start()

A VS Code task that uploads the file

tasks.json
located in ProjectRoot\.vscode\

{
	"version": "2.0.0",
	"tasks": [
		{
			"label": "upload_webrepl",
			"type": "shell",
			"command": "${config:python.pythonPath} ${config:webrepl_cli} -p password ${fileBasename} ip-or-name.of.micropython.device:${fileBasename}"
		}
	]
}
+ Replace the above IP address or name with the network address for your Micropython device.
+ e.g. 192.168.0.123
+ e.g. light-switch.cable.rcn.com

A VS Code keyboard shortcut that runs the task

keybindings.json
located in %UserProfile%\Appdata\Roaming\Code\User

// Place your key bindings in this file to override the defaultsauto[]
[
	a bunch of stuff that was already there,
	{
		"key": "ctrl+alt+u",
		"command": "workbench.action.tasks.runTask",
		"args": "upload_webrepl"
	}
]

@Niko-La
Copy link

Niko-La commented Dec 10, 2019

would be nice to Circuitpython to this issue as well.

Is it possible to get usb serial monitor working with circuitpython? Circuitpython I believe is simple to deploy code to as the hardware just show up as a mass storage and the script are just written to it.

@tongclement
Copy link

tongclement commented Dec 14, 2020 •
edited

+1, can't wait for micropython support with platform IO.

@ivankravets ivankravets mentioned this issue Apr 24, 2021
Closed
@SubaruArai
Copy link

SubaruArai commented Aug 16, 2021 •
edited

@ivankravets I would like to accelerate the implementation by contributing. Is there any branch I can start working on? (like, an internal development branch that's unfinished) Just to be clear, if it happens it will be my personal project, (it will be slow)I can't do it in my work hours.

Also, I found adding stubs for hype tinting helps a lot. Here's what I'm using right now: micropy-cli, micropy-stubs, Pico-Stub. I guess the manager (micropy-cli) could be implemented into platformIO (either running in the background or make a similar package), so that selecting boards or platforms will automatically generate corresponding stubs?

@lerra
Copy link

lerra commented Jan 13, 2022

It would be awesome to get micropython support for platformio :)

@nicklasb
Copy link

@ivankravets, @SubaruArai
While I am not new to developing stuff, I am new to the maker world, and to me it seems like a great opportunity gone to waste if not at least some starting point for MicroPython support development was established to get things going.

A growing number of projects and supplier example code are now MicroPython only, which forces newbies like me away to Arduino IDE, and without newbies, all projects eventually loses traction. Where could one start helping out?

@stefan-sherwood
Copy link

@nicklasb Not sure what you mean "forces newbies like me away to Arduino IDE". I develop my MicroPython projects using VS Code, which is much nicer than Adruino IDE.

@nicklasb
Copy link

nicklasb commented Jan 24, 2022 •
edited

@stefan-sherwood Just to be clear, I'm not just considering the language, but the integration with the boards and so on, the things platform IO does for C++. What is your setup?

@stefan-sherwood
Copy link

See my previous comment: #728 (comment)

What else specifically are you looking for?

@nicklasb
Copy link

nicklasb commented Jan 24, 2022 •
edited

Maybe I am wrong, but doesn’t PlatformIO offer a lot more functionality than that? Or have I misunderstood something completely?

@TravisJoe
Copy link

I think I have a good perspective of several sides of this argument. I use Micropython for testing as well as some professional products that are in production.

For the sake of just getting started with Micropython PlatformIO might be too big, and VS Code with the serial terminal turned on is extremely simple and easy to use. The annoying part I get around with a script I wrote which which syncs files between my device and my active directory (which is also versioned with Git). That handles almost all of my development on a board that is already defined. If the board is not defined you have to go build firmware for it.

Now where PlatformIO would be great is the build system and automating that. Being able to quickly build a new "Board" definition folder would be great. Minimizing the complexity and learning curve there with ideally visual tools would be helpful even to me. Then as a product matures it is much better to compile your python code into the firmware and if PlatformIO could help manage both the application and build process to put it all together with a custom board definition would be amazing.

I have done something similar using Docker where in my Git project I have the board file defined, and a build config file which includes the names and paths to all the files I want compiled into my firmware. VS Code lets you run Docker as a test space and I can run my scripts with a Docker image that includes the entire build system already setup. It will move the board definition and application files over and compile the MBOOT and Firmware DFU files and save it into the project folder which is retained even after Docker is closed.

This one is not setup to run the latest Micropython version but it could easily be updated for that.
https://hub.docker.com/repository/docker/travistconsulting/travist_docker_micropython

@aprs434
Copy link

aprs434 commented Apr 29, 2022 •
edited

Here is a rundown of what I would expect Platformio to do for a Micropython project:

1. Installing esptool.py and rshell

$ pip install esptool rshell

2. Installing Micropython

3. Using pyboard.py

The pyboard.py utility comes preinstalled with this repository.
Here is the official pyboard.py documentation.

4. Installing the Firmware Scripts

From within the cloned repository folder, copy the firmware scripts main.py and lib.py to the device flash drive, called :

$ pyboard.py --device /dev/ttyUSB2 -f cp main.py lib.py :

The correct device port was derived from the dmesg device messages on Linux.

Library files can better be grouped in a /flash/lib folder. If so, a boot.py file needs to be added to the root /flash folder of the device, with the following contents:

import sys
sys.path[1] = '/flash/lib'

5. Using rshell to Access the Micropython REPL

A device running Micropython has a shell environment, called the read–eval–print loop (REPL).
It can be accessed by running the rshell tool.

$ rshell -p /dev/ttyUSB2 -b 115200
repl
  1. After typing repl followed by a Return, hit Ctrl+D to soft reboot the device.
  2. You can now monitor the output of the main.py script running on the device.
  3. Hit once or twice Ctrl+X to leave the REPL and to return to rshell.
  4. Hit Ctrl+D again to leave rshell and to return to the command line.

@frankcohen
Copy link

Is there an update on this? I am the principal maintainer for the Reflections project. It’s an open-source wrist watch with a round ESP32-based main board. See github and ReflectionsOS. I’m considering writing the project OS in MicroPython to make it easy to write, maintain, and teach. Next I need a MacOS compatible IDE to develop my MicroPython ReflectionsOS. Considering CircuitPython, Platform.io, Arduino IDE, Thonny IDE. It seems ironic to me that Platform.io is written in Python and doesn’t yet support MicroPython development. -Frank

@TravisJoe
Copy link

@frankcohen
I use Micropython and Circuitpython in production systems, and after the years since posting one of the first comments on this thread saying I want this also, I can also see how once you get a flow and IDE is not as critical as it feels initially. That said something that helps sync projects to the board and streamline that would be nice. But once you have a basic setup it works quite well in development. I like the idea of a manifest for what files are synced and having a user friendly REPL built in.

Mac and Linux you can use rsync for example to sync files to the flash. A basic python script can do this, and VSCode has an easy what to have a terminal in the bottom of your screen.

I have thought about making more utilities and make them more generic for different setups, but have not had the time.

Probably not the best answer, but there is at least options that could be used for the project and teaching.

@frankcohen
Copy link

frankcohen commented Jan 16, 2023 via email

@stroobandt
Copy link

stroobandt commented Jan 16, 2023 •
edited

"[…] after the years since posting one of the first comments on this thread saying I want this also, I can also see how once you get a flow and IDE is not as critical as it feels initially."

The above may be true for seasoned makers & hackers, but this is certainly not true for aspiring programmers like children and their educators in the educational system. For the latter groups, a smooth IDE workflow is essential for a good uptake and Micropython does an extremely poor job at it. By consequence, the uptake of Raspberry Pi and Arduino is several orders higher.

A programming language of high educational value like Python deserves much better…

@idesignstuff
Copy link

idesignstuff commented Jan 16, 2023 via email

@tmountjr
Copy link

tmountjr commented Feb 9, 2023

I just got started poking around with my new Pico W and while it wasn't too laborious to get a circuitpython-specific VSCode plugin running (which comes with built-in serial monitor), the one thing that I really missed as a career software developer who's gotten used to making sure all his projects are kicked up to github is that literally the only place the source code for a Pico board exists is on the board itself. And while I know I could probably git init right on the mounted volume, I do worry about the 2mb flash limit. Also having a way to compile down to .mpy as part of a build process would be kinda nice rather than having to manually install the entire toolchain.

Also keenly aware that setting up a full-blown dev and build environment with source control is maybe a little overkill if the primary audience is hobby/education. PIO could be an overengineered solution in search of a problem. Having said that, it would be nice to have the option of scaling up.

@jcw
Copy link

jcw commented Feb 25, 2023

This issue truly baffles me. PlatformIO compiles code, uploads it, runs it, and provides a serial connection or test runner. Doing this for MicroPython requires adding a platformio.ini file to that project and working out the details (a complex task, which MPy currently does in its own way just fine, without having to depend on PIO).

Once that is done, and MPy is running on the µC, it's an app like any other. Things like rshell help using MPy from a host. If using rshell is the issue, then a new custom target can be created for PIO, which streamlines that scenario.

Most of the comments above (and I admit to only skimming some of them) seem to be about using MPy (and developing with it, since it is a build & dev system itself). But what does this have to do with PIO, which is aimed at C/C++ code?

Is this issue not simply due to conflating two different development roles, perhaps because they both use Python?

@nicklasb
Copy link

nicklasb commented Feb 25, 2023 •
edited

@jcw Yeah...I think that we all seem to misunderstand each others situations in some way.
I know that at least I am totally confused. This might not only apply to this though. :-)

My scenario, or at least goal scenario, is that:

  • I develop library and framework-y things in C, for optimization and maximum compatibility.

  • I write the application code in Micro- or Circuit Python, because to my experience, Python is just a way better language for business logic. It is like clarity- and specificity wonderland compared to C.

In a technical sense, these two things could be done separately.
But in practice, and to be productive and uninterrupted, I want to do both these things in the same environment, and in many cases, the same project.

If this is already possible and there is an example for that that I can try and I just haven't got it, great, but if it isn't, it would be awesome if PlatformIO in some way could support it, at least in some way.

@TravisJoe
Copy link

I would agree mostly with @jcw, there are two aspects of this and people that might be less familiar with Python and Micropython might be trying to make it act too much like C. This is common when transitioning to Micropython.

The one use-case which is more advanced is in the production environment it is nice to freeze your code within Micropython firmware.

A good way to think of this is Micropython is the bare metal OS, and that is your firmware. Then there is your application layer, which for development usually lives in the mounted flash storage. This does not require any compiling, just syncing to your device. You should avoid unnecessary rights (flash wears) and definitely not put GIT on it, it is just for code, configs, and sometimes image assets.

You develop locally, sync files, run, test, write, sync, run, test.

One of the benefits that C does not have is the interactive REPL that python provides. It makes manual testing of modules a breeze.

You can use tools in windows and linux to a binary sync (rsync in linux and mac) which is great for syncing specific files for development. I use a script for this and a config file to say which files are synced. If you want to go one set further you could pre-compile files but that is overkill for most people.

What I was hoping for awhile that PIO would initially make a nice config and syncing scheme that would be easy to use, eventually it could get into a good custom build system to integrate custom boards and freezing firmware, but this is advanced.

@teaalltr
Copy link

Any update on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature framework platform
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests