Help testing Deviot 2.0

[gepd]

Hi everyone!,

I have been working in the version 2 of Deviot. Now I'm ready to to receive feedback of this version, there are still some features not ready, but I think you can use without problem with most of your projects.

The plugin has been written from scratch, so maybe there is not much new features, but I hope this version is more consistent and with less bugs. Plus the code now is more organized, this is really good to maintain the plugin!.

To install this version read Install developer version but instead of use the developer branch use:

https://github.com/gepd/Deviot/tree/deviot_2

New features already implemented

• PlatformIO console.

• Quick bar, access to all deviot features easily

• Auto-clean the serial monitor window (after 1600000 characters)

• Clean serial monitor window command (from menu)

• Support to symlink python installation

• Search library with less than 4 charactes

Features NOT implemented Yet:

• Auto-discover OTA services

• Automatic discovery of PlatformIO updates

• Full Translation of the plugin

• Plugin information in the status bar (optional)

• Console not colored

• Highlight lint error

• Options Menu

• Fix Shorcuts

• Add code highlight & completions

Please test all features and report any problem in this post.

Thanks!

[tablatronix]

After update I only have a blank drop down with greyed out languages. Cannot select any.

 File "C:\Users\user\Documents\projects\bin\sublimetext\Data\Packages\Deviot\libraries\I18n.py", line 68, in set_lang
    file_path = self.id_name_dict[lang]
KeyError: 'en'

[gepd]

Can you try removing Main.sublime-menu from the plugin folder and restarting the ST? Is the language of your O.S english?

[tablatronix]

windows 10

yes en, I reinstalled and now have menus.
on load i get

  File "C:\Users\user\Documents\projects\bin\sublimetext\Data\Packages\Deviot\beginning\pio_install.py", line 229, in check_python
    version = sub(r'\D', '', out[1])
TypeError: 'NoneType' object is not subscriptable

on select board i get

  File "C:\Users\user\Documents\projects\bin\sublimetext\Data\Packages\Deviot\libraries\file.py", line 57, in read_json
    return json.loads(file)
  File "./python3.3/json/__init__.py", line 316, in loads
  File "./python3.3/json/decoder.py", line 351, in decode
  File "./python3.3/json/decoder.py", line 369, in raw_decode
ValueError: No JSON object could be decoded

all options sub menus greyed out

[gepd]

I've pushed a fix, try again... you shouldn't see the first error, the second one is related with the file boards.json in Packages\User\Deviot\pio if the content of that isn't json, please remove it and restart ST

If it keeps showing it, please paste here the content of boards.json

[tablatronix]

this is in the file

'pio' is not recognized as an internal or external command,
operable program or batch file.

[tablatronix]

Now its telling me needs python, so maybe my env path is broken is not actually working, let me check that real quick.

[tablatronix]

Ok working now, my python path was not working.

[gepd]

Excellent, I've fixed that too, so next time will not care if the python isn't in your global envs

[tablatronix]

My options menus are still greyed out however.

[gepd]

All of them? Is the console showing some error?

[tablatronix]

yes all menu items under options, no errors.

[gepd]

oh, that menu. Yes, it's disabled on purpose, I'm working in that menu, I will update it soon.

I have added it to the task list

[airdrummingfool]

Hi @gepd , I'm happy to help test. I updated via git (I manually pull commits and then restart Sublime). Here's a few issues I've found (please let me know if there's a better way to report these:

1. I don't see the current port shown down in the Sublime status bar like it was in the old version. (EDIT: I just saw that this in in your unfinished list, my bad)

2. I tried to do an OTA upload (went to Deviot menu > Serial Port(s) and selected the "Generic" port with my ESP's IP address), but it ended up uploading over the USB serial connection anyways (the upload worked fine :). Unfortunately, it doesn't show up reliably, and there doesn't appear to be a way to manually specify a network IP address to use as a port.

The next time it showed up (in Deviot > Serial Port(s)), I clicked it, unplugged my USB serial cable, and clicked upload. I got the following message:

[ Deviot 2.0.0.dev ] Starting...
01:24:37 Select a Port From The List

The "Generic 192.168.1.119" port was in the list, but when I clicked on it, it just showed the message and the port list again. After clicking "upload" and choosing the "Generic" port entry a few more times, I got it to attempt an upload, but it didn't use the OTA password I'm almost positive I entered earlier (so the upload failed).

I then clicked Deviot > Change OTA Password, re-entered my OTA password, and hit enter. An OTA upload immediately started (which I wasn't expecting), and it worked!

I just thought to check the Sublime console, and both of these errors show up multiple times:

Traceback (most recent call last):
  File "/Applications/Sublime Text 3.app/Contents/MacOS/sublime_plugin.py", line 389, in run_callback
    expr()
  File "/Applications/Sublime Text 3.app/Contents/MacOS/sublime_plugin.py", line 452, in <lambda>
    run_callback('on_close', callback, lambda: callback.on_close(v))
  File "/Users/Tommy/Library/Application Support/Sublime Text 3/Packages/Deviot/Deviot.py", line 45, in on_close
    serial.serials_in_use.remove(port_id)
ValueError: list.remove(x): x not in list

Traceback (most recent call last):
  File "/Users/Tommy/Library/Application Support/Sublime Text 3/Packages/Deviot/libraries/quick_menu.py", line 185, in callback_serial_ports
    port_selected = port_list[selected][1]
IndexError: list index out of range

my guess is that they're related to the "Generic" (network) port disappearing from the list.

EDIT:
3. I tried to open the library folder (Deviot > Library Options > Open Libraries Folder), but it seems to be broken on MacOS:

4. There's no "Build"/"Upload"/etc options in Sublime's right-click menu anymore.

5. I used to be able to click Deviot > Build from any .ino or .cpp file in my project, but now it appears that I have to build from the main ino file. If I try it from a separate cpp file, the console hangs at

[ Deviot 2.0.0.dev ] Starting...

6. If I click "Upload" while running the serial monitor, but the build fails, the errors disappear immediately and the serial monitor comes back up. I think previously, the errors would stay in the console so they could be read.

[gepd]

Thanks for the report @airdrummingfool

2. Fixed, now it will not ask for a port when the IP is selected.

• The "Add IP Manually" is available again

• The first error in the console was related with the serial monitor, and it is fixed too

• The second one I was not able to reproduce it, so if you keep seeing it, please let me know to search it again

3. Fixed

4. it will available soon again, it was never officially released, that is why is not in the main list of this post.

5. Can you check if there is an error after the Starting... message, it should be working same as the past

6. Fixed.

[airdrummingfool]

Wow, thanks for addressing things so quickly! Here's some updates:

2. When I did "Add IP Manually" and typed in an IP address, it immediately triggered a build and upload. I wasn't able to reproduce this after the first instance.

• The second error appears to happen when I select the "Generic (192.168.1.119)" entry in the Serial Ports list. That entry corresponds to my ESP on Wifi, and doesn't always show up when it should. However, when it does show up and I select it, I get the following error in the console:

Traceback (most recent call last):
  File "/Users/Tommy/Library/Application Support/Sublime Text 3/Packages/Deviot/libraries/quick_menu.py", line 189, in callback_serial_ports
    port_selected = port_list[selected][1]
IndexError: list index out of range

3. I still get the same error as described before: There is no application set to open the URL .... I saw that your commit referenced opening the library folder "when it's not created yet", but my library folder exists and has files in it. I was able to browse to it manually by typing the address exactly as it appears in the error message. It looks like the error message is stating that MacOS things Deviot is trying to open that directory as if it were a file; instead of opening the directory in Finder it is trying to figure out what application can handle opening it.

4. Yes, there is an error message in the console when I try to build a .cpp file (TimeHelper.cpp, in the same directory as my main ino file):

Exception in thread Thread-11:
Traceback (most recent call last):
  File "./python3.3/threading.py", line 901, in _bootstrap_inner
  File "./python3.3/threading.py", line 858, in run
  File "/Users/Tommy/Library/Application Support/Sublime Text 3/Packages/Deviot/platformio/compile.py", line 38, in start_compilation
    self.add_board()
  File "/Users/Tommy/Library/Application Support/Sublime Text 3/Packages/Deviot/platformio/initialize.py", line 59, in add_board
    self.run_command(cmd)
  File "/Users/Tommy/Library/Application Support/Sublime Text 3/Packages/Deviot/platformio/command.py", line 41, in run_command
    universal_newlines=True, shell=True)
  File "./python3.3/subprocess.py", line 819, in __init__
  File "./python3.3/subprocess.py", line 1448, in _execute_child
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/Deviot/TimeHelper'

6. Confirmed fixed! Errors remain in the deviot console as expected.

[gepd]

2. That was because the first error let the setting corrupted, but after that, it was automatically fix, that is why only happened one time.

However this behavior will happen always when you try to compile a sketch and it's interrupt to select a board or port, in that case it will always continue compiling/uploading the sketch automatically.

-I have increased the time to wait the mdns service, I hope it improves the reliability detecting it (the IP). My ESP displays the same name (Generic) and I do not see that error, so I left two prints to know the list of ports. It will be printed after you select the port, please paste the output here

It will look like this:

[['SELECT A PORT FROM THE LIST'], ['Add IP Manually'], ['Silicon Labs CP210x USB to UART Bridge (COM3)', 'COM3'], ['Generic (192.168.0.17)', '192.168.0.17', 'no']]
3

3. It should be fixed now.

4. Fixed too!

[airdrummingfool]

2. Here's the debug output. It appears that the "Generic" entry is disappearing as soon as it is clicked:

[['SELECT A PORT FROM THE LIST'], ['Add IP Manually'], ['/dev/tty.Bluetooth-Incoming-Port', '/dev/tty.Bluetooth-Incoming-Port'], ['/dev/tty.UEBOOM-LWACP', '/dev/tty.UEBOOM-LWACP'], ['/dev/tty.TommysiPhone', '/dev/tty.TommysiPhone']]
5
Traceback (most recent call last):
  File "/Users/Tommy/Library/Application Support/Sublime Text 3/Packages/Deviot/libraries/quick_menu.py", line 191, in callback_serial_ports
    port_selected = port_list[selected][1]
IndexError: list index out of range

3. Confirmed fixed!

4. Confirmed fixed!

Thanks!

[gepd]

Let me know if the last commit solved the problem

[airdrummingfool]

Yep, that seems to have fixed it. I'm still having a few issues with OTA:

• My ESP doesn't reliably show up in the Serial Ports menu (I have to open the menu 3-6 times before it shows up, and then it likely doesn't show up the next time I open the menu). This also occurred using the old version of Deviot, so it's nothing new.
• I had one OTA upload fail (probably not Deviot's fault), but when I retried it, my OTA password was no longer saved (it didn't send one). Unfortunately, I can't seem to reproduce this, so I'm not exactly sure where the issue is (though it seems that changing ports does not erase the password, which is good).

Those are really minor issues that probably don't affect Deviot much - I'll keep testing and let you know if I find anything else. Thanks again!

[gepd]

Thanks you for all reports!

• I'm working to improve the reliability discovering the IP ports, the networking subject is quite new to me.
• When Deviot discover the device, besides of the IP it get if auth is set to true or false, so when is false it will remove it automatically from platformio.ini and when is true it's added (the password is always in the preferences file).

What could be causing the problem is the reliability reading the device information, Deviot checks this information each time before to upload the sketch. So I'll keep adjusting this last point.

I'll let you know when I push a new fix for this

[gepd]

Hey @airdrummingfool I've though in a way to improve the reliability of the IPs, I've pushed some changes, let me know if it make a change in your machine

[airdrummingfool]

@gepd that is definitely an improvement! I've opened the serial port menu now ~20 times, and it has only not been listed once. Previously I would only see it one out of every five or so times I checked the menu.

[jwhendy]

Confirming that #64 seem to be alright so far. I'm on arch linux with /usr/bin/python --> /usr/bin/python3.6 so Deviot is picking up python2 nicely (I had to change the symlink previously every time I used it). This is the first time in a long while I've been able to build/upload from Deviot vs. using platformio directly. Thanks!

[gepd]

Thanks for the report @jwhendy!

[m4rcu5]

I thought that after #119 the Python detection also worked on Gentoo. But after a fresh install, I still get the warning to install Python. A couple of sublime restarts later, I do get the Deviot menu, but it seems non-functional.

The terminal gives me the following output:

$ pio -h 
/bin/sh: /home/mvandam/.config/sublime-text-3/Packages/User/Deviot/penv/bin/pio: No such file or directory

How can I troubleshoot this?

[jwhendy]

@m4rcu5 What strikes me is that it's looking in the Deviot package, whereas my pio is owned by platformio itself. Or perhaps I'm not clear on if Deviot ships with it's own platformio as well? Here's some reference info from arch linux:

$ pio --version
PlatformIO, version 3.3.1

$ which pio
/usr/bin/pio

$ pacman -Qo /usr/bin/pio
/usr/bin/pio is owned by platformio 3.3.1-1

[m4rcu5]

@jwhendy Your PlatformIO seems to be installed by pacman (Arch Linux has a nice AUR package for it, opposed to Gentoo). In that case this function https://github.com/gepd/Deviot/blob/deviot_2/beginning/pio_install.py#L251 will exit the installer routine and use the external PlatformIO.
In the case (like mine) where there is no version of PlatoformIO found, this function https://github.com/gepd/Deviot/blob/deviot_2/beginning/pio_install.py#L126 should install PlatformIO using virtualenv (installing venv first), which I guess is failing. I see no traces of virtual env nor PlatformIO components in my Packages folder.

[m4rcu5]

I found some related log entries. The following is logged during the package installation. At this point I also get the prompt to install Python2.7 although this is already installed.

Exception in thread Thread-21:
Traceback (most recent call last):
  File "./python3.3/threading.py", line 901, in _bootstrap_inner
  File "./python3.3/threading.py", line 858, in run
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/libraries/messages.py", line 151, in <lambda>
    thread = Thread(target=lambda: self.print())
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/libraries/messages.py", line 166, in print
    self.print_screen(text)
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/libraries/messages.py", line 27, in print_screen
    size = self.panel.size()
AttributeError: 'NoneType' object has no attribute 'size'

reloading /home/mvandam/.config/sublime-text-3/Packages/User/Package Control.sublime-settings
reloading settings Packages/User/Package Control.sublime-settings
reloading settings Packages/User/Preferences.sublime-settings
reloading settings Packages/User/deviot.sublime-settings
Exception in thread Thread-23:
Traceback (most recent call last):
  File "./python3.3/threading.py", line 901, in _bootstrap_inner
  File "./python3.3/threading.py", line 858, in run
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/platformio/update.py", line 148, in check_update
    pio_version_int = int(sub(r'\D', '', pio_version))
ValueError: invalid literal for int() with base 10: ''

[jwhendy]

Thanks for the clarification, and I did not understand that. Would it goof things if your virtualenv was using the python3 version? I forget why, but I have a commented .bashrc line with this: alias virtualenv='/usr/bin/virtualenv2'. At some point, I must have needed to do that, and made me wonder if pio is being installed in the wrong env. Total stab in the dark and I'll probably just keep my mouth shut now and wait for the experts!

[gepd]

Fixed that erros in the console (there was not related with symlink)

• I've installed Arch Linux and I had checked the problem, I've also push a fix for that. I've test the installation with PlatformIO already in the machine and without it. (@jwhendy)

Please confirm you haven't problem too

• Plus I've added a new option in Options > Rebuild Boards Files for the cases when boards.js is corrupt. And the way as that file was being generated was improveed too.

If you can't see the new option your menu go to Language and select your current language to regenerate the menu file.

[m4rcu5]

It seems your latest commits have fixed my issues on startup.

One more console log remaining during first startup:

Exception in thread Thread-26:
Traceback (most recent call last):
  File "./python3.3/threading.py", line 901, in _bootstrap_inner
  File "./python3.3/threading.py", line 858, in run
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/platformio/update.py", line 148, in check_update
    pio_version_int = int(sub(r'\D', '', pio_version))
ValueError: invalid literal for int() with base 10: ''

[gepd]

Thanks, fixed with c278181

[m4rcu5]

Creating a new sketch, results in the following error after selecting a path:

Traceback (most recent call last):
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/libraries/paths.py", line 408, in <lambda>
    sublime.set_timeout(lambda: quick_panel(paths_list, lambda index: folder_explorer(path, callback, key, paths_list, index)), 0)
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/libraries/paths.py", line 378, in folder_explorer
    return callback(path)
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/commands/deviot_new_sketch.py", line 21, in on_done
    create_sketch(self.name, path)
  File "/home/mvandam/.config/sublime-text-3/Packages/Deviot/libraries/tools.py", line 186, in create_sketch
    sketch_path = path.join(path, sketch_name)
TypeError: join() takes exactly one argument (2 given)

I noticed that the path selection is acting a bit erratic. "Select Current Dir ($PATH)" sometimes displays 0 as current path. Or lacks behind, for example entering /home/mvandam/git, will still show /home/mvandam as current path.

[gepd]

Fixed both issues :)

[m4rcu5]

@gepd I can confirm :)
I just played around with Deviot2 for a couple of hours and have not encountered any issues.

Something I do notice is that, even when not actively using any Deviot functions, the plugin is using up to 25% CPU on my machine. Do you have any background loops running?

[gepd]

Most of the process are running in background because the main thread is used for the UI

I'm working in the completions and the highlight features, I'll finishing that to check whatI let in the background.

Thanks a lot for your reports!