Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simulator for ev3dev or ev3dev-python #264

Open
DJuego opened this issue Nov 24, 2016 · 38 comments

Comments

Projects
None yet
@DJuego
Copy link

commented Nov 24, 2016

Hi!

I have ten hours of experience with ev3dev-python and I am pretty pleased. In fact I am considering seriously to use python-ev3dev in a educational environment.

The only weakness that I find is lacking of a simulator for learning/experimenting/working without robot.

Of course, to develop an accurate emulator is a HUMONGOUS project (!!!). However a toy simulator , similar to Open Roberta simulator would be a fantastic addition: A 2D top-down view of a two drive wheels mobile robot offers potential for learning and it is an inmediate visual feedback

Any related projects (robotic simulators in python): here, here and here

Any thought, suggestion, hint, idea or link in this direction? 👍

DJuego

P.S V-Rep seems promising. Would it be adapted?

@WasabiFan

This comment has been minimized.

Copy link
Member

commented Nov 25, 2016

This is a very interesting concept. Firstly, I think that something like this would need to be implemented at the language binding/user program level, as both emulating the OS and drivers in situ would be a lot of work to do efficiently without a bunch of installation work. However, I think it would be relatively easy to modify the existing Python library so that file I/O could be replaced with proper calls to a simulated environment (preferably something that could be re-used by others as well). However, such an environment would need a "mock" implementation of the ev3dev motor and sensor drivers; this presents the challenge of keeping them in sync with changes to the real ones, because nuances can make a large difference. Optimally, you'd want a way to compile the existing drivers standalone for userspace so that you could both feed them your own input and intercept writes to PWM for motors. @dlech am I correct in assuming that there is no means of doing this that wouldn't cause difficulties in core driver maintenance? Do you have any ideas in this area?

In the end, I think that all the wrapper logic around the motor drivers would be easy, assuming any physics would be limited to 2D and an simple robot model. It's just a matter of getting the realistic logic working that translates ev3dev driver inputs to motor power outputs.

@ensonic

This comment has been minimized.

Copy link
Contributor

commented Nov 25, 2016

@DJuego did you try the OpenRoberta system (https://lab.open-roberta.org). This has a built-inb simulator for the ev3/nxt. It is a bit higher level (we're using this in classrooms with kids staring 3rd grade).

@DJuego

This comment has been minimized.

Copy link
Author

commented Nov 25, 2016

Thank you for your comments, @WasabiFan, Yes, yes. That is the idea. In the same vein of what i was thinking. Perhaps it is more easy to use the remote API of V-Rep. See this example.

@ensonic: Yes, yes. I know OpenRoberta System. :-) It is a powerful tool (intuitive, free, online but it is possible to work offline,...). In fact, I request to translate that simulator concept to ev3dev-python.

P.S:@ensonic, Do you use ev3dev with OpenRoberta? Does it work well? Or is it better to use the operating system for Open Roberta? What is your experience?

@ensonic

This comment has been minimized.

Copy link
Contributor

commented Nov 25, 2016

@DJuego are you sometimes in IRC or the gitter chat? Don't want to hijack this bug report for discussion. Anyway, as quick pointers:

  • works offline: run the openroberta server on a raspberry pi 3 (I am doing this for events)
  • using ev3dev: you need a custom firmware for the ev3 anyway, if you use lejos, openroberta generates java, if you use ev3dev then openroberta generates python. I wrote the ev3dev connector for openroberta (https://github.com/OpenRoberta/robertalab-ev3dev) and I've been using it for quite a few events.
@dlech

This comment has been minimized.

Copy link
Member

commented Nov 26, 2016

@dlech am I correct in assuming that there is no means of doing this that wouldn't cause difficulties in core driver maintenance? Do you have any ideas in this area?

I would not recommend using Linux drivers as part of the simulation stack. It would limit you to running simulations on Linux only and add unnecessary complexity.

However, I think it would be relatively easy to modify the existing Python library so that file I/O could be replaced with proper calls to a simulated environment

I think this is the "right way" to go about doing something like this.

@WasabiFan

This comment has been minimized.

Copy link
Member

commented Nov 26, 2016

The main issue that I see here is the ev3dev simulation logic itself: This type of virtual environment would need to have an implementation of the motor drivers available to translate driver input to duty cycle. Do you have any suggestions there? Or is it simple enough that an alternate implementation could be written and parity maintained without issue?

@dlech

This comment has been minimized.

Copy link
Member

commented Nov 26, 2016

This type of virtual environment would need to have an implementation of the motor drivers available to translate driver input to duty cycle

Not necessarily. If you are telling the motor to run at a fixed speed, then just have the simulation run the motors at that speed. If you are making a simple OpenRoberta-style simulator, there is no need to model the actual physics of the motor. For example, if speed_sp=400 and the simulation has a wheel diameter of 20 pixels, then the robot should be moving across the screen at 400 tacho counts per second / 360 tacho counts per rotation * (3.14 * 20) pixels per rotation = 69.8 pixels / second. Duty cycle does not come in to this equation at all.

If you want a simulator that is fancier than that, then you are beyond my expertise. 😃

@WasabiFan

This comment has been minimized.

Copy link
Member

commented Nov 26, 2016

The problem is that you need to expose something resembling the interface of the real drivers. That means that commands, polarity, position, input constraints, PID dynamics (for positional control), ramping, state and stop actions would all need to be operational as they are in the real environment for consistent results. You could, of course, decide that it would only support the speed-control mode, immediate breaking, "running" state, etc., but at that point the simulation would probably not actually simulate real circumstances and would fail when you tried to use more than just the bare essentials of functionality.

I'll have to think over how this could work. There isn't an obvious answer that I can come up with but maybe I'll think of something.

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Dec 9, 2016

Of the simulators linked above do we know if any them support a plugin architecture where we could swap out them emulating say EVG-3 for a block of Python code?

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 20, 2017

I started playing around with making a very basic simulator. The init goal is just to have a single large motor appear as connected, will worry about simulating functionality later.

The problem is that we need to write to /sys/ which you cannot do from user space, even as root. What if we could trick the kernel into thinking a motor is connected...that would get us past the inability to write to /sys/.

@dlech when a motor is connected how does the kernel detect that it is connected?

@dlech

This comment has been minimized.

Copy link
Member

commented Jan 20, 2017

You could write a dummy kernel driver to simulate the sysfs structure, but I think it would be much easier to just replace the functions that wrap calls to sysfs with something else. If this simulator is going to run on non-Linux systems, then that is also your only choice. FWIW, you can find the EV3 output port device connection manager here.

@WasabiFan

This comment has been minimized.

Copy link
Member

commented Jan 20, 2017

In my mind, I think the ideal structure would be to intercept calls one level up: make an abstraction layer between the core library code and file I/O, then have an implementation of that layer which calls a separate Python module which operates the simulator. That avoids the file I/O and works cross-platform. Preferably the simulator code would expose an interface which is also usable by other languages.

@ensonic

This comment has been minimized.

Copy link
Contributor

commented Jan 21, 2017

Can we mount an overlayfs to sysfs to shadow the real files? Or we have a fusefs replicating the relevant sysfs structure and the language bindings try two locations (first the fusefs one, then the sysfs one). We could even use some API to set the prefix and thus select between real and emulated.

@WasabiFan

This comment has been minimized.

Copy link
Member

commented Jan 21, 2017

What I'd like to know is: Why would we want to emulate the files in any fs? If our goal is to make a simulator that runs on the desktop, isn't making it dependent on a Linux kernel counterproductive? I think we should be designing this thing to be entirely platform-independent, and cut out all the file I/O in the simulator.

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 21, 2017

I was going to take the ev3dev bin and run it in a virtual box VM, I figure the closer we are to the real thing the better and virtual box will run on windows, linux and mac. My thought was the user launches the emulator then launches their robot code without making any changes to the robot code.

I think dlech makes a good point though that it might just be easier to have the python library look somewhere other than /sys/ if we want to run in emulator mode. So the user would

  • start the emulator which creates the needed filesystem structure somewhere other than /sys/
  • start the robot but pass it some flag on the command line to tell it to look at the emulated directory instead of /sys/

Perhaps then we wouldn't need virtual box.

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 21, 2017

@WasabiFan if the filesystem is emulated then we shouldn't need to change anything in ev3dev-lang-python other than telling it to look at the emulated directory instead of /sys/. That seems much easier than adding a code path to ev3dev-lang-python that allows it to function without file I/O. Maybe I am missing something though, what are your thoughts on how to do the simulator without file I/O?

@ensonic

This comment has been minimized.

Copy link
Contributor

commented Jan 21, 2017

@dwalton76 an alternative would be to do this in the platform probing and have a new emulated platform

@rhempel

This comment has been minimized.

Copy link
Member

commented Jan 21, 2017

Am I missing something? Don't we just need to change

'''
DEVICE_ROOT_PATH = '/sys/class'
'''

In 'core.py'

@WasabiFan

This comment has been minimized.

Copy link
Member

commented Jan 22, 2017

Am I missing something? Don't we just need to change

No, you're not missing something; if we want to have the library write to files and then the simulator read from them, that'll work. However, I am advocating for entirely bypassing the file I/O and intercepting calls between our external library interfaces and the I/O calls; that way we aren't using the hard drive for data transfer between modules, which seems nonsensical.

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 25, 2017

I'll be honest I am not connecting the dots yet on what clear advantage we get by avoiding file I/O. Sure we could come up with something that avoids file I/O but why are we concerned about file I/O? My gut says we could pull off an emulator pretty quickly if we just point DEVICE_ROOT_PATH at a directory that is being manipulated by the emulator and in theory we shouldn't have to change anything else in ev3dev-lang-python. But if we want to do an emulator that avoids file I/O I don't see how we could do that without significant changes in ev3dev-lang-python.

@WasabiFan

This comment has been minimized.

Copy link
Member

commented Jan 26, 2017

I'll be honest I am not connecting the dots yet on what clear advantage we get by avoiding file I/O.

Disk I/O is not an interface to be used for live communication. Hard drives and their related software are designed for long-term storage, and as such they don't work well when you want to talk to other usermode software: it doesn't immediately invoke the other app, it puts needless pressure on system resources, and it wears down the physical media. Maybe this is a difference stemming from the fact that I've learned with Windows while you have with *Nix; it's important to keep in mind, however, that sysfs and memory-mapped files are not the same as real files on disk.

But if we want to do an emulator that avoids file I/O I don't see how we could do that without significant changes in ev3dev-lang-python.

The only I/O calls that motors and sensors invoke comes from the Device class. All we'd need is an alternative implementation of three methods. In reality, implementing this simulator while bypassing file I/O is likely easier than writing one that has to handle high-frequency reads and file watching.

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 26, 2017

I think part of our misunderstanding here is assumption that file I/O implies disk I/O. which as you pointed out, is not always the case in linux given filesystems like tmpfs which are using ram and not a hard drive. So if we went the "emulate the filesystem" route we would certainly want to avoid writing to the SD card all the time and would use tmpfs or some equivalent. /var/run/ is one place we could do this, it uses tmpfs.

In terms of using the filesystem (not the disk) for communication, user space apps do this all the time with the kernel, the fact that we would have two user space apps communicating this way isn't a bad thing IMO. Heck even if we had the emulator communicate with the ev3dev-python code via a unix domain socket that is still just a file in the file system.

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 26, 2017

The only I/O calls that motors and sensors invoke comes from the Device class. All we'd need is an alternative implementation of three methods. In reality, implementing this simulator while bypassing file I/O is likely easier than writing one that has to handle high-frequency reads and file watching

But if we do this then the way you troubleshoot a robot changes a good bit if it is being emulated. I often do a cat of a file in /sys/class/tacho-motor/ to get the position of a motor or I echo a number into one of those files to write some value.

If the emulator and the app are not communicating via the filesystem in the same way the kernel and the app would communicate, then that is pretty fundamental change even if it is only three methods in the Device class that would need to be changed.

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 26, 2017

Something else that just came to mind, if we simulate the filesystem then it should be trivial for the other ev3dev programming language repos (Go, C, C++, etc) to also take advantage of the emulator.

@WasabiFan

This comment has been minimized.

Copy link
Member

commented Jan 26, 2017

I think I'm starting to see where our thinking diverged.

In my mind, an ev3dev simulator as described in this issue is software that allows you to run your ev3dev code on a development PC (whether that happens to be a desktop, laptop or dead badger, running any Python-supported OS) without the need for an EV3 or ev3dev installation available. There's no "SD card" in that scenario, and file I/O must inherently be assumed to be disk I/O (unless you fancy writing file system drivers for the aforementioned dead badger).

What have others been imagining?

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 26, 2017

I guess I was picturing a virtual box image of ev3dev (plus the emulator), that way everything is as close as we can get it to the real thing. If they need to "apt-get install python-opencv" in the VM to make their robot work then they know they'll need to do the same thing on their ev3.

@Dave-Baum

This comment has been minimized.

Copy link

commented Feb 7, 2017

A virtual box is going to get as close as possible to running on real hardware. There are certain kinds of problems that will only show up that way. But there is also a lot of value in a less faithful simulation that runs natively on your development computer. Easier setup, using your IDE's debugger will be a breeze, etc. From my perspective, if real hardware is available then the primary advantage of a simulator is faster iteration, better debugging, and setup of test environments. I suspect all of these are going to be simpler and easier when running native.

With native simulation there are still two basic options:

  1. Manipulate stuff in the filesystem namespace so that ev3dev runs unmodified, except for perhaps a new root for where to look for files. This has a good architectural boundary and allows the simulator to be used with any language. I think the biggest risk is coming up with a portable and efficient mechanism to watch a hundred files for writes.

  2. Introduce an abstraction layer near the bottom of python ev3dev/core.py that allows the simulation to patch in its own attribute objects. They can still reside in a filesystem like namespace, but they are going to behave more like device drivers (albeit written in python). Super easy to make this portable (and pure python), but the boundary isn't quite as clean. I wrote a very simple proof-of-concept by monkey patching ev3dev. I think the approach is sound and would be cleaner if we introduced a real abstraction layer instead of patching. I think this also paves the way for fast unit tests of ev3dev and/or user code.

@loopspace

This comment has been minimized.

Copy link

commented Apr 4, 2017

I'd like to chime in on this thread from an education point of view.

What would be really useful to have is a python package that could be imported into a script on a PC and used to do a simple sanity check on a program before it is transferred to the device running ev3dev. This would enable the students to check for standard errors (indentation, missing colons, 'true' instead of 'True') without the overhead of transferring to the device.

It wouldn't have to be all that accurate, and wouldn't have to provide any sort of graphical/textual demonstration of what was going on. These would be nice, but I'd go with the maxim that the perfect can be the enemy of the good, and say that something that simply allowed program validation would be a huge benefit here.

Education environments tend to be locked down, so a simple module would be the best solution here.

@Dave-Baum

This comment has been minimized.

Copy link

commented Apr 4, 2017

Compiling is a good way to catch indentation and syntax errors. I don't think it will catch true vs. True, or most other cases of typos in variable/function/method names.

@Symbian9

This comment has been minimized.

Copy link

commented Jul 22, 2017

What about re-born RoboSim?

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Jan 11, 2018

@WasabiFan @ddemidov @dlech we've had this open for 14+ months and none of us has gotten to it, any objections to closing this one? We can re-open if someone has the urge to work on it.

@dlech

This comment has been minimized.

Copy link
Member

commented Jan 11, 2018

I don't see any harm in leaving it open - just tag it as "wishlist" or something like that so people know it is not likely to be implemented.

@WasabiFan WasabiFan added the wishlist label Jan 11, 2018

@alanwilter

This comment has been minimized.

Copy link

commented Jan 16, 2018

Sorry if it sounds I am hijacking this thread, but my idea for simulator right now would be something like this:

  • I have a backup, on my Mac, of the SD Card flashed and configured with eve3dev;
  • I would like to use this backup file to load in a, say, Virtual Box EV3 brick simulator, and update, install etc. using a likely much faster environment with steady internet connection;
  • then flash back the updated backup on my SD card and put it on my brick.

The point for me here is that EV3 Brick hardware is very slow for, say, upgrading ev3dev kernel and debian packages. It's time consuming and battery draining.

Do we have anything around that could do this?

@dlech

This comment has been minimized.

Copy link
Member

commented Jan 16, 2018

@alanwilter: It will take you much longer than running things on the EV3, but it is possible. It works like this:

  1. on a linux machine (or virtual machine) mount the EV3DEV_ROOTFS partition somewhere
  2. mount the EV3DEV_BOOT partition to boot/flash of the rootfs that you just mounted
  3. copy qemu-arm-static to usr/bin in the rootfs
  4. poke a few more files to prevent services from trying to run when you install packages
  5. poke some more files to make flash-kernel work correctly
  6. chroot into the rootfs and upgrade your packages
  7. exit the chroot and undo all of the poking and copying of files that you did
  8. unmount the partitions
  9. put the SD card back in the EV3 and hope you didn't break anything

Probably better to invest in some rechargeable batteries.

@Zincfox

This comment has been minimized.

Copy link

commented Feb 14, 2018

Hey, I just thought I'd let you guys know that I'm giving this a shot. Unfortunately it's been quite some time since I've used python but what's the worst that could happen? I already made some progress and will make my repo public once I feel like I have something presentable. I do not plan to add anything fancy like a position tracker or the ability to use duty-cycle (at least currently). The GUI will just consist of the Brick, the screen/display + buttons and various slots for the sensors (probably interactive) and motors.
My current approach is to have a "host" process and a "client" process which can communicate over a multiprocessing-Pipe. The host will start the client and hold the current robot-state and also do the rendering. The fs-methods in init.py (ev3dev2) will be overwritten by the client process before the main code starts by "catching" the import of the init.py and replacing the methods of the delivered file with the simulator bound version. This has two advantages:

  1. The "real" and the "simulated" robot code are separated - no checking for simulators on the actual brick and no trying to access the filesystem (aside from the code itself of course) on the simulator.
  2. Backwards-compatibility. Because the modification is "in place", the original source code doesn't need to be modified - I hope that it's possible to support ev3dev and ev3dev2 with this setup (the different names allow for loading different "patches" - if the methods fluctuated to much in the past it should in theory also be possible to use version-files you seem to have in place).

I also have the idea of using a "progressive" simulation model instead of an iterative one. An example would be the Motor-component: Instead of incrementing the rotation by the current speed each tick, I simulate the rotation as a (currently linear) mathematical function. Setting values recalculates the parameters for these functions (the slope for rotational speed) and sets the current time. Then every time those values are read the current value is calculated by using the (now different) time. This should make the robot behave more realistically as we don't risk loosing "simulation ticks". Currently this is only implemented for motors and obviously isn't appropriate for every device - the buttons should obviously just stay atomic.

What do you guys think? I abstracted the simulator-simulatee gap into an "connector" class which should make it possible to switch to fs- or port-based communication later on to support other languages if this works out.

@dwalton76

This comment has been minimized.

Copy link
Collaborator

commented Feb 14, 2018

Sounds great to me 👍 Looking forward to trying it out

@cho934

This comment has been minimized.

Copy link

commented Apr 1, 2019

Sorry if it sounds I am hijacking this thread, but my idea for simulator right now would be something like this:

  • I have a backup, on my Mac, of the SD Card flashed and configured with eve3dev;
  • I would like to use this backup file to load in a, say, Virtual Box EV3 brick simulator, and update, install etc. using a likely much faster environment with steady internet connection;
  • then flash back the updated backup on my SD card and put it on my brick.

The point for me here is that EV3 Brick hardware is very slow for, say, upgrading ev3dev kernel and debian packages. It's time consuming and battery draining.

Do we have anything around that could do this?

It would be very interesting to update the ev3dev by taking the sd card inside a virtual machine, because it's very very long !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.