Flaky, the little triangle, lives in a 2D world. It has actuators (tiny little thrusters) and an sensor (an eye that casts rays into the world). The actuators take input and translate it into movement, whereas the sensors take input from the world around Flaky and translate it into output. In-/output is just a set of floating point values between 0 and 1. These could easily be interpreted as e.g. neuron potential.
All elements in the world are subject to a physics simulation. Depending on what Flaky hits on its way, items may move, collide, bounce, etc. This is done continuously in the background in real time. To save CPU cycles, physics calculation stopf if items are no longer moving.
The purpose of this little simulation is to allow for a connection with some kind of artificial neuronal network, which takes the input and produces output. This way any action that is performed by flaky has a direct impact on sensor data, which in turn can be processed by the neural net.
Everyone likes screenshots! This one is old, but it is suited to give an impression:
- 2013-09-21
It’s been a while! Updated the whole project to work with Qt 5.1, cleaned up some internal structure and updated documentation. Moved Client/Server controllers to separate shared classes and started work on flakysbrain.
- 2011-08-27
Updated this document to reflect everything that has happened in the last few months.
- 2011-03-13
Implemented network communication. Described it as interface specification. See below for details.
- 2011-02-27
Wrote this readme after code seems stable. Currently Flakysworld consists of the following basic object structure:- Bodies that are subject to physical interactions
- Sensors that can cast rays into the world and receive the nearest hit body as well as the distance.
- A world that contains bodies and manages physics logic using Box2D. — An engine that keeps the world in regular motion.
- A set of views for bodies/sensors resulting in QGraphicsItems that can be added to a scene/view in Qt.
- Basic GUI to see whats happening and perform rough control. Network communication layer is still missing, but soon to come.
- Feb 2011
Started the project after realizing that playing with ANN requires a simulated world to produce feedback for actions.
Currently the project is split in three parts:
- flakysworld is a console application and it is intended to run on a server. It listens on network interfaces for clients such as flakysview or flakysremote. It contains the world and calculates physics.
- flakysview is a Qt GUI application. It tries to connect to a flakysworld server (currently on localhost) and displays everything that happens there.
- flakysremote controls the actuators of Flaky. It currently simply sends a JSON string to flakysworld server on localhost. This way actuators (the tiny little thrusters) can be engaged.
The applications use Qt heavily for internal processing. Flakysworld additionally depends on Box2D for physics simulation. In-/output is given via JSON-wrapped objects over a socket, so the processing neuronal net does not have any technical dependencies. It even runs asynchronously by default.
The entities in the system communicate via Qt signal/slots wherever possible to keep a) the code clean and b) to allow for multiple threads without the hassle of mutexes. This way the application is basically divided into a processing unit, the universe within flakysworld. Flakysview models a set of widgets and views, the GUI. The network communication entity in each application is modeled as a simple socket connection that sends and receives JSON data. It ignores everything non-JSON (e.g. HTTP headers), so one can test output generation using a plain browser.
I’ve developed and tested everything using OSX 10.8, Qt 5.1 and Box2D 2.1.2. Qt and Box2D are platform independent, so feel free to run this on your windows/linux machine.
See below for the JSON interface spec.
Actuators, visual sensors and physical world interaction are implemented. Flakysview is a simple view that shows what happens in flakysworld. Data from sensors/to actuators can be received/sent using plain JSON over a socket. Development of brain has started, but will take a while.
Below you see an overview of the components of this whole projects, their curent development status and which kind of communication they have to handle.
- Google Protocol Buffers + ZeroMQ interface: Currently there is a socket connection using JSON for serialization. Other projects have shown that GPB and ZeroMQ are easier to maintain and more stable.
- Configurability: Port and interface should be configurable.
- Save/Load of the universe in a file or into a DB.
- Time flies mode: Do not keep in sync with real time, but run as fast as possible. May need sync signal for clients attached via network.
Data is sent and received over the same socket connection. Flakysworld currently binds to port 2345 on any interface. Sensor updates are sent from flakysworld to all connected clients. Every client can send actuator data to falkysworld. All data is in the form of plain JSON objects.
Here a more or less formal definition of a sensor output in form of a JSON object. Note that the characters
{}[]":
used here mean the actual characters, whereas
()
group elements as entity.
+*
mean “one or more” and “zero or more” of the previously described entity. Multiple occurrences of an entity have to be comma separated.
When connecting to flakysworld, a registration to a data feed is necessary. This is intended to allow applications to choose between world updates (shapes, positions and rotations) or AI relevant updates, i.e. sensors.
Registration is done by sending a JSON string the consists of the following elements:
{ "type" : "register", "concerns" : ( "WHAT" ) }
Where WHAT is either sensors or world. When registering to sensors, the registering client will receive current sensor status initially and updates when available, i.e. after each time step of physics simulation. When registering to world data, intially the current world is sent to the client completely, followed by updates when objects are moved.
Cients can register to multiple feeds. Disconnecting the socket removes registration. Data feed registration can be cancelled by a client using an unregister command:
{ "type" : "unregister", "concerns" : ( "WHAT" ) }
Sensor data is output according to the following scheme:
{ "type" : "sensoroutput", "being" : "BEING_ID", "sensors" : { ( "SENSOR_ID" : [ REAL* ] )+ } }
This means: For exactly one being, which is denoted by its ID (BEING_ID), there may be multiple sensors defined. Each sensor is described with its ID (SENSOR_ID) as key and an array consisting of multiple REAL values as sensor data. We define REAL as a value in [0..1], which is a closed interval.
The same scheme that is used for sensors holds for clients sending actuator data to flakysworld:
{ "type" : "actuatorinput", "being" : "BEING_ID", "actuators" : { ( "ACTUATOR_ID" : [ REAL* ] )+ } }
For a being, denoted by BEING_ID, multiple actuators can be described. Actuator data consists of key (ACTUATOR_ID) and value, an array of real values in [0..1].
Multiple data is, as usual in JSON, comma separated. A complete dataset, which has to be a JSON conform object, is finished by a newline. This holds for both directions. This means: Before a newline character, no JSON input is handled by flakysworld, and no complete output should be expected before a newline character.
Non-existing beings or actuators are silently ignored. At the moment, there is only one being, called flaky with only one sensor, called eye and two actuators thrl and thrr (thruster left and thruster right).
Our world consists of things. A thing is described by the following JSON data:
{ "type" : "thing", "thing" : "THING_ID", "shape" : [ ( { "x" : REAL, "y" : REAL } )* ], "position" : { "x" : REAL, "y" : REAL }, "rotation" : REAL }
Where THING_ID is usually a GUID string, except for flaky, where it is flaky. Thing’s shape is described as an array of points (x and y reals) in thing’s coordinates, so they have to be translated and rotated. Currently, a shape cannot be modified. Thing’s position is a point, too. Thing’s rotation is given in radians.
A possible return value for sensor data might look like this:
{ "type" : "sensoroutput", "being" : "flaky", "sensors" : { "eye" : [ 0.0298899, 0.032246, 0.0359727, 0.255273, 0.256103, 0.264073, 0.258331, 0.252873, 0.253623, 0.261305, 0.668216, 0.676755, 0.687302, 0.329974, 0.212865, 0.201672, 0.197739, 0.197683, 0.201475, 0.21226, 0.177706, 0.166087, 0.161403, 0.159795, 0.160635, 0.164229, 0.172641, 0.73212, 0.288457, 0.272602, 0.269097, 0.272273 ] } }
- Check out the code of Flakysworld.
- Make sure you have Qt SDK installed. I use 5.1. This includes Qt Creator, which I use as IDE.
- Make sure to have Box2D installed. On my machine it resides in /usr/local/include/Box2D and /usr/local/lib. This can be adjusted in the flakysworld.pro file to fit your needs.
- Load the .pro file with Qt Creator.
- Build and run.
To allow for easy comprehension and modification of this project the source is organized thematically. This chapter provides an insight of the object structure as well. Everything is in the src/ directory, which has the following subdirectoies:
being/Abstract being with its organs, i.e. sensors and actuators.bodies/Body, wich is an object in Box2D’s physics world. Contains specialized bodies (ellipse, polygon) as well.external/External libraries/headers. Currently contains Box2D for physics and QJson for serialization.flaky/Flaky is a being with specialized sensor eye.gui/Qt’s GUI stuff. Contains view classes for bodies and organs in further subdirectories.infrastructure/World (i.e. Box2D wrapper), engine (physics simulation thread) and universe (manages world and engine)interface/Serialization and TCP communication. Universe server and client are intended as interface so applications do not have to care for connections and serialization.
- Q: I don’t see anything!
- A: The rendering is done using OpenGL per default. This is set up by ui→graphicsView→setViewport() in surface.cpp. Comment this line if you need to work without OpenGL and use software rendering instead.
- Q: So how do I control Flaky?
- A: By sending values for the thrusters, one for left and right. This applies a pulse to the corresponding thruster. Currently, this happens using flakysremote.
- Q: Where is the AI?
- A: Nowhere here. The idea is to have a simulated world with an easy interface to connect any AI you like. To answer your question: That’s your job!
- Q: You call this “controls”? Are you serious?
- A: Yes, steering flaky using JSON strings is more than ugly to play with, but that’s okay. The purpose is just to test if interaction works basically. Go on and code a ANN that controls them in a sophisticated way, this is why all this is here!
- Q: Nah, I don’t get this.
- A: Dump all comments, questions and suggestion here: gre@g0r.de.
- Q: I like it. How can I help?
- A: Test, find bugs, improve performance, document, develop some stuff mentioned in the “Future Plans” section, do anything useful or nice you think may be fun!
All the code in Flakysworld, except its dependencies which have separate licenses (see below), is published under the MIT License. See here for details. Basically you can do whatever you like with it.
Qt and its libraries are licensed (as one possibility) under the LGPL. See Nokias licesing page for details.
QJson is LGPL 2.1 licensed. A copy of version 0.5.0 is present in flakysworld’s code.
Box2D has its own license:
Copyright © 2006-2007 Erin Catto http://www.gphysics.com
This software is provided ‘as-is’, without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software.
Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following
restrictions:
1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.