Online calibration feature implemented #39
Conversation
|
Very cool. Calibrating the axes is a serious issue, and this helps. I like the idea of having a button to press when setting the min/max values. What are
Is the If we extended the Calibrate model to another calibration, the surrounding code would need to understand the hidden state machine of the calibration? I guess that's not too bad-- they all likely need their own special data points anyway. One issue is that this means |
|
I think you refer to the code to save data for calibration: This code is a little tricky. Think in the case we make a reset before calibrate. We set a=1 and b=0. if we apply this we have : So we dont have transformation. We save the raw data. And if you apply all the ecuations you see you have a simple linear calibration If we calibrate once and we want to calibrate again, but composing with the new calibration with the current calibration. I need the data in that way, so i can resolve the ecuations to compose the calibration. It is tested and works. Perhaps, I need to put comment with the ecuations i develop to get this solution. So the user_value is the data the game see in that moment, but the controller is in the state the user wants at the end of the calibration. We only need to extend the calibration class and implement all the methods:
If you want to create a new calibration you can use whatever states you want. Because I define the enum inside the class, is not visible outside. With this interface calibratable only needs to know if the state machine is sleeping or not. For example: Now we can take 6 points to a second grade polinomial calibration. Other thing we need to extend, is the declaration model to can choose a calibration model or other. For instance: About wm_home, if you see the documentation: So we can put wm_home in first place or another key. for instance wm_1. It would be a good idea put power button of wiimote? I dont know if it is posible, but it would be very useful for this example. |
|
Ah. I think this is becoming clearer to me. The calibration just needs pairs of data points (user_value, game_value), where the first value is the observed (possibly already calibrated) value when the axis is held at a desired point, and the second value is the one we wish to output after calibration when we observe that user_value again. So it actually isn't important that the first test points be at the top left corner and the second test points at the bottom right, we just need two distinct locations. You described the calibration process as
If my understanding is correct, this would achieve the same calibration (but possibly harder to do precisely)
And then the equations are just the first order linear solution that passes through those two points. A quadratic solution would need three test points, and so on. From the perspective of the calibratable event translator, oblivious to the underlying calibration method, it just needs to keep sending (user_value, game_value) pairs until the underlying calibration is sleeping/satisfied. If this is the case, can we change the send_calibration_value interface to just directly take pairs rather than the hidden state machine? i.e. Overfitting polynomials can be a big issue as the degree increases, so it might be wiser to let the calibration receive more test points than it requires, and doing a least-squares solution. This might motivate using the start calibration button also as a stop calibration button, and let the user determine how many test points they care to send. (if they send too few, we can just abort the calibration). However, I don't expect users to ever want much more than a linear fit. About composing the calibrations: My understanding is that this just simplifies steps 2 and 4 of the process above, since setting the game-dependent values might be difficult if the original calibration is bad. Steps 1 and 3 should be the same regardless (i.e. same wiimote motions), and the math will work out to do the right thing: pointing the wiimote as in step 1 will output values that match step 2. I think if we just always send (raw user_value, raw game_value) pairs, the math works out the same. Since the game_value is just the raw data that makes the game respond as desired, and the user_value is just the raw data that comes from pointing the wiimote as desired, neither really depends on the calibration (i.e. these are just some ideal constants). Composing the calibrations just makes it easier to key in and set the game value we desire. Doing it this way would likely avoid some numerical/rounding issues from doing the intermediate calculations, and avoid needing to adjust like on lines 102/103 of calibration.h. Since a bad calibration makes it difficult to accurately set the test points, it is important to have the ability to reset it. However, I think composing the calibrations might be the better default behavior. Yes, it is good that the recalibrate button can be set to something else, but my issue was that whatever button it uses can't be used for anything else. However, in a wiimote-pointer setting, wm_1 and wm_2 are pretty good choices. They likely aren't being used for anything anyways. The wiimote power button is unavailable from the current linux kernel. |
Your understanding is correct. It would work, but is harder to be accurate because of human factor.
If we have configurated wm_home and wm_b. We have these cases:
That is the reason we have start calibration with the bool reset_calibration like argument: |
|
Another idea is: if would be possible to return some feedback to the user about the calibration process. For instance, we put a led blinking when we have saved data, or when we start or finish calibration. But I think it would be hard, because calibratable should be generic, this should works on wiimote or a gamepad or an steer wheel. |
|
Yes, I saw that you had the feature to reset the calibration. My comment was that we should flip the behavior, and compose by default and reset it in the special case. (Holding a second button is a special case). So perhaps we should change it to
Composing might seem more complicated/special from the code perspective, but I think resetting is the more special behavior from a user perspective. It would be cool to somehow signal to the user via the controller. Not currently possible in MG, but worth thinking about. It might be possible in the future to do this generically, as most game devices have at least one LED and rumble, so we could ask input sources to implement a generic "notify" method. This however, is not a priority on my to-do list. A similar future feature is making it so that certain messages from MG can be sent off as desktop notifications via D-Bus. I still think sending calibration value pairs is the better way. From a code maintenance perspective, it is a lot clearer what is going on, rather than passing different data through the same argument based off of a hidden counter. It makes it a ton clearer about what the calibration actually needs: input-output pairs. And a bit easier to modify. Regardless of whether it is the calibratable translator or the calibration, somewhere we need to store the values. I think the clarity gained by changing the calibration interface to take pairs is worth it. We can still do other data collecting orders, such as the user-user-game-game order example you gave. Instead, it would be the calibratable translator deciding the order (and storing the values to make pairs). I think this is actually a cleaner separation of responsibilities.
Overall, this is pretty cool, and a nice first implementation. |
|
About "flip the reset behavior" I agree. Could be more usable in that way. About "sending calibration value pairs", I am not agree. I am going to give you another example. You want to calibrate the y axes of an stick to adjust ABS_Y, CENTER and -ABS_Y. So you need only 3 points, not 3 pairs. In that case there isn't pairs. This is a case, but is probably, in future we found more, so in my point of view taking pairs is worse solution. We are losing flexibility and is not clearer. About that calibration should not save data, I agree, perhaps is clearer to move that task to calibratable. I have a proposal: class Calibration{
public:
virtual int get_number_samples_needed_to_calibrate() = 0;
// Get a string to show information about the sample input.
virtual std::string get_feedback_msg(int index) = 0;
// Reset calibration, now the class returns raw input.
virtual void reset_calibration();
// Readjust the calibration
virtual void calibrate(std::vector<int64_t>& input_samples) = 0;
// Aply the transformation from user_value to game value
virtual int64_t get_game_value(int64_t user_value) = 0;
virtual std::shared_ptr<Calibration> clone() = 0;
virtual ~Calibration(){};
};In this new interface we remove save_calibrate_data, start_calibration and is_calibrating. And we add 4 new functions:
calibratable now has to save the inputs on the vector and show feedback information. calibration only gives information to describe calibration input data and do the maths. |
|
I like those changes -- having a message for each index will make it a lot clearer, and I also like the idea of passing a vector of values. I see your point about possibly allowing non-pair-based calibrations. However, I don't think that example is the best one: it is still fits well as pairs. (user_min, output_min) (user_center, 0), (user_max, output_max), where output_min and output_max are the constants -ABS_RANGE and ABS_RANGE in the code. I'll go ahead and merge the current pull request onto devel, but I would still be interested in the modifications above if you find the time. Finally a couple ideas for interesting future calibrations:
|
|
I have been busy these days. I will add the changes we speak in the next days. I have a way to calibrate two axes at once with four points. I put it on documentation: We start the calibration and save the data with the same buttons at the two axes, and with 4 points the work is made. It is working, is tested. |
|
Okay, I look forward to the changes. I tried it out, and it works well on my end. Good work. That doesn't calibrate both axes in a way that uses info from both together, it does each separately. That wasn't what I meant. If you stand directly in front of the screen, scaling both axes separately does the right thing. However, imagine the extreme case, where you are standing off center. One side of the screen will be closer to you, it will be bigger from your perspective. From the perspective of the wiimote, the screen is a trapezoid rather than a rectangle. How much you want to scale the For a light-gun arcade game use, requiring the user to stand in the center is fair, and the innacuracy when off to the side is fine. In a smartboard set up like the one I linked, these perspective differences are much more important. It can be hard to line the wiimote up with the projector exactly, and even small inaccuracy from the non-rectangular perspective will be apparent when trying to draw on the projection. All of this is of course just an idea for future work, as I expect more people to use wii motes as light guns in arcade games rather than as a smartboard thing. |
|
Ok, I see your point with 2 axis calibration. We have now simple calibration, and this great project continues growing. I have committed the changes, but i havent tested that is working (Now, I am not in home). I only know that is compiling. I expect tomorrow I can test the changes. |
|
Merging onto devel, I presume it has been tested, and some more testing will be done before it hits master. |
|
I am trying to use this new calibration feature to calibrate my wiimote that I am using as a lightgun. The calibration itself appears to be working and I am getting expected values for a and b during the calibration process and it is being applied to the user input. The only weird thing that I am getting is that after calibrating the X axis (as an example, it also does it for Y), if I move the cursor to the left everything is lined up perfectly with the controller, however once I reach the leftmost position of the game screen that I set during the calibration, the cursor jumps back about one quarter of the screen and then if I continue moving left, the cursor will continue left until it hits a hard stop at the end of the game screen. I confirmed using jstest that the X value does spontaneously jump back when it reaches end of the game screen initially. I tried looking through the code to see if I could find the error, but I wasn't able to. Any help or thoughts as to how this is happening would be very much appreciated. |
|
This is likely due to the IR support in MG being really dumb. I took the quick-and-lazy approach to processing that data before passing it along to the event translators. The sensor bar has two clusters of IR LEDs. When both clusters are visible, MG outputs the IR X position as the average of these two clusters (e.g. relative to the middle of the sensor bar, which is what a user would expect.) When you move the wiimote too far, one of the two clusters goes out of view. MG sees only one IR point, and falls back to outputting the X position relative to the single visible cluster. This was chosen since when I started, PC-based sensor bars were rare. Even having one IR source for sensing was a luxury, so it was important to support both single-source DIY IR setups as well as two-source sensor bar setups. Also, the single-source is a lot easier to code. To avoid this behavior with two IR sources, some one will need to code up the IR processing of the wiimote MG driver to either:
Another possible solution is to step farther back: calibrate the IR data such that the full range of the screen can safely be pointed at without risking placing one of the IR sources out of view. Then this issue will only occur when you point particularly far left. Another possible cause of this problem: somewhere to the left you have reflected sunlight or something providing another IR source that confuses the IR data. |
|
I thought that might be the case where one set of LEDs was moving out of the camera, however when I reviewed the code, it looks like you are currently only ever tracking the left most LED: Which would mean that this wouldn't happen when moving the wiimote to the left. Likewise I wouldn't expect this same behavior along the Y axis since you would either see all LEDs or none, however I can replicate this issue in all 4 directions (up, down, left and right). I plan to add some debugging to see if I can figure it out, but I don't fully understand the code yet, especially the order things are fired. |
|
Also I should have mentioned I don't experience the same problem before I perform the calibration which leads be to believe that is somehow the cause. |
|
Ah, you seem to be right. The current code doesn't do any averaging, and it is suspicious that it occurs in all four directions. From what I recall, the wiimote IR sensor (and thus the linux kernel interface) provides X/Y pairs for up to 4 IR points. The sensor also performs some amount of tracking, such that IR point 2 should continue to be IR point 2, even if IR point 1 disappears and the wiimote moves a little. I don't think I ever tested how reliable this tracking is. MG ignores the 1/2/3/4 IDs of the IR points, and instead attempts to select the left-most IR point. These values are then scaled so that the edges of the sensor view become +/- ABS_RANGE. In terms of how things are fired:
So to debug the IR data, you likely want to modify compute_ir. |
|
I eventually figured out the issue, but I'm not sure of the best fix. I found that when the axis is calibrated, MoltenGamepad still emits the original uncalibrated axis event and then subsequently emits the new calibrated axis event before calling the SYN_REPORT. The trouble arises when the calibrated axis value becomes saturated (>32767 or <-32767), then because of the check to only emit the calibrated value when it has changed, MoltenGamepad stops emitting the calibrated value and therefore only the uncalibrated value is emitted. I manually fixed it by removing the saturation adjustment from calibration.get_game_value and added it to calibratable.process_syn_report after the value is cached. This way while the original value is changing it will continue to emit both values until they are both saturated. |
I have added a new feature that is online calibration. Yo can calibrate the gamepad while you are playing. How to use and a review of the features is explained in a file that I added to documentation.
I have implemented this like a new group that is calibratable. It needs a key to start calibration process, an axis to calibrate and other key to say save this value to calibrate the data. It is implemented thinking in using wiimote like light gun in MAME or nestopia, and calibrate the gun while you are playing. And it works!!
I have implemented a linear calibration, but you can extend the abstract class calibration and implement other calibration models. All we have to do is to add parameters to the group saying which calibration model we want.