Implementation of Perturbation in Id

[PaulTheLionHeart]

Paul:
Instead of having a global parameter to say we are using perturbation, why not use a g_params[] value that specifies we are using perturbation? That way we only offer it for fractals that can work on perturbation. I’m implementing it now and removing ‘p’ as a viable calc_mode.

Then I can do the work of fixing the plot modes without having to interpret the mode through ‘p’.

Richard:
Using a parameter is not the right way to go because things should be "obvious" and parameters are not for this purpose.

Perturbation is a speedup technique, but it can introduce errors from the "true" image because it is an approximation to the true orbit.

So there are three choices:

• force use of perturbation when available (perturbation=yes)
• inhibit use of perturbation (perturbation=no)
• let Id decide (perturbation=auto, the default)

Additionally there is the desire to specify the amount of tolerated error before a new reference point is forced:

perturbation=yes/1e-6

This implies a new variable in perturbation.h:

enum class PerturbationMode
{
    AUTO = 0, // the default
    YES = 1,
    NO = 2
};
extern PurturbationMode g_perturbation;
extern double g_perturbation_tolerance;

..and of course cmdfiles.cpp gets a new parameter parser for the parameter that sets the mode and tolerance.

When a fractal type indicates it has perturbation support in its flags, the standard engine consults the perturbation mode and calls the corresponding routines based on the mode.

The display should be updated to indicate whether or not perturbation is being used.

If you look at the UltraFractal docs, this is similar to what they have implemented: perturbation is used by default, but you can disable it and you can specify the error tolerance manually if you want.

Paul:
I don't think the Perturbation "errors" are worse than those created in solid guessing.

There are a number of reasons for not using Perturbation until we are into arbitrary precision as the code is slower than normal plotting modes. As the speed is relatively independent of the number of digits in the location, the only delay is the calculation of the reference matrix. In the current implementation branch (Perturbation250128) plotting modes work far better once we get to arbitrary precision anyway.

The outstanding issues are with plotting modes that have more than one pass, e.g. Guessing and two passes. Tesseral plotting is the fastest plotting mode at this stage. It will be good to get solid guessing working with perturbation for speed.

[LegalizeAdulthood]

I don't think the Perturbation "errors" are worse than those created in solid guessing.

Probably not, but every optimization of orbit computation, including the bailout test and periodicity checking, has a trade-off and sometimes you want to force things one way or another. For instance, technically bailout value of 4.0 is sufficient for determining inside and outside of the M-set, but when computing some types of images (for instance potential), you want to increase the bailout value dramatically beyond 4. Similarly, periodicity checking can be turned off if so desired. That's why I think we want to expose the ability for force perturbation calculations off and allow the user to specify the error tolerance. There's also the possibility that a perturbation formula was coded incorrectly. The image renders correct without perturbation, but renders incorrect with perturbation, so you want to force it off, even though it's slower.

UltraFractal (help section on perturbation) switches to perturbation if the formula supports it when arbitrary precision is being used. I think we can follow suit on that and only use perturbation when switching to arbitrary precision. UF has the settings "Precise", "Fast" and "Very Fast" for setting the error value used in perturbation. We could follow suit and map those terms to specific allowed amounts of error. That certainly simplifies things for the user instead of having to understand what numeric value should be used for the error.

There are a number of reasons for not using Perturbation until we are into arbitrary precision [...]

I think that's fine; it's what UF does.

The outstanding issues are with plotting modes that have more than one pass, e.g. Guessing and two passes. Tesseral plotting is the fastest plotting mode at this stage. It will be good to get solid guessing working with perturbation for speed.

I thought we covered that previously; the perturbation algorithm should be completely insensitive to the order in which pixels are plotted. If the current pixel generates too much error, then a new reference point needs to be chosen, etc. There's nothing in the perturbation algorithm that requires the entire image to be generated from a single reference point, nor does it require you to discard one reference point simply because a pixel couldn't be accurately computed from it.

[PaulTheLionHeart]

Maybe I should just keep working on the branch Perturbation250122. One reason for merging with master on my local Github was so I could use the mouse support.

[LegalizeAdulthood]

I think maybe we're talking past each other a little bit due to the way we are using terminology; it sounds like you're using the word 'merge' casually, whereas I am referring to the git merge command. You should aways use git rebase to get your branch updated with respect to master and never use git merge.

The reasons for this are esoteric in the way git works, but it is the policy I've chosen for keeping a linear history of commits on the master branch.

[PaulTheLionHeart]

When I say merge, I mean that I pull the latest master to my local repository with a git pull request. Then I get all the files I have changed since the last time I pulled master to my repository and di a compare. I add all my changes into the files from master in VC studio. Once I've done a reasonable amount of testing, I create a new branch, add all the changed files, commit and git push paul new_braqnch_name. So the changes are only on my local repository . When you let me know we are ready, then I do a pull request. So far, all the changes are on my local repository and my own github repository. I hope this clarifies what I've done.

[LegalizeAdulthood]

OK, I think what you're describing is essentially manually rebasing. This is fine, although I think as you get more comfortable with git you'll find that using git rebase will be more natural.

After all the integer fractal types were removed, the existing xxx_FP fractal types were renamed to drop the _FP suffix and a small number of help label names were changed to match, so when I pulled your branch and attempted to rebase it, I got a couple conflicts due to those changes. I have a Utah C++ Programmers presentation on Wednesday, so I won't be able to look at rebasing your branch onto master until after that. Then we can collaborate on working out whatever remaining issues there are with the various passes methods and perturbation.

[PaulTheLionHeart]

I haven't been successful with git rebasing. It's yet another challenge. I'm excited that we are so close to getting perturbation working through standard_calc(). Things are going very well.

[LegalizeAdulthood]

BTW, the name is either Iterated Dynamics (both words capitalized) or Id for short, not ID. :)

[PaulTheLionHeart]

"There's also the possibility that a perturbation formula was coded incorrectly. The image renders correct without perturbation, but renders incorrect with perturbation, so you want to force it off, even though it's slower."

The formulae are hard coded in id so far. I.e., Mandelbrot, Man z power and Burning Ship. Are you suggesting a perturbation formula parser? That would be cool. In fact, if we can write code to automatically interpret standard formulae into the appropriate perturbation algorithms, that would be even better. Maybe id52.0 :)

[LegalizeAdulthood]

Fussy!

I still get the capitalization wrong for ManpWIN :) ... not even sure if I got it correct right now!

[LegalizeAdulthood]

"There's also the possibility that a perturbation formula was coded incorrectly. The image renders correct without perturbation, but renders incorrect with perturbation, so you want to force it off, even though it's slower."

The formulae are hard coded in id so far. I.e., Mandelbrot, Man z power and Burning Ship. Are you suggesting a perturbation formula parser? That would be cool. In fact, if we can write code to automatically interpret standard formulae into the appropriate perturbation algorithms, that would be even better. Maybe id52.0 :)

There's always the possibility of bugs, either in the hard-coded C++ implementation of the perturbation formulas we've implemented, or in a user's perturbation formula. And yes, we're going to need to support perturbation formulas for the type=formula fractal type that we have now. UltraFractal has this and we should strive for parity. I've asked around to see if there is an existing parser for their data files, but that came up empty, so I'll be writing one at some point.

[PaulTheLionHeart]

Yes, The spelling is ManpWIN :)

Do we need to support perturbation formulae or simply be able to translate the more common fractal formulae into perturbation code? Polynomials and Mandelbrot derivativesare easy. Trig functions are much harder.

[LegalizeAdulthood]

Do we need to support perturbation formulae or simply be able to translate the more common fractal formulae into perturbation code? Polynomials and Mandelbrot derivativesare easy. Trig functions are much harder.

At some point we will want to allow the user to write their own perturbation formula for type=formula. For now we have the function pointers in the FractalSpecific structure and associated flag that indicates if a particular fractal type supports perturbation.

[PaulTheLionHeart]

"..and of course cmdfiles.cpp gets a new parameter parser for the parameter that sets the mode and tolerance."

Does that mean when we want to change the perturbation mode we have to edit a file? I don't understand how this all works. Is it set in sstools.ini? This seems far more difficult than simply having a field that we change. Wouldn't it be better to add it to the 'Z' screen or something?

[LegalizeAdulthood]

Does that mean when we want to change the perturbation mode we have to edit a file?

There are multiple ways to assign a value to a parameter:

• sstools.ini
• parameter files with @file/set-name
• the G command

For this new parameter, I think we would want to add it to one of the options screens. It probably belongs on the same screen as periodicity checking in passes_options.

Approaching the problem incrementally, first make a parameter that can be set from one of the 3 methods above (some parameters are can only be set at startup time, but this is not the case here) and later we can add an interactive setting on the passes options screen.

[PaulTheLionHeart]

I don't like having to edit a file just to make a small change. I even put in a direct formula parser into ManpWIN so I didn't have to keep writing formula files to test new algorithms. I can't wait until id has a new GUI as this all becomes so much easier.
I will choose the G command at this stage.

[LegalizeAdulthood]

Like I said, it can be added to the passes options screen.