Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

astrometry.net address hardcoded, impossible to use a local running nova server #80

Closed
papaf76 opened this issue Aug 17, 2021 · 67 comments
Closed

astrometry.net address hardcoded, impossible to use a local running nova server #80

papaf76 opened this issue Aug 17, 2021 · 67 comments

Comments

@papaf76
Copy link

papaf76 commented Aug 17, 2021

Hi,
looking here: https://github.com/rlancaste/stellarsolver/blob/master/stellarsolver/onlinesolver.cpp#L325
It seems the software is using a hardcoded value of nova.astrometry.net in place of the specified server (eg. from ekos config).
This makes it impossible to use a local copy of nova.
@rlancaste
Copy link
Owner

Hi there, we can definitely change that to support this. But I must ask, why do you need that? If it is on Mac, you can directly access astrometry.net's solve-field binary. If it is windows, you can directly access the Cygwin or ANSVR solve-field binary. Are you hosting another astrometry server on your local network on a different machine?

@rlancaste
Copy link
Owner

I think I found a post relating to that on the INDI forum just now. But I can't seem to log in. From that forum it looks like I found that you are hosting a local astrometry server for multiple imagers imaging nearby. That makes sense. Do you have any details about how you are hosting your astrometry server? Is this an ANSVR server or does it work exactly like nova.astrometry.net? If it is the second is that easy to set up? If I was going to make such a change I would need to be able to test it.

@papaf76
Copy link
Author

papaf76 commented Aug 18, 2021 via email

@papaf76
Copy link
Author

papaf76 commented Aug 18, 2021

BTW, I had to make a change to my ANSVR server just now, as I discovered the configuration panel from Ekos has an enormous issue: clicking or not on use position and/or use scale does nothing if you're not using the internal solver. Actually, I'm not able to check if it does something with the internal solver, as that doesn't log anything despite me having the verbose option on on the alignment log.
So I made it that my server always blind solves, no matter what the json object contains, it will always discard ra, dec and radius.

@laheller
Copy link

Hi there, we can definitely change that to support this. But I must ask, why do you need that? If it is on Mac, you can directly access astrometry.net's solve-field binary. If it is windows, you can directly access the Cygwin or ANSVR solve-field binary. Are you hosting another astrometry server on your local network on a different machine?

Hi @rlancaste

I have my own custom Cygwin build of astrometry.net on Windows. I also have KSTARS for Windows installed.

Can U please show me (when possible via a screenshot), how to access my solve-field binary from KSTARS?
For example when binary is at
C:\astrometry.net\bin\solve-field.exe
how to specify that in KSTARS/EKOS Windows version?

Another question is, whether is it possible also specify custom cmdline arguments to the solve-field (again from KSTARS/EKOS on Windows)?
Or it is calling it using hardcoded set of arguments?

Thanks and regards,

Ladislav

@rlancaste
Copy link
Owner

So one thing that I discovered when I was doing all this work on astrometry.net last year was that I could directly control ANSVR and Cygwin based astrometry.net on windows just like astrometry.net on Macs and Linux. This function is built into StellarSolver and also Ekos as a result. This method of access proved to be way way faster than using ANSVR the traditional way. This is why I made the change to just use online for the actual astrometry.net server and to use "local" for ANSVR, since it worked so much better.

@rlancaste
Copy link
Owner

I had not considered the use case of having ANSVR set up on one computer and a bunch of other machines accessing it in the field. I have always put either astrometry, ANSVR, or cygwin ANSVR on the same machine I was using to run Ekos since it is orders of magnitude faster than ANSVR. But I could see if you are using a bunch of older raspberry pi's or you were short on space, it might be good to have a local astrometry server. I will look into supporting this use case then later today.

@rlancaste
Copy link
Owner

Ladislav to answer your question, cygwin astrometry is supported just like ANSVR and local astrometry on Macs and Linux. As long as cygwin or ANSVR is running on the same windows machine you are using for Ekos, KStars will send it all appropriate command line arguments you would like to send it using the Ekos options. I believe it now supports every command line option that astrometry.net offers (at least the ones that matter).

@rlancaste
Copy link
Owner

I will see if I can get a screenshot for you

@rlancaste
Copy link
Owner

It might take a little bit because while I had a nice virtual machine all set up when I was doing this work last year, I don't think I have that machine anymore. It wouldn't help as much to show you a Mac or Linux screenshot.

@rlancaste
Copy link
Owner

Hmm, let's try this, here are some Mac screenshots where I filled in the settings that I think I was using for Cygwin on Windows. Your paths should be similar, but you should check

Screen Shot 2021-08-20 at 9 10 21 AM

Screen Shot 2021-08-20 at 9 10 21 AM

@rlancaste
Copy link
Owner

Screen Shot 2021-08-20 at 9 10 00 AM

@rlancaste
Copy link
Owner

For the source extraction method, you have two options available to you, you can use internal SEP, or astrometry.net's built in Star extraction. For the command line options, that is what the "Options profile" is for. There are so so many options. I found it best to try to develop profiles of options for specific tasks. If you don't like them though, you can customize the options or you can make your own set.

@laheller
Copy link

laheller commented Aug 20, 2021 •
edited
Loading

@rlancaste
Thanks for the screenshots and explanation, I'll try.

@rlancaste
Copy link
Owner

papaf76, as I said a couple of minutes ago, I will run some tests today/this weekend to verify how well StellarSolver is playing with ANSVR when used in place of astrometry. I can easily fix the problem you found, but my concern right now is whether some features I am relying on from astrometry.net (such as downloading logs) will be supported the same way on an ANSVR server.

@rlancaste
Copy link
Owner

Ladislav, so the "scale and position" options are separated from the rest. They are the settings that get changed all the time, they are in another tab. The rest of the settings can be accessed in the profile editor if you need to. But note that the profile that gets used is the one selected in the other screenshot at the far right. You shouldn't need to change these much, my goal is to figure out what the best settings should be for certain situations and make a profile that the user can select. You can make a profile of the "best" settings as well and save it or send it others as a file. It is exportable.

@rlancaste
Copy link
Owner

papaf76, I think I have fixed the problem you brought up. I believe the hard coded nova.astrometry.net was just an oversight on my part because just a few lines above where I hard coded nova.astrometry.net, it was using the custom astrometry server path. So probably just an accident. I also made a couple of edits to make the tester work with alternate astrometry servers and made the log output display what server is being accessed in its statements. I was able to solve an image with your remote ansvr server that you posted above after making these changes. Note that this change was made in master on GIT, so if you are using a stellarsolver that is packaged with windows KStars or if you are using a stellarsolver that is installed from a package manager, then there will be a delay before these changes show up there. Let me know if there are more issues.

@rlancaste
Copy link
Owner

papaf76, as for the other thing you brought up, that the online solver is not using the scale or position settings. It definitely should be using them since those options are passed by stellarsolver to the online server along with the image and other info. However, I could certainly check that they correctly get passed on, since there are several steps in the process and there could be a bug somewhere.

@papaf76
Copy link
Author

papaf76 commented Aug 20, 2021 via email

@rlancaste
Copy link
Owner

Ok, did you try this with ekos's load and slew feature, or is this with plate solving? If you are plate solving in the align tab, you can select to have it upload the whole image, or do SEP on it and just upload the star list. You can set that with star extraction method.

Also are you saying that Ekos is passing the position and scale on to the solver separately or that the image is uploaded with position and scale in the headers and then astrometry on the server is using what is in the headers?

@papaf76
Copy link
Author

papaf76 commented Aug 20, 2021 via email

@papaf76
Copy link
Author

papaf76 commented Aug 20, 2021 via email

@rlancaste
Copy link
Owner

Load and slew is different. When I first made my updates to integrate StellarSolver, I made Load and Slew use the "use position" and "use scale" checkboxes in Ekos (actually I think that I first made it blind solve every time, but that was short lived). I think someone might have changed it more recently so that load and slew ignores the boxes, but we can check that.

@laheller
Copy link

laheller commented Aug 20, 2021 •
edited
Loading

@rlancaste

I have KSTARS v3.5.4, this version:
https://www.indilib.org/jdownloads/kstars/kstars-3.5.4.exe

Found the settings you had on your screenshots. Here are the settings I used:
kép

kép

kép

kép

kép

As you can see in the first screenshot I am using custom profile called "Lacko". Also on the fourth screenshot there are settings for the "Lacko" profile. Unfortunately I can set only the solving parameters provided by this editor.

Here is the log output from the EKOS Alignment window:

2021-08-20T17:40:20 Solver Failed.
2021-08-20T17:40:20 Base: "C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_13", basefile "externalSextractorSolver_13.fits", basedir "C:/Users/Ladislav Heller/AppData/Local/Temp", suffix "fits"
2021-08-20T17:40:20 Reading input file 1 of 1: "C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_13.fits"...
2021-08-20T17:40:20 Command: C:/cygwin64/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --downsample 3 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -vv --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_13.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_13.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_13.wcs --keep-xylist C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_13.xyls C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_13.fits
2021-08-20T17:40:20 Starting external Astrometry.net solver with the Lacko profile...
2021-08-20T17:40:20 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-20T17:40:20 Configuring external Astrometry.net solver classically: using python and without Sextractor first
2021-08-20T17:40:20 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

From the log it is clear that it uses the "Lacko" profile during solving but it seems to ignore ScaleLow and ScaleHigh parameters.
Also there are a lot of hardcoded parameters, some of them I would completely ignore. But the most important is that there are missing the parameters I really need during solving and I don't know how/where to enter them.

A typical WORKING solver command line in my custom astrometry.net build is following:
solve-field --timestamp -O -l 30 -D /tmp -L 40.0 -H 90.0 -u dw -z 3 -y --no-plots --no-remove-lines --uniformize 0 --fits-image /cygdrive/d/Astro/Sas.fits

I guess it is not possible to call solve-field like the above example, from EKOS, right?
What I really miss in the Alignment options window (in the profile editor) is a text field where we can enter our custom command line parameters for solve-field and when this text field is filled-in, EKOS shall pass them to solve-field and ignore everything else in the profile setting.

BR,

Ladislav

@papaf76
Copy link
Author

papaf76 commented Aug 20, 2021 via email

@rlancaste
Copy link
Owner

I agree, I made it that way, it wasn't my decision to change it. I think it was changed though because the load and slew is using totally a different position and scale than the current ekos position and scale which is shown next to the "Use Position" and "use scale" checkboxes. I think it was causing some confusion on the part of users, which is why it was changed.

@papaf76
Copy link
Author

papaf76 commented Aug 20, 2021 via email

@rlancaste
Copy link
Owner

Yeah, there is always room for improvement. I will see what Jasem thinks about it.

@laheller
Copy link

@rlancaste @papaf76
Sorry for jumping in to your conversation.
@rlancaste can you please check my last comment above?

@rlancaste
Copy link
Owner

Ladislav,

Sorry I missed that comment. I spent hundreds of hours going through all the details of parameters and code in astrometry.net last year, and working with people over the last 5 years who had no end of problems with plate solving. This project was intended to get on top of all the issues and get rid of most of them. I can help you with these details. The options in astrometry.net can get a little crazy. I studied all of them, exposed the ones you need in an interface, and hard coded the ones that shouldn't be turned off or it will fail. There are also some options that I added that make other features work. The custom box was nice, but it caused many many problems. When users entered one too many spaces for instance, or accidentally deleted something that needed to be there. It was bad.

If it failed immediately, please make sure first that all the paths you entered are correct for your application.

You told me that your solver was here:
C:\astrometry.net\bin\solve-field.exe

But you entered this:
C:\cygwin64\bin\solve-field.exe

Please check to make sure that all 3 paths exist and are entered correctly. You can probably not worry too much about the config file, since StellarSolver should make its own and you should use that, but still check all the paths.

To answer your questions about the parameters, I will try to explain here:

Here was the option set you listed:
solve-field --timestamp -O -l 30 -D /tmp -L 40.0 -H 90.0 -u dw -z 3 -y --no-remove-lines --uniformize 0 --fits-image /cygdrive/d/Astro/Sas.fits

The -O and --no-plots options match what was passed exactly.
The -z parameter was passed, it is the same as --downsample
The -L/--scale-low and -H/--scale-high parameters were not passed because you turned off "use scale", however if you check the temporary config file, you will find that it used the minwidth of 40 and maxwidth of 90 parameters in that file since they are settings that are to be used when the scale is not provided.
The -u parameter was not passed because you turned off "use scale". The parameters in the temporary config file are in degrees by default.
The --timestamp parameter is not really needed, but might be nice to add.
The --no-remove-lines and --uniformize 0 options are added to avoid python usage if you decide to use an external star extractor or SEP, you should leave those options out if you want it to use the built in star extraction process.
The -D parameter is not needed here since StellarSolver copies the file into the temp directory first and then all the files end up in there.
The --fits-image parameter is unnecessary since it automatically assumes it is a fits image.
The -l parameter was not passed because it was generated in the temporary config file.

I think that addresses everything that was in your options list. Please let me know if you have more questions.

Thanks,

Rob

@laheller
Copy link

@rlancaste

Just tried the combination of InternalSolver & BuiltInMethodForSolver and it WORKED for both Capture&Solve and Load&Slew !!! So I am 99% happy and thank you for this StellarSolver stuff, it's really a good tool!

However I stil can't believe that it fails for my local custom build of astrometry.net.
One last idea, if you have time, I can upload somewhere my astrometry.net build and you can download and try with that...

BR,

Ladislav

@rlancaste
Copy link
Owner

Ok so then you didn’t decide to try ANSVR?

Yes, for the Cygwin build it should work as well unless there is a configuration issue or I guess if astrometry made some big changes and they changed some parameters on us since last year. Did you just install Cygwin and build astrometry with the latest sources following their instructions? I could try that again and see. Yesterday I only tested ansvr since that is much easier to set up.

@laheller
Copy link

@rlancaste

Did not tried yet with ANSVR, since the internal solver works.

Anyway, if you want to check it with my custom build of Cygwin/astrometry.net, you can download it from:
https://drive.google.com/file/d/1oJ6JveXm0nVNiwse4CUmF7ozaVyGlSX8/view?usp=drivesdk

If you decide to try it, just simple extract the downloaded file (and also the TAR inside) using 7-zip utility to any folder at the root drive.

BR,

Ladislav

@rlancaste
Copy link
Owner

Ok, so I tried your custom astrometry.net build. I unzipped it as instructed, I changed the paths to match the correct programs:
The solver binary was located at:
C:/astrometry.net/bin/solve-field.exe
The wcsinfo program was located at:
C:/astrometry.net/bin/wcsinfo.exe

Then I tested it using my INDIServer simulator as I did yesterday on ANSVR. I set it to use the following options:

Screen Shot 2021-08-22 at 5 10 04 PM

It captured and solved successfully. But then I tried these options:

Screen Shot 2021-08-22 at 5 11 01 PM

and it failed. This can be seen in the following output:

Screen Shot 2021-08-22 at 5 03 36 PM

To investigate why it failed, I went to the cygwin terminal and I tried to plate solve an image using a terminal command and it said the following:

/bin/sh: C:/astrometry.net/bin/image2pnm: /usr/bin/python3: bad interpreter: No such file or directory

Which leads me to think that the reason it failed for you might be because python3 is not properly installed in your cygwin build.

As an interesting aside, a very similar issue is the very reason that I started to embark to create StellarSolver in the first place. A bunch of people were having all kinds of python issues with astrometry.net (mainly on Macs) and it was driving me crazy. So I made StellarSolver to combine SEP source extraction with astrometry.net solving. Then it evolved from there to become the StellarSolver internal solver and the library.

@rlancaste
Copy link
Owner

Let me know if there is something I can do with the python3 issue in your cygwin build and I can try again

@rlancaste
Copy link
Owner

Wait, it just got weirder. I think python3 does exist, but maybe it's a broken link.
If you say which python2 you get: "/usr/bin/python2"
If you say which python3 you get: "which: no python3 in" (then a whole lot of paths it isn't in)
But I see the symbolic link in /usr/bin and in /usr.

@rlancaste
Copy link
Owner

Just tested that theory, it is even weirder still. its a rabbit hole of links. The link "/usr/bin/python3" links to:
/cygdrive/c/astrometry.net/etc/alternatives/python3
Which is another symbolic link, which leads to:
/cygdrive/c/cygwin64/astrometry.net/usr/bin/python3.8
Which doesn't appear to exist in the files you sent.

@rlancaste
Copy link
Owner

Ok so I changed your python3 link by doing a
rm /usr/bin/python3
and replaced it by doing a
ln -s /usr/bin/python3.8.exe /usr/bin/python3

And then I opened KStars again and tried the built in star extraction again, and it solved:

Screen Shot 2021-08-22 at 5 41 53 PM

So yes, I think the main problem with your custom astrometry setup was the broken link to python3.

@laheller
Copy link

@rlancaste

Nice!

I'll check that once I am back from work.
Thanks for investigating!

BR,

Ladislav

@papaf76
Copy link
Author

papaf76 commented Aug 23, 2021

As an interesting aside, a very similar issue is the very reason that I started to embark to create StellarSolver in the first place. A bunch of people were having all kinds of python issues with astrometry.net (mainly on Macs) and it was driving me crazy. So I made StellarSolver to combine SEP source extraction with astrometry.net solving. Then it evolved from there to become the StellarSolver internal solver and the library.

Just out of curiosity, might SEP be the reason why the behaviour in communication with nova changed? I mean specifically the need to pass the image size to the solver. That could explain a thing or two...
Another thing, if you know: with my modifications, ANSVR is now able to properly solve with a recent kstars version. Still, I'm not getting the same FOV dimensions compared to the proper nova server. Do you happen to know how to use stars.wcs to calculate this? Right now I'm multiplying the pixel scale to the image size but as I said, I don't seem to get the same result as nova.

Thanks again, for everything! You've been nothing short of awesome!

@rlancaste
Copy link
Owner

For the first item, you will find that if you use internal SEP or external Sextractor for star extraction then a number of parameters that StellarSolver sends will change. Yes image height and width are required by astrometry to solve an xy list of sources. If you switch to the built in method and just send an image, width and height will not be sent.

For the second item, do you mean the final scale result sent back by nova? Did you downsample or bin the image? That can affect the scale astrometry reports. Do you have an example image?

@laheller
Copy link

@rlancaste

Just askin', why the python(3) is needed for the solving of an image when there is no plotting and the input image is always converted to FITS before even the solver is started? I mean in StellarSolver.

Using the ProcMon tool for Windows and documentation for astrometry.net I figured out that python is completely skipped in case, when the input to solve-field is a FITS format image and the command line has following parameters:

-p (--no-plots)
-9 (--no-remove-lines)
--uniformize 0

So to summarize there are 4 conditions to skip python. If only one is not met, python is started. But we don't need python. We need only center coordinates of the image, FOV, pixel scale, etc but nothing else.
I checked this behaviour many times using the ProcMon tool and it's valid. Got always the same result.

I would suggest to include these cmdline parameters hardcoded into the command line in case of Cygwin, because I think in the 100% of cases the python is not needed for plate solving.

BR,

Ladislav

@rlancaste
Copy link
Owner

One of the main reasons for starting StellarSolver was originally to get rid of python. In fact those exact three parameters are passed to local astrometry and Cygwin and ANSVR most of the time. It is only when the user wants to do the traditional “built in” solver for astrometry.net that astrometry is allowed to solve it how it wants.

@laheller
Copy link

@rlancaste

According to astrometry.net documentation python is used only in two cases:

  1. When input image is NOT the FITS format => converts it using python scripts to FITS
  2. When user wants to plot annotations on the solved input image (other than FITS) => annotations are done by a python script

In our astrophotography scenarios none of the above is the case. We simply use FITS format and do NOT plot.
Next to just start another new python3 process by solve-field, without using its result is simply wasting of resources and it just slows down the overall plate solving.

So as you wrote:

One of the main reasons for starting StellarSolver was originally to get rid of python.

If that's the main reason, then why in some cases python is allowed when it's anyway not needed? :-)

BR,

Ladislav

@rlancaste
Copy link
Owner

To be honest, the main reason I left astrometry.net to its own devices in that case was so that we could have a basis for comparison. The main goal was to use Sextractor or SEP combined with astrometry.net to avoid needing any python or netpbm at all. And when I did my first experiments, that method was giving far superior results to the astrometry internal star extraction and solving. I definitely see it as the preferred method in almost all cases, including using online astrometry.net or remote ANSVR, because then you are uploading a text file instead of an image file, its WAY smaller. The only reasons I left in the ability to solve using an image and astrometry's built in methods was so that we could compare the performance to what astrometry naturally wants to do and also so that people would still be able to use astrometry.net if for some reason SEP or Sextractor didn't work for them and astrometry.net's method worked better. But so far, SEP and sextractor still work much better almost universally.

Actually one of the best abilities of StellarSolver is its ability to plate solve jpeg images very very fast. To compare its performance to using astrometry.net natively, we would need to allow it to plate solve using python to do the conversion which is what it naturally does. If we are converting it to FITS first, that isn't normal astrometry behavior, so it isn't really a good comparison. Now I could see adding an option that allows the user to force all images to be converted to FITS before being sent to astrometry.net, but that is adding complexity.

It is true that many times we are working with FITS images in KStars, but the "load and slew" feature often works with Jpeg images. Those Jpegs would need to be converted before being sent.

@rlancaste
Copy link
Owner

Let me think about this, because we could either convert all images to FITS first before sending them to StellarSolver, or I could add another option to StellarSolver for always converting to FITS first. We could hard code that as turned on in KStars and then we could remove all references to Python from the Align options. That also would allow me to delete the crazy recipe's in the KStars Mac craft build that require the building of netpbm and the inclusion of python. That might be a nice idea. KStars does not need to be sending JPEGS to astrometry if we an avoid it. Good idea!

@rlancaste
Copy link
Owner

Ok I made the python avoidance commands universal and I made an option that would only sent FITS files to astrometry.

The next step will be to add the usage of this feature to Ekos. Then hopefully we can get rid of some unnecessary recipes in craft and remove clutter from KStars. Maybe this will end our astrometry python issues for good. Actually I thought I had done that, but they came back in this thread. Hopefully now. . .

There will be a few steps before it will get into the windows and Mac releases.

@laheller
Copy link

laheller commented Aug 24, 2021 •
edited
Loading

@rlancaste

Just made a short test, how python affects the overall plate solving performance.
I used the free tool called ProcMon, which is able to monitor running processes on Windows.

Configured its filter to allow monitor only solve-field, shell processes and ProcessCreate operation:
image

First I executed a plate-solving (in the Cygwin terminal) of a PNG image, where also python is involved at least by converting PNG to FITS via netpbm. I also let solve-field to plot annotations.
Command line in Cygwin was:

solve-field -O -l 30 -D ./Solve -L 40 -H 90 -u dw -z 3 ./M31.png

In ProcMon we can see, that solve-field creates a lot of subprocesses including among others python:
image

As we can see in the monitor, python was 3 times called.

In the second case I used again a PNG image, but this time with following command line:

solve-field -O -l 30 -D ./Solve -L 40 -H 90 -u dw -z 3 -9 --uniformize 0 -p ./M31.png

From the output we can clearly see that python was called only once to do PNG->FITS conversion:
image

Finally in the third case I used FITS input image with additional command line parameters to completely skip python:

solve-field -O -l 30 -D ./Solve -L 40 -H 90 -u dw -z 3 -9 --uniformize 0 -p -y --fits-image ./M31.fits

Please note that the --fits-image parameter is also required to reach our goal! When this parameter is not specified for a FITS input file, python is still called! Just checked your already pushed commit and saw that this parameter is mising ;-)

As we expected, the python was not called by solver and the overall plate-solving was much faster:
image

Performance and the fact that we don't (really) need python are the only reason why I suggest to improve StellarSolve by addig the above command line parameters!

BR,

Ladislav

@laheller
Copy link

@rlancaste

In the meantime I tried again with my local custom built Cygwin/astrometry.net using this combination:
kép

Means:
Solving method = Local Astrometry
Source Extraction Method = Internal SEP

Using the above combination plate-solving works well, here is the output:

2021-08-24T21:06:14 Solver RA (296.14873) DEC (9.83861) Orientation (-178.26570) Pixel Scale (124.45108) Parity (neg)
2021-08-24T21:06:14 Solver completed after 1.43 seconds.
2021-08-24T21:06:14 Shutting down other child solvers
2021-08-24T21:06:14 Successfully solved with child solver: 3
2021-08-24T21:06:14 Finished Loading WCS...
2021-08-24T21:06:14 Loading WCS from file...
2021-08-24T21:06:14 Field parity: neg

2021-08-24T21:06:14 Field rotation angle: up is -178.266 degrees E of N
2021-08-24T21:06:14 Pixel Scale: 124.451"
2021-08-24T21:06:14 Field size: 63.31 x 44.84 arcminutes
2021-08-24T21:06:14 Field center: (RA H:M:S, Dec D:M:S) = (19:44:35.694, +09:50:18.994).
2021-08-24T21:06:14 Field center: (RA,Dec) = (296.149, 9.83861) deg.
2021-08-24T21:06:14 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 Command: C:/astrometry.net/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --resort --objs 50 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -L 20.625 -H 52.5 -u degwidth --width 2048 --height 1364 --x-column X_IMAGE --y-column Y_IMAGE --sort-column MAG_AUTO --sort-ascending --no-remove-lines --uniformize 0 --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.wcs C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.xyls
2021-08-24T21:06:13 Starting external Astrometry.net solver with the 3-LargeScaleSolving profile...
2021-08-24T21:06:13 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 Command: C:/astrometry.net/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --resort --objs 50 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -L 105.625 -H 180 -u degwidth --width 2048 --height 1364 --x-column X_IMAGE --y-column Y_IMAGE --sort-column MAG_AUTO --sort-ascending --no-remove-lines --uniformize 0 --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.wcs C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.xyls
2021-08-24T21:06:13 Starting external Astrometry.net solver with the 3-LargeScaleSolving profile...
2021-08-24T21:06:13 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 Command: C:/astrometry.net/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --resort --objs 50 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -L 52.5 -H 105.625 -u degwidth --width 2048 --height 1364 --x-column X_IMAGE --y-column Y_IMAGE --sort-column MAG_AUTO --sort-ascending --no-remove-lines --uniformize 0 --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.wcs C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.xyls
2021-08-24T21:06:13 Command: C:/astrometry.net/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --resort --objs 50 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -L 10 -H 20.625 -u degwidth --width 2048 --height 1364 --x-column X_IMAGE --y-column Y_IMAGE --sort-column MAG_AUTO --sort-ascending --no-remove-lines --uniformize 0 --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.wcs C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.xyls
2021-08-24T21:06:13 Starting external Astrometry.net solver with the 3-LargeScaleSolving profile...
2021-08-24T21:06:13 Starting external Astrometry.net solver with the 3-LargeScaleSolving profile...
2021-08-24T21:06:13 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 Configuring external Astrometry.net solver using an xylist input
2021-08-24T21:06:13 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 Configuring external Astrometry.net solver using an xylist input
2021-08-24T21:06:13 The external ANSVR solver on windows does not handle the inparallel option well, disabling it for this run.
2021-08-24T21:06:13 Configuring external Astrometry.net solver using an xylist input
2021-08-24T21:06:13 Configuring external Astrometry.net solver using an xylist input
2021-08-24T21:06:13 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:13 The external ANSVR solver on windows does not handle the inparallel option well, disabling it for this run.
2021-08-24T21:06:13 The external ANSVR solver on windows does not handle the inparallel option well, disabling it for this run.
2021-08-24T21:06:13 The external ANSVR solver on windows does not handle the inparallel option well, disabling it for this run.
2021-08-24T21:06:12 Solver # 4, Low 105.625, High 180 arcminwidth
2021-08-24T21:06:12 Solver # 3, Low 52.5, High 105.625 arcminwidth
2021-08-24T21:06:12 Solver # 2, Low 20.625, High 52.5 arcminwidth
2021-08-24T21:06:12 Solver # 1, Low 10, High 20.625 arcminwidth
2021-08-24T21:06:12 Starting 4 threads to solve on multiple scales
2021-08-24T21:06:12 Stars Found after Filtering: 50
2021-08-24T21:06:12 Keeping just the 50 brightest stars
2021-08-24T21:06:12 Removing the stars with a/b ratios greater than 1.5
2021-08-24T21:06:12 Stars Found before Filtering: 497
2021-08-24T21:06:12 Starting Internal StellarSolver Sextractor with the 3-LargeScaleSolving profile . . .
2021-08-24T21:06:12 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T21:06:12 There should be enough RAM to load the indexes in parallel.
2021-08-24T21:06:12 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 0.634597 GB, Installed RAM: 15.9458 GB, Free RAM: 10.352 GB

.

Tried again also the combination:
Solving method = Local Astrometry
Source Extraction Method = BuiltIn method for solver
It again failed like many times before, but this time I took the command line from log, as you did and put it to the Cygwin terminal and started and guess what happend: It works from Cygwin! Can't believe, but the same command line works from Cygwin terminal!!!

I will do further investingating, but now it is clear that my local custom Cywin/astrometry.net is fully working, but somehow when started from StellarSolver, it fails for the above combination.

BR,

Ladislav

@rlancaste
Copy link
Owner

if you think the --fits-image parameter is really needed when a user wants to use the "built in" star extraction in order to avoid python. I am not sure why it would need that when its being given a FITS file already? maybe it isn't sure it is a FITS file till it tries?

@rlancaste
Copy link
Owner

For your experiments, I don't think you want the "large-scale solving" profile, which I see selected in your screenshot. I designed that one to work best for DSLR lens scale images. You probably want "Parallel small scale"

@rlancaste
Copy link
Owner

When I checked your cygwin build it had a broken link as I said, when I fixed that it worked fine.

@laheller
Copy link

if you think the --fits-image parameter is really needed when a user wants to use the "built in" star extraction in order to avoid python. I am not sure why it would need that when its being given a FITS file already? maybe it isn't sure it is a FITS file till it tries?

@rlancaste
Just try it with and without this parameter, while watching subprocesses in ProcMon tool, if you don't trust me :)
In my environment python was only skipped when input file was a FITS and following parameters were passed to solve-field:

  • -p (--no-plots)
  • -9 ( --no-remove-lines)
  • --uniformize 0
  • --fits-image

Any other combination of input format and command line parameters involved python.

BR,

Ladislav

@laheller
Copy link

When I checked your cygwin build it had a broken link as I said, when I fixed that it worked fine.

@rlancaste
Yes I know, but in my cygwin build that symbolic link is correct! Anyway, plate-solve failed.

BR,

Ladislav

@laheller
Copy link

For your experiments, I don't think you want the "large-scale solving" profile, which I see selected in your screenshot. I designed that one to work best for DSLR lens scale images. You probably want "Parallel small scale"

@rlancaste
Tried all 4 default profiles and the builtin extractor but always the same error:

2021-08-24T22:57:33 Solver Failed.
2021-08-24T22:57:33 Child solver: 2 did not solve or was aborted
2021-08-24T22:57:33 Child solver: 4 did not solve or was aborted
2021-08-24T22:57:33 Child solver: 1 did not solve or was aborted
2021-08-24T22:57:33 Child solver: 3 did not solve or was aborted
2021-08-24T22:57:32 Command: C:/astrometry.net/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --resort --objs 50 --downsample 2 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -L 0.71875 -H 2.575 -u degwidth --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.wcs --keep-xylist C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.xyls C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_1.fits
2021-08-24T22:57:32 Starting external Astrometry.net solver with the 4-SmallScaleSolving profile...
2021-08-24T22:57:32 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T22:57:32 Command: C:/astrometry.net/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --resort --objs 50 --downsample 2 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -L 2.575 -H 5.66875 -u degwidth --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.wcs --keep-xylist C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.xyls C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_2.fits
2021-08-24T22:57:32 Starting external Astrometry.net solver with the 4-SmallScaleSolving profile...
2021-08-24T22:57:32 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T22:57:32 Command: C:/astrometry.net/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --resort --objs 50 --downsample 2 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -L 0.1 -H 0.71875 -u degwidth --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.wcs --keep-xylist C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.xyls C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_0.fits
2021-08-24T22:57:32 Command: C:/astrometry.net/bin/solve-field.exe -O --no-plots --no-verify --crpix-center --match none --corr none --new-fits none --rdls none --resort --objs 50 --downsample 2 --odds-to-solve 1e+09 --odds-to-tune-up 1e+06 -L 5.66875 -H 10 -u degwidth --backend-config C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.cfg --cancel C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.cancel -W C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.wcs --keep-xylist C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.xyls C:/Users/Ladislav Heller/AppData/Local/Temp/externalSextractorSolver_1_3.fits
2021-08-24T22:57:32 Starting external Astrometry.net solver with the 4-SmallScaleSolving profile...
2021-08-24T22:57:32 Starting external Astrometry.net solver with the 4-SmallScaleSolving profile...
2021-08-24T22:57:32 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T22:57:32 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T22:57:32 Configuring external Astrometry.net solver classically: using python and without Sextractor first
2021-08-24T22:57:32 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T22:57:32 The external ANSVR solver on windows does not handle the inparallel option well, disabling it for this run.
2021-08-24T22:57:32 Configuring external Astrometry.net solver classically: using python and without Sextractor first
2021-08-24T22:57:32 Configuring external Astrometry.net solver classically: using python and without Sextractor first
2021-08-24T22:57:32 Configuring external Astrometry.net solver classically: using python and without Sextractor first
2021-08-24T22:57:32 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T22:57:32 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T22:57:32 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2021-08-24T22:57:32 The external ANSVR solver on windows does not handle the inparallel option well, disabling it for this run.
2021-08-24T22:57:32 The external ANSVR solver on windows does not handle the inparallel option well, disabling it for this run.
2021-08-24T22:57:32 The external ANSVR solver on windows does not handle the inparallel option well, disabling it for this run.
2021-08-24T22:57:32 Solver # 4, Low 5.66875, High 10 arcminwidth
2021-08-24T22:57:32 Solver # 3, Low 2.575, High 5.66875 arcminwidth
2021-08-24T22:57:32 Solver # 2, Low 0.71875, High 2.575 arcminwidth
2021-08-24T22:57:32 Solver # 1, Low 0.1, High 0.71875 arcminwidth
2021-08-24T22:57:32 Starting 4 threads to solve on multiple scales
2021-08-24T22:57:32 There should be enough RAM to load the indexes in parallel.
2021-08-24T22:57:32 Evaluating Installed RAM for inParallel Option.  Total Size of Index files: 0.634597 GB, Installed RAM: 15.9458 GB, Free RAM: 9.86173 GB
2021-08-24T22:57:32 Automatically downsampling the image by 2

BR,

Ladislav

@rlancaste
Copy link
Owner

Ok I added the --fits-image parameter

@rlancaste
Copy link
Owner

Ok I think I should close this issue here now since we pretty much resolved both issues in the thread.

@rlancaste
Copy link
Owner

The next release of KStars should get these changes in it.

@laheller
Copy link

Ok I think I should close this issue here now since we pretty much resolved both issues in the thread.

OK for me.

The next release of KStars should get these changes in it.

Great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@laheller @rlancaste @papaf76