What is the necessary Polar format ?

[rgleason]

I have been able to compile the plugin now and have tried loading Polar files, but am uncertain whether the polar file format is correct. Can you advise? I have tried *.pol files and *.csv files from Weather_Routing_pi. The pol files use a Tab delimiter while the csv files used ";" semicolon delimiter.
The csv files had the tws/twd in lower case I believe.

Ptulp has modified the Polar_pi plugin to read both file types which makes life a bit easier. I wonder if his code would work with tackandlay?

[cowelld]

Send a copy of the file and I'll see if I can make some adjustments. There
isn't a standardized format for this sort of thing and I had to choose a
couple that I ran across.

----- Original Message -----
From: "Rick Gleason" notifications@github.com
To: "cowelld/tackandlay_pi" tackandlay_pi@noreply.github.com
Sent: Friday, February 27, 2015 9:30 AM
Subject: [tackandlay_pi] What is the necessary Polar format ? (#6)

I have been able to compile the plugin now and have tried loading Polar
files, but am uncertain whether the polar file format is correct. Can you
advise? I have tried *.pol files and *.csv files from Weather_Routing_pi.
The pol files use a Tab delimiter while the csv files used ";" semicolon
delimiter.
The csv files had the tws/twd in lower case I believe.

---

Reply to this email directly or view it on GitHub:
#6

[rgleason]

Dave, Thanks very much! I am going to try to attach several here (may have to change the suffix to png, so you will have to remove it.)

[cowelld]

Can you just send them as attachments rather than an pseudo image at the
GitHub?

[rgleason]

It accepts only images, nothing else. Oh I see the website changes the file name somehow when I try to "Save As". I can send to your email address? Is there a place where would I find that?

[rgleason]

Dave
Have attached them to this forum thread (Take the doc off please)
http://www.cruisersforum.com/forums/f134/tack-and-laylines-plugin-beta-138487.html#post1764527

[cowelld]

I looked at those POL files and see some of the issues. Generally I've
restricted my POL files to having a five degree True Wind Angle increment
and a five knot wind speed increment. Those numbers will be adjusted for
POL's with different values AS LONG AS THEY ARE CONSTANT THROUGHOUT THE
FILE. In the Boat-Polar file the wind speed increments go in 2,3,4,5 knots
groups. with a 3 degree increment. Ditto for Alexandra. The 60ftmono has 1
knot and 2 degrees but that's excessive. I didn't set the array up that way.
The A-35 and Class 40 go from 2 to 60 knots by 2 and so that's also
excessive although the angles are at 5 degrees but start at 5 degrees off
the wind. Can they do that?
Generally I believe these are computer generated VPP files and so have
all that high resolution but questionable practical application. My POL
files are generated from actual sailing tracks with actual conditions and so
have a great deal of input variation. I've dropped to the five on five
"bins" to allow those variations to be averaged or maxed (users choice) in
the POL generation program.
Someone sent me the attached POL file for a Farr 40 which I also used as a
model of input. You'll see that the wind speeds and angles are not on the
five on five but are constant increments. Some of your files loaded but
obviously weren't interpreted correctly. How are you coming by these files?

----- Original Message -----
From: "Rick Gleason" notifications@github.com
To: "cowelld/tackandlay_pi" tackandlay_pi@noreply.github.com
Cc: "Dave Cowell" david.cowell@ymail.com
Sent: Tuesday, March 03, 2015 6:45 PM
Subject: Re: [tackandlay_pi] What is the necessary Polar format ? (#6)

Dave
Have attached them to this forum thread (Take the doc off please)
http://www.cruisersforum.com/forums/f134/tack-and-laylines-plugin-beta-138487.html#post1764527

---

Reply to this email directly or view it on GitHub:
#6 (comment)

[rgleason]

Dave,
I understand the increment problem, and having excessively small increments when the actual data ranges so much, so your practical suggestions make a great deal of sense. Merging data by averaging to the constant angles makes sense. I had not actually noticed the excessively small increments of some of these files.

I was looking at the delimiters [ ";" colon] and [tab] and the various ways they seem to have the start of the first row TWA or TWA/TWS or twa or twa/tws which may be one of the other problems.

You ask where I get these files. They come from various sources:
boat_alexandra weather_routing_pi
boat_polar weather_routing_pi
A-35 I believe this is from Sailgrib (snarfed from my cell phone)
Class 40 I believe this is from Sailgrib (snarfed from my cell phone)
I am not sure how these bottom two were generated.

I have just been collecting polar files for a long time.
I have a set of excel and polar files from "Set-Sail" which are early MaxSea files.
I also a directory full of Sailgrib polars which is a cell phone grib and routing app.
I also have polars for various specific boats such as mine and hartmut's in different formats.some hand translated so they work in different programs.

For users and testing it would be very helpful to know what the requirements are for polar files and what tackandlay does with them if possible. Basically the parameters that they must meet, and how they are then interpreted.

Perhaps we should be trying to standardize this between Polar_pi, Weather_routing_pi and TackandLay_pi to eliminate problems interfacing and to reduce program work? I know Sean has made some effort to accept different types of polars and to provide prompts for what the problem might be (in weather_routing_pi) and Peter [ptulp] has also done some work on this for Polar_pi.

If it would be some help, I have uploaded some of the polar files to Opencpn Beta File Thingie
https://tgp-architects.com/files use Username:: rguser Password: rgpass
Then look under the Polar File directory there is one subdirectory with lots of polar files called
Perf_polar Files those are from the cell phone program SailGrib.

I can upload the two old Set-Sail directories of Excel and pol files as well, if that will help.
That won't get you all the polars that I have, but it covers a lot of them.

[rgleason]

Sorry I was wrong the Perf_Excel and Perf_Polar directories are from Setsail [early MaxSea polars].
I thought I had downloaded the cell app SailGrib polars, but many of those are derived from my sources anyway. I've uploadeed the Perf_Excel now.

[cowelld]

Wow! I see how it's a real mélange. There was one authoritative source that quoted the golden rule of Polars. It's pretty broad but the whole issue is in parsing the text lines and the rule makes it somewhat easier. I don't follow numbers 7-9 closely though and it didn't look like any of the other files did either. Some of them plotted but not correctly. I'll look to see if there's an easy fix else plan a major change. If you have a NMEA track file (VDR file) you can use STE_pi to generate the STE track file which will input to Polar_pi. I'm trying to get that program to work correctly but have issues with the plug-in interface. (The program blows up at odd times.) Inputting the STE track is kind of fun as you get to watch the Polar_pi load the graph. You also get to use either an average or maximum speed or if available STW or SOG for any wind speed/angle combination. As you can see in the attached polars each line has the first digit as True Wind Speed, after that comes pairs of data using blank separators (can be commas) with the first digit being the TWA and second SOG (or STW). I don't recall where this file format came from. Personally I believe STW is a better choice but SOG is often more accurate from a GPS. I mostly sail on the San Joaquin river in California and the tides are a knot or two from all directions with the river course. That throws a lot of corrections into the SOG and my knot log doesn't seem to be linear. The pdf file has a lot of info I've relied on and is a good insight into the mechanics. Still, unless you have a new boat with racing sails it's more realistic than a VPP product. What kind of boat and where are you located. Let's use the personal emails for some of this traffic to reduce clutter on the hub. I'll look at the other files used by Weather_routing_pi and see if there's an easy solution for that. Hasta, Dave Rules for the data in polar files: 1) One windspeed curve per line. 2) The curves must be listed in increasing order of windspeed. 3) There must be a least 3 points per curve. 4) There must be at least one non-zero windspeed curve. 5) In each line, the TWA's must be increasing down the list. 6) The minimum TWA is zero, and the maximum is 180. 7) The 2nd TWA in a line is the best VMG upwind angle for that windspeed. The 2nd to last TWA is the best VMG downwind angle for that windspeed 8) The first TWA should be less than any 2nd TWA in any curve. Ideally, zero. 9) The last TWA should be greater than any 2nd-to-last TWA in any curve. Ideally, 180 5 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180 10 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 4.70 110 115 4.70 120 125 130 4.10 135 4.30 140 145 5.45 150 5.00 155 4.50 160 4.97 165 4.10 170 4.70 175 4.93 180 15 25 30 3.90 35 3.80 40 3.80 45 3.95 50 3.80 55 60 65 70 75 2.70 80 4.90 85 5.15 90 5.33 95 5.25 100 5.40 105 5.53 110 5.57 115 5.42 120 5.40 125 5.50 130 5.73 135 5.61 140 5.22 145 5.55 150 5.66 155 6.28 160 5.77 165 5.84 170 5.63 175 5.42 180 4.20 20 25 30 4.40 35 5.45 40 4.60 45 6.17 50 5.60 55 5.25 60 4.43 65 4.82 70 3.60 75 5.12 80 5.40 85 5.97 90 5.60 95 5.88 100 5.98 105 5.31 110 5.22 115 5.11 120 5.87 125 6.40 130 5.72 135 6.53 140 6.13 145 5.73 150 6.02 155 5.75 160 5.80 165 5.74 170 6.08 175 5.55 180 5.67 25 25 4.40 30 5.69 35 5.53 40 5.92 45 6.04 50 6.23 55 5.94 60 5.50 65 5.11 70 3.92 75 4.44 80 4.81 85 5.71 90 5.30 95 5.96 100 5.57 105 5.61 110 5.49 115 5.70 120 5.39 125 6.10 130 6.67 135 5.67 140 6.45 145 6.15 150 6.25 155 5.86 160 5.45 165 5.84 170 5.78 175 5.59 180 5.77 30 25 4.52 30 5.24 35 5.67 40 6.17 45 6.33 50 6.51 55 6.29 60 6.15 65 6.63 70 3.60 75 6.00 80 85 6.05 90 6.60 95 6.65 100 105 110 6.20 115 120 6.30 125 130 135 5.85 140 145 5.50 150 155 160 5.70 165 6.15 170 5.73 175 5.95 180 5.30 35 25 30 6.10 35 6.15 40 5.73 45 6.50 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 5.90 170 4.80 175 5.00 180 40 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180 45 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180 50 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180 5 25 3.10 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180 10 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 4.70 110 115 4.70 120 125 130 4.10 135 4.30 140 145 5.60 150 5.40 155 4.50 160 5.50 165 4.10 170 4.70 175 5.10 180 15 25 30 3.90 35 3.80 40 3.80 45 4.00 50 3.80 55 60 65 70 75 2.70 80 4.90 85 5.50 90 5.60 95 5.60 100 5.80 105 5.90 110 5.80 115 5.80 120 6.10 125 6.10 130 6.20 135 6.00 140 6.10 145 6.20 150 6.60 155 6.50 160 6.40 165 6.10 170 6.30 175 5.90 180 4.20 20 25 30 5.80 35 7.00 40 6.80 45 7.30 50 7.50 55 7.40 60 7.40 65 7.50 70 4.20 75 6.60 80 6.40 85 6.90 90 6.70 95 6.70 100 6.60 105 6.50 110 6.40 115 6.60 120 6.30 125 6.50 130 6.50 135 6.80 140 6.80 145 6.90 150 6.80 155 6.10 160 6.20 165 6.60 170 6.50 175 6.10 180 6.30 25 25 5.90 30 7.00 35 7.30 40 7.50 45 7.50 50 7.50 55 7.50 60 7.50 65 7.50 70 7.30 75 6.90 80 7.00 85 7.10 90 7.20 95 7.00 100 7.10 105 7.20 110 7.00 115 6.70 120 6.90 125 6.70 130 6.90 135 7.10 140 6.80 145 6.70 150 6.40 155 6.30 160 5.70 165 6.40 170 6.10 175 6.50 180 6.60 30 25 5.90 30 6.60 35 7.30 40 7.50 45 7.50 50 7.50 55 7.50 60 6.30 65 7.50 70 3.60 75 6.00 80 85 6.10 90 6.60 95 6.90 100 105 110 6.20 115 120 6.30 125 130 135 6.80 140 145 5.50 150 155 160 5.70 165 6.30 170 6.10 175 6.20 180 6.20 35 25 30 6.10 35 6.60 40 6.60 45 6.90 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 5.90 170 4.80 175 5.00 180 40 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180 45 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180 50 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180

[rgleason]

Dave, Thank you.
I will send you my email via forum PM.
This looks great. I found the Polars from Sailplan and uploaded them to Opencpn Beta File thingie these are labeled "Polars" under the "Polar_Files"
I just finish having a colonoscopy and need to get my email server and emails working first...

[rgleason]

Maybe we send any polar files through a little program first that cleans them up for use in Polar_pi, TackandLay_pi and Weather_Routing_pi and save the file out as our standard version of POL? or CSV? or whatever. Then would they get saved as a different file that can be reused, or does the program just run at the startup every time?

[cowelld]

I don't want to create a Tool program. It's a lot of extra bother and
something else to manage. I'd rather just make a change to the Polar_pi Load
function. The issue then is whether the user knows the type of file he has
and can select the proper parser via a checkbox else I add a number of
parser options to discover the file construction. A third option would be to
just have the user manually load his data from the file by hand into the
grid then save it out. I'll try to add some more auto-parsers based on the
file types you've given me. In your collection, do you have them sorted by
type or source? I think I'll add a note in the Load function about the need
for the constant Wind Speed and TWA bin sizes.
As Lincoln the programmer said you can parse some of the files all of the
time.....

Hasta,
Dave

----- Original Message -----
From: "Rick Gleason" notifications@github.com
To: "cowelld/tackandlay_pi" tackandlay_pi@noreply.github.com
Cc: "Dave Cowell" david.cowell@ymail.com
Sent: Friday, March 06, 2015 2:08 PM
Subject: Re: [tackandlay_pi] What is the necessary Polar format ? (#6)

Maybe we send any polar files through a little program first that cleans
them up for use in Polar_pi, TackandLay_pi and Weather_Routing_pi and
save the file out as our standard version of POL? or CSV? or whatever.
Then would they get saved as a different file that can be reused, or does
the program just run at the startup every time?

---

Reply to this email directly or view it on GitHub:
#6 (comment)

[rgleason]

Understood about a tool program.
Insofar as the header and delimiters - There are maybe 4 types.
Regarding angle and wind speed increments - you suggestion sounded good.
At the end of the weather_routing_pi Wiki on opencpn.org Opencpn Manual there is a
writeup about various types of polars that I did several years ago to help Sean get the polar ingestion better. I can't link because the site is down right now.. But it might be worthwhile to take a look there.

BASIC TYPES (related to headers and delimiters)
weather_routing_pi
qtvlm
SetSea- Maxsea
Other

[rgleason]

Dave, There is some slightly concurrent work going on in several plugins
ptulp/polar_pi - 5 days ago
Update angle display in 5, 10 or 15 degree steps. I believe he removed 8, 24 and 20 degrees
ptulp/polar_pi@48c76d9
cowelld/tackandlay - now

and Earlier in Sean's plugin
weather_routing_pi
https://github.com/seandepagnier/weather_routing_pi
remove bad polars, bundle data, and correct some polars

I it would be nice to have these plugins working in concert.
I am going to post a new thread in the forum asking for input from the authors.

[rgleason]

http://www.cruisersforum.com/forums/showthread.php?p=1768749#post1768749

[cowelld]

The first polar_pi is a fork of Konnibo's original version which was done
years ago. It's not related to mine.
The second action by Sean was to modify his attached data files to be more
in line with the status-quo. See my other email.

----- Original Message -----
From: "Rick Gleason" notifications@github.com
To: "cowelld/tackandlay_pi" tackandlay_pi@noreply.github.com
Cc: "Dave Cowell" david.cowell@ymail.com
Sent: Sunday, March 08, 2015 5:56 AM
Subject: Re: [tackandlay_pi] What is the necessary Polar format ? (#6)

Dave, There is some slightly concurrent work going on in several plugins
ptulp/polar_pi - 5 days ago
Update angle display in 5, 10 or 15 degree steps. I believe he removed 8,
24 and 20 degrees
ptulp/polar_pi@48c76d9
cowelld/tackandlay - now

and Earlier in Sean's plugin
weather_routing_pi
https://github.com/seandepagnier/weather_routing_pi
remove bad polars, bundle data, and correct some polars

I it would be nice to have these plugins working in concert.
I am going to post a new thread in the forum asking for input from the
authors.

---

Reply to this email directly or view it on GitHub:
#6 (comment)