Where to get load_tide. ini and ocean_tide. ini files

[waterlovers]

Hi, may I verify where the load_tide.ini and ocean_tide.ini files are located? Visiting the bitbucket URL mentioned in a previous (now closed) issue mentions that the directory doesn't exist.

[kvos]

it looks like the project has been migrated to Github, https://github.com/CNES/aviso-fes, can you test downloading it from there and let me know if it works, thanks

[waterlovers]

Ah, it does work. As of now I'm a bit confused as to why the following error appears when I try installing the fes package in the anaconda command line (this error shows up after I key in "conda install -c fbriol fes" in my activated environment).

Does my version of Python have something to do with that? (Running Python 3.7 in anaconda, but my computer uses 3.10)

[kvos]

conda install pyfes -c fbriol this should be the command, it looks like they also changed the package name from fes to pyfes

[waterlovers]

Running the "conda install pyfes -c fbriol" command shows:

When attempting to resolve conflicts, the following error occurs and the (rather numerous!) conflicts seem to remain unresolved:

Leaving these unresolved conflicts appears to mean that pyfes cannot be installed and imported as desired.

I have tried to reinstall anaconda, started over with a new environment replacing the old one, but nothing works so far.

[seamuslo]

Hello @kvos , on attempting to get the FES up and running, I have run into the same issues as @waterlovers . After running through conflicts, I get the following:

And then the installation does not appear successful.

Do you have any thoughts on a path forward?

[kvos]

hi @seamuslo and @waterlovers ,

So it looks like there is an issue with the conda environment as they have updated the pyfes package. From your screenshots it seems that the conflict is with the numpy library: pyfes may require a numpy version that is different that the one that is already installed in coastsat. I don't think this should be a big issue as numpy is such a popular library and usually easy to update. You will need to check that numpy version you have installed (run conda list) and check what numpy version the pyfes package requires.

[seamuslo]

Hi @kvos thanks for the suggestion with respect to numpy. I followed your instructions and checked the version of numpy in coastsat

I looked through the pyfes documentation here (https://github.com/geopython/pyfes), but was unable to find where they specify the specific version of numpy that pyfes requires. I'm still relatively new to python. Is there a common place where this information usually lives? Otherwise I could reach out to the developers.

As a first pass, I just updated numpy to the latest version:

And then tried the installation again. This is the first part of the error message that I get. Now in addition to the numpy portion of the message, there are other "unsatisfableError"(s).

Do you have any thoughts on next steps with respect to numpy or these other errors? Additionally, there is significantly more information output about conflicts after the errors in the above screenshot. I've attached the full terminal output here.
full error log.txt

Any thoughts you might have would be greatly appreciated!

[waterlovers]

Hello @kvos, I've tried updating numpy to its latest-available version as well, and have run into the same issues as @seamuslo.

[kvos]

one alternative is to install pyfes in a separate environment (so there are no conflicts with already installed packages in coastsat), then generate the tide predictions, save them into a .csv, and then activate the coastsat environment and load them into the workflow. You want to have a go at that? let me know if any issues

[kvos]

you can use this page to see the commands to manage an Anaconda environment https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html

[seamuslo]

Hi @kvos and @waterlovers , I was able to get this suggestion (using a separate environment) to work. I used the attached script which pulls from the original code with some alterations (attached as txt file). Its a little janky because I did it quickly, but I think it should be understandable, let me know if its not.
tide_generation_script_janky.txt

However, once I went through the process of importing this data into CoastSat, I ran into some things I didn't fully understand.

The final line of section 4 of CoastSat.slope (see below), uses both date_sat and tide_sat as inputs into the function find_tide_peak
settings_slope['freqs_max'] = SDS_slope.find_tide_peak(dates_sat,tide_sat,settings_slope)

However, the tide_sat that is output from the attached script that I ran in the other environment (attached as CSV) is longer (158 elements) than the date_sat produced earlier in the script (105 elements) in section 3. Having these two arrays be different lengths produces errors in the find_tide_peak call in section 4.

tide_sat.csv

However, if I utilize the date_sat array that I produced in the script from the other environment (see below and attached as CSV), it is the same size, and allows me to completely run section 4 of the code and get the two plots.

date_sat_ts.csv

dates_sat = pd.read_csv(os.path.join(os.getcwd(),'data','Pekalongan_longer_time','date_sat_ts.csv'))
...
# plot time-step distribution #t = np.array([_.timestamp() for _ in dates_sat]).astype('float64') t = dates_sat t = pd.DataFrame.to_numpy(t) print(t) print(type(t)) print(t.size) print(t.ndim)
...
#settings_slope['freqs_max'] = SDS_slope.find_tide_peak(dates_sat,tide_sat,settings_slope) #attempting to feed t in here instead, as it looks like the same thing is done to date_sat in the function as it is in the script settings_slope['freqs_max'] = SDS_slope.find_tide_peak(t,tide_sat,settings_slope)

@kvos would you be able to explain a little more about what is going on in these lines where date_sat is produced in the script originally? Any information you could provide would be helpful:
# clip the dates between 1999 and 2020 as we need at least 2 Landsat satellites idx_dates = [np.logical_and(_>settings_slope['date_range'][0],_<settings_slope['date_range'][1]) for _ in output['dates']] dates_sat = [output['dates'][_] for _ in np.where(idx_dates)[0]]

[waterlovers]

Hi @kvos, this isn't exactly related to the FES2014 issues we have been discussing here, but I was wondering if we could use coastsat.slope extension for temporary coastal features like spits or barrier islands?
Would any shorelines delineated by Coastsat throw any errors if more than one shoreline was found along any drawn transect at any time? Would it influence the accuracy of the tidal corrections that the FES2014 model makes?

[kvos]

hey @waterlovers , that's a good question, it becomes tricky when there are several intersections, you can set it up to always take the most seaward intersection (there is a setting parameter for this). The problem is that as the feature moves, the transect may not be cross-shore any more so the tidal correction will be in the wrong direction. I suggest that you have a look at the CoastSat.islands repository (https://github.com/mcuttler/CoastSat.islands) as it is more suited to transient features, it looks at the shape of the sand pixels rather than shorelines

[kvos]

hi @seamuslo , if you loaded the same file in both environments they should be the same :)
It could be that you went from 158 to 105 after calling some of the QA functions like the ones below:

# remove duplicates (can happen that images overlap and there are 2 shorelines for the same date)
output = SDS_slope.remove_duplicates(output)
# remove shorelines from images with poor georeferencing (RMSE > 10 m)
output = SDS_slope.remove_inaccurate_georef(output, 10)

[waterlovers]

Hi @kvos @seamuslo Apologies for taking this long to get around to testing the solution of installing environment, but I'm still confused about avoiding conflicts between pyfes and existing coastsat packages by installing pyfes in a separate environment.

When creating the new environment, does it have to be in the same directory as the one where the coastsat folder was downloaded (where an "environment.yml" file already exists)? I'm still not sure how conda enviromments work, and I'm not sure if using the command "conda env create -f environment.yml -n pyfes_envt" will irreversibly alter the environment.yml file initially made for the coastsat packages.

Is it possible to alter the command so that the name of the environment made isn't the same as the one for the coastsat packages? (I tried typing "environment2.yml" instead of "environment.yml", but doing this returned an error saying "SpecNotFound: Invalid name, try the format: user/package")

In addition, once the environment is set up and pyfes package is successfully installed, how would I activate that, then run the code in the jupyter notebook to be able to generate the tide predictions using FES2014?

[waterlovers]

As for the images used for tidal correction with the FES2014 model, the description of the extension mentions that they have to be Landsat imagery from 1999-2020. Would this mean that the range of imagery of shorelines that need to be used for tidal correction can only be restricted to 1999-2020, even for shoreline changes over 2021, 2022 etc.?

Or can this time range of imagery used be extended across the years to contain the time range across which a CSV of tides/a value(s) beach slope is needed?

[kvos]

you can use any images after 1999. You can read on how to manage conda envrinments here https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html, I hope this helps

[ricsatjr]

one alternative is to install pyfes in a separate environment (so there are no conflicts with already installed packages in coastsat), then generate the tide predictions, save them into a .csv, and then activate the coastsat environment and load them into the workflow. You want to have a go at that? let me know if any issues

Thanks. Will try this out.

[waterlovers]

I've tried to create a new environment, installed pyfes using the "conda install pyfes -c fbriol" command in my new environment. In the conda environment that I've made, I've verified that the conda environment does contain the pyfes package, but have been unable to import the pyfes module in Python or in the Jupyter notebook (note that these were attempted within the environment forementioned). Trying to do so returns an "ImportError: DLL load failed: the specified module could not be found".

I've tried to add the paths to the package in my system's environment variables, but the problem persists.

[kvos]

not sure what the DLL error is, make sure you open the Anaconda prompt as administrator

[waterlovers]

@kvos Thank you very much for all the help, I finally managed to run it without issue!

I opened Anaconda as administrator, created the environment and installed the pyfes package in the environment. I made two notebooks - one I used for the processes executed in the coastsat environment, and the other for processes to be executed in the environment for the pyfes package.

I think I've got an issue for the tidal frequency band plots, though, and I'm not sure what the cause is. I'd like to ask about it on another thread.

[kvos]

hi @waterlovers , it seems like the error was that pyfes is only compatible with python 3.7, so if you have python 3.8 installed in your coastsat env you will need to re-install it with python 3.7 as conda will not downgrade your python version.