-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Breaking] Switch to DimensionalData #249
Conversation
This is currently failing at the CubeIterator. I suspect it does not put the name of the name of the dimension into the column but the whole column.
The currently failing tests are how to access the dimension of a cube by name Another failure is, that the YAXArray now does not throw an exception when there is a length mismatch between the dimensions and the data. |
could you pin point exactly where in the code is this happening? Maybe @rafaqz could see a possible solution very quickly, if he happens to have a minute 😄 . |
The refdims is more a conceptual question of what do you try to achieve with it. I will have a look into the not throwing with the length mismatch tomorrow. What would be the way to access a dimension of a DimArray based on the dimension name? |
Its just a tuple like dims. You can define it to always return an empty tuple if you dont want it. |
One gotcha may be the need to run See here: Then if you use strings it will choose a Without |
Skip the batchextract tests for now, because it is not clear how to include it into the getindex dispatch.
The plan is to move the findAxis functions that are needed for mapcube into the DAT.jl module and let every Axis interaction be handled by DimensionalData.jl
@meggart I just pushed the current state of this PR and it "only" fails because of the getindex of the Dataset and I also run into a getindex ambiguity between YAXArray and AbstractDimArray getindex functions. I solved it first by removing the YAXArray getindex method, but then the getindex seems to be not lazy anymore in some cases. |
Co-authored-by: Fabian Gans <meggart@users.noreply.github.com>
The subsetting is now working. I don't understand both tests not well enough to proceed from here. |
Project.toml
Outdated
@@ -21,6 +21,7 @@ IntervalSets = "8197267c-284f-5f27-9208-e0e47529a953" | |||
IterTools = "c8e1da08-722c-5040-9ed9-7db0dc04731e" | |||
Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a" | |||
OffsetArrays = "6fe1bfb0-de20-5000-8ca7-80f57d26f881" | |||
OhMyREPL = "5fb14364-9ced-5910-84b2-373655c76a03" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I love OhMyREPL but I guess this should not be here :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, true.
DimensionalData
This has been included by accident
DD has limited its exports a lot so it might be ok to just reexport everything now. The main pain point is there are a few annoying clashes with DataFrames.jl, but im in the process of getting rid of some of them. |
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
These need to be switched to the new syntax later.
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
The documentation builds locally, but I have no idea how to debug the Documenter build on the Github Actions. @lazarusA do you have some hints on fixing the Documenter build on CI? |
just removing esdl_study1 should do. And then remove |
@@ -12,7 +12,7 @@ function getfiles() | |||
srcsfiles = [] | |||
for f in folders | |||
names = readdir(joinpath(@__DIR__, ".", "examples", f)) | |||
setdiff!(names, [".DS_Store", "examples_from_esdl_study_4.jl"]) | |||
setdiff!(names, [".DS_Store", "examples_from_esdl_study_1.jl","examples_from_esdl_study_2.jl","examples_from_esdl_study_3.jl","examples_from_esdl_study_4.jl", "create_from_func.jl"]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the issue is only examples_from_esdl_study_1.jl
due to conda/pyplot/pycall
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also, you will need to remove
- name: Install matplotlib |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the others because I haven't updated the EarthDataLab package to the DimensionalData syntax yet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mmm... we should not removed them I think unless @meggart says is ok. These are really recent examples. Even the one with pyplot should be replaced and not removed, I will do it later this week. But, the others, your tests should also pass those 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem is not the examples per se, but we would need to bring EarthDataLab to the DimensionalData syntax and this might also be a bit more work. They are only removed until we managed to update EarthDataLab.
This should make the CI work
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
This should enable the build of the docs on CI because we have to keep track to use the new version of our package.
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
This pull request is going to make the YAXArray type a subtype of AbstractDimArray.
With that we will have a better interoperability between YAXArrays.jl and Rasters.jl
The tests are working, but the documentation is not deploying yet so I will fix the documentation and then I am going to merge it.
This PR is heavily breaking, because it changes the indexing syntax.