Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add upper bounds to DSP.jl #5230

Merged
merged 3 commits into from
Nov 22, 2019
Merged

Conversation

dlfivefifty
Copy link
Contributor

This should fix JuliaDSP/DSP.jl#320 though I'm not sure I've done the upper bounds correctly.

I'm not the maintainer for the package so @ararslan might have better ideas.

@ararslan
Copy link
Member

ararslan commented Nov 9, 2019

Thanks for doing this, I think it's a good idea. I can't speak the correctness of the bounds though, so I'll request some reviews.

D/DSP/Compat.toml Outdated Show resolved Hide resolved
@galenlynch
Copy link
Contributor

This seems like a great idea. It seems like people are still experiencing issues caused by JuliaMath/AbstractFFTs.jl#26 (e.g. JuliaDSP/DSP.jl#323 and maybe JuliaDSP/DSP.jl#324) that these bounds should prevent. As such, it would be great if this PR could be merged sometime soon to fix those problems.

Is this waiting for @dlfivefifty to implement the changes suggested by @fredrikekre or for the additional reviews?

Co-Authored-By: Fredrik Ekre <ekrefredrik@gmail.com>
@fredrikekre fredrikekre merged commit 66a9c4a into JuliaRegistries:master Nov 22, 2019
fredrikekre added a commit that referenced this pull request Nov 22, 2019
fredrikekre added a commit that referenced this pull request Nov 22, 2019
@fredrikekre
Copy link
Member

Reverted, causes

ERROR: Unknown package AbstractFFTs found in the compatibility requirements of DSP [717857b8]
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] Pkg.GraphType.Graph(::Dict{Base.UUID,Set{VersionNumber}}, ::Dict{Base.UUID,Dict{Pkg.Types.VersionRange,Dict{String,Base.UUID}}}, ::Dict{Base.UUID,Dict{Pkg.Types.VersionRange,Dict{String,Pkg.Types.VersionSpec}}}, ::Dict{Base.UUID,String}, ::Dict{Base.UUID,Pkg.Types.VersionSpec}, ::Dict{Base.UUID,Pkg.Types.Fixed}, ::Bool) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/GraphType.jl:278

Please at least try to use the packages you modify before submitting PRs.

@galenlynch
Copy link
Contributor

Weird, I did get that error transiently on my own computer, but not reliably. AbstractFFTs is a registered package, and the UUID in DSP.jl's Deps.toml matches that of AbstractFFTs' Package.toml. @fredrikekre what could be causing that error?

@dlfivefifty
Copy link
Contributor Author

Please at least try to use the packages you modify before submitting PRs.

I did. Someone else can make the PR then, I was just trying to help since its affecting a lot of users.

@fredrikekre
Copy link
Member

For a version X you are not allowed to put compat on Y unless version X depends on Y.

@dlfivefifty
Copy link
Contributor Author

So how do we fix the bug then? version X depends on Z which in turn depends on Y.

@galenlynch
Copy link
Contributor

Oh I think I understand the problem... the merge with master changed the headers in Compat.toml but not Deps.toml

@fredrikekre
Copy link
Member

You can try to use a tool such as https://github.com/bcbi/RetroCap.jl

@dlfivefifty
Copy link
Contributor Author

More precisely: DSP depends on FFTW which depends on AbstractFFTs. So I guess we need to put the upper bound on FFTW?

@fredrikekre
Copy link
Member

I don't know what the original problem was, but yea, sounds like it.

@galenlynch
Copy link
Contributor

I was mistaken. However, it seems like this fixes it...

in Deps.toml AbstractFFTs is currently under the "0-0.5" section, but it's also required for DSP v0.6. If you move the AbstractFFTs line from "0-0.5" to "0" then it works.

galenlynch added a commit to galenlynch/General that referenced this pull request Nov 22, 2019
@fredrikekre
Copy link
Member

@galenlynch
Copy link
Contributor

Ah, you're right.

@galenlynch
Copy link
Contributor

So what's the path forward? DSP versions v0-0.6.0 are incompatible with AbstractFFTs v0.5, but because DSP v0.6 does not explicitly depend on AbstractFFTs, it can't specify which versions it's compatible with. But as @ararslan pointed out in JuliaMath/FFTW.jl#124, FFTW re-exports AbstractFFTs, and FFTW v0-v1.0 does not specify an upper bound on AbstractFFTs. Therefore, it seems like DSP cannot restrict which version of AbstractFFTs it is used with by setting an upper bound on FFTW. Is the only solution to make DSP depend on AbstractFFTs so it can specify its compatibility?

@galenlynch
Copy link
Contributor

Maybe another solution would be to add compatibility upper bounds to older versions of FFTW.

@galenlynch
Copy link
Contributor

It seems weird that you have no control over what FFTW exports if you use a version of it lower than v1.1, because it re-exports AbstractFFTs and and has no upper bound on the version of AbstractFFTs.

@dlfivefifty
Copy link
Contributor Author

Probably a simple solution is just restrict FFTW to < 1

@galenlynch
Copy link
Contributor

galenlynch commented Nov 22, 2019

I don't think that will fix it, since FFTW < 1 doesn't place any bounds on which version of AbstractFFTs you get.

EDIT: it doesn't place any upper bound on which version of AbstractFFTs you get.

@dlfivefifty
Copy link
Contributor Author

Then we should add upper bounds to FFTW.jl

@fredrikekre
Copy link
Member

I guess this is an example why it is a bad idea to @reexport one of your dependencies -- packages depend on something they can't control.

galenlynch added a commit to galenlynch/General that referenced this pull request Nov 23, 2019
This is another attempt at JuliaRegistries#5230, which was an attempt at fixing the breakage
caused by JuliaMath/AbstractFFTs.jl#26 (e.g. JuliaDSP/DSP.jl#320,
JuliaDSP/DSP.jl#323, JuliaDSP/DSP.jl#324). The underlying problem is that
JuliaMath/AbstractFFTs.jl#26 moved functions from DSP.jl to AbstractFFTs.jl,
requiring users to have versions of both of those packages either before that
change or after it. Because FFTW reexports AbstractFFTWs, users also have to
coordinate versions of DSP.jl and FFTW.jl.

This problem was fixed for the most recent versions of DSP.jl and
FFTW.jl (v0.6.1 and v1.1, respectively) by JuliaDSP/DSP.jl#322 and
JuliaMath/FFTW.jl#124. However, because earlier versions of FFTW do not have an
upper bound for its dependency on AbstractFFTs, and neither does DSP, Pkg cannot
currently tell that some combinations of versions for AsbtractFFTs and DSP.jl
are incompatible, which is causing users to experience issues even after the
problem was fixed for the most recent versions (JuliaDSP/DSP.jl#323,
JuliaDSP/DSP.jl#324).

Fixing the problem is made slightly more complicated because DSP does not
explicitly depend on AbstractFFTs, but instead DSP implicitly depends on
AbstractFFTs through FFTW's reexport of it, and in turn DSP explicitly depends
on FFTW.

To fix this problem, I have changed the Compat.toml for FFTW to place a upper
bound on AbstractFFTs, so that packages get a consistent interface when using
FFTW. I have also changed the Compat.toml for DSP, to avoid breakage caused by
AbstractFFTs v0.5 taking some functions that were previously defined in DSP.

In addition, the Compat.toml for DSP showed versions before 0.5.1 to be
compatible with Julia 1, which was inaccurate. I have therefore changed this
TOML table to restrict the DSP versions to the range that is compatible with
Julia 1.

I have tested that these changes work for the most recent versions of DSP and
FFTW, as well as older versions.
ararslan pushed a commit that referenced this pull request Nov 25, 2019
This is another attempt at #5230, which was an attempt at fixing the breakage
caused by JuliaMath/AbstractFFTs.jl#26 (e.g. JuliaDSP/DSP.jl#320,
JuliaDSP/DSP.jl#323, JuliaDSP/DSP.jl#324). The underlying problem is that
JuliaMath/AbstractFFTs.jl#26 moved functions from DSP.jl to AbstractFFTs.jl,
requiring users to have versions of both of those packages either before that
change or after it. Because FFTW reexports AbstractFFTWs, users also have to
coordinate versions of DSP.jl and FFTW.jl.

This problem was fixed for the most recent versions of DSP.jl and
FFTW.jl (v0.6.1 and v1.1, respectively) by JuliaDSP/DSP.jl#322 and
JuliaMath/FFTW.jl#124. However, because earlier versions of FFTW do not have an
upper bound for its dependency on AbstractFFTs, and neither does DSP, Pkg cannot
currently tell that some combinations of versions for AsbtractFFTs and DSP.jl
are incompatible, which is causing users to experience issues even after the
problem was fixed for the most recent versions (JuliaDSP/DSP.jl#323,
JuliaDSP/DSP.jl#324).

Fixing the problem is made slightly more complicated because DSP does not
explicitly depend on AbstractFFTs, but instead DSP implicitly depends on
AbstractFFTs through FFTW's reexport of it, and in turn DSP explicitly depends
on FFTW.

To fix this problem, I have changed the Compat.toml for FFTW to place a upper
bound on AbstractFFTs, so that packages get a consistent interface when using
FFTW. I have also changed the Compat.toml for DSP, to avoid breakage caused by
AbstractFFTs v0.5 taking some functions that were previously defined in DSP.

In addition, the Compat.toml for DSP showed versions before 0.5.1 to be
compatible with Julia 1, which was inaccurate. I have therefore changed this
TOML table to restrict the DSP versions to the range that is compatible with
Julia 1.

I have tested that these changes work for the most recent versions of DSP and
FFTW, as well as older versions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FFTW and DSP collision over Periodograms
4 participants