Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency tree of stdlibs #25951

Open
KristofferC opened this issue Feb 8, 2018 · 14 comments

Comments

7 participants
@KristofferC
Copy link
Contributor

commented Feb 8, 2018

Below is the dependency graph of our stdlib:

screen shot 2018-02-08 at 14 29 31

There are a few unfortunate dependencies. One for example is the Test --> Distributed --> LinearAlgebra, which means that you cannot use the Test stdlib without having the LinearAlgebra stdlib.
Perhaps some refactorings to avoid this would be useful. Test uses only myid from Distributed for example.

Figure created with:

cd("/Users/kristoffer/julia/stdlib")
import TOML
using LightGraphs
using TikzGraphs
using TikzPictures

const STDLIB_DIR = "."
const STDLIBS = readdir(STDLIB_DIR)
deps = Dict{String, Int}()
for (i, stdlib) in enumerate(STDLIBS)
    deps[stdlib] = i
end
g = LightGraphs.SimpleGraphs.SimpleDiGraph(length(STDLIBS))

for (i, stdlib) in enumerate(STDLIBS)
    proj = TOML.parsefile(joinpath(STDLIB_DIR, stdlib, "Project.toml"))
    if haskey(proj, "deps")
        for (stdlib_dep, uuid) in proj["deps"]
            add_edge!(g, deps[stdlib], deps[stdlib_dep])
        end
    end
end
t = TikzGraphs.plot(g, Layouts.SimpleNecklace(), STDLIBS)
TikzPictures.save(SVG("graph"), t)  

@KristofferC KristofferC added the stdlib label Feb 8, 2018

@JeffBezanson

This comment has been minimized.

Copy link
Member

commented Feb 8, 2018

Nice, +1 to breaking dependency edges.

Distributed only uses LinearAlgebra in order to call BLAS.set_num_threads(1). That could be replaced with some kind of hook mechanism instead, so all threaded libraries can register callbacks to do the same thing.

Another one I noticed: SharedArrays uses a single sprintf call that could easily be replaced.

@Keno

This comment has been minimized.

Copy link
Member

commented Jun 24, 2018

I also just noticed we have a cycle: ["REPL", "InteractiveUtils", "Pkg"], Would be nice to break if possible.

@fredrikekre

This comment has been minimized.

Copy link
Member

commented Nov 12, 2018

Pkg -> REPL -> InteractiveUtils -> LinearAlgebra makes it impossible to use the package manager without BLAS 😛

@fredrikekre

This comment has been minimized.

Copy link
Member

commented Nov 12, 2018

Updated version of the tree
graph

@StefanKarpinski

This comment has been minimized.

Copy link
Member

commented Nov 12, 2018

Is it actually a DAG now?

@rschwarz

This comment has been minimized.

Copy link

commented Nov 21, 2018

GraphViz/dot says yes:

stdlib
(see gist)

Sorry, my analysis applies to the 1.0.2 release, of course, not some branch where refactoring is happening.

@KristofferC

This comment has been minimized.

Copy link
Contributor Author

commented Nov 21, 2018

Nice graph!

@StefanKarpinski

This comment has been minimized.

Copy link
Member

commented Nov 21, 2018

Yes, very nice graph! This dependency graph is starting to look a bit more reasonable.

@KristofferC

This comment has been minimized.

Copy link
Contributor Author

commented Nov 21, 2018

Test -> Distributed, InteractiveUtils -> LinearAlgebra, Distributed -> LinearAlgebra are probably the worst ones right now.

@fredrikekre

This comment has been minimized.

Copy link
Member

commented Nov 21, 2018

Fortunately #29978 removed InteractiveUtils -> LinearAlgebra and #30004 removes Distributed -> LinearAlgebra.

@fredrikekre

This comment has been minimized.

Copy link
Member

commented Dec 7, 2018

Update after #29978 and #30004 :

stdlibs

In particular LinearAlgebra is now pretty isolated to packages that are actually linalg related and e.g. the package manager is not dependent on LinearAlgebra anymore 🎉

@KristofferC

This comment has been minimized.

Copy link
Contributor Author

commented Dec 7, 2018

Very good, this looks quite reasonable.

@StefanKarpinski

This comment has been minimized.

Copy link
Member

commented Dec 7, 2018

Wow, that's such a nice, sane dependency graph now!

@Sacha0

This comment has been minimized.

Copy link
Member

commented Dec 7, 2018

Fantastic work! ❤️ 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.