The repository manager caches but does not use the cache #54

Closed
quintesse opened this Issue Oct 27, 2012 · 5 comments

Comments

Projects
None yet
2 participants
@quintesse
Member

quintesse commented Oct 27, 2012

I'm seeing remote dependencies downloaded each time I compile or run code.

@ghost ghost assigned alesj Oct 27, 2012

@quintesse

This comment has been minimized.

Show comment Hide comment
@quintesse

quintesse Oct 27, 2012

Member

I have a module that depends on a Java JAR module io.netty and this is the full CMR debug log:

$ ceylon run --verbose org.codejive.ceylon.httpd/1.0.2
Debug: Repository lookup order: 
Debug:  - /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:  - ./modules 
Debug:  - ./modules 
Debug:  - /home/tschotan/.ceylon/repo 
Debug:  - http://modules.ceylon-lang.org/test 
Debug: Looking for org.codejive.ceylon.httpd-1.0.2.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Found at /org/codejive/ceylon/httpd/1.0.2/org.codejive.ceylon.httpd-1.0.2.car 
Debug: Looking for io.netty-3.5.0.Final.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository /home/tschotan/.ceylon/repo 
Debug:   -> Not Found 
Debug:  Trying repository http://modules.ceylon-lang.org/test 
Debug: Got 404 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.car 
Debug:  Trying repository /home/tschotan/.ceylon/cache 
Debug:   -> Not Found 
Debug: Looking for io.netty-3.5.0.Final.jar 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository /home/tschotan/.ceylon/repo 
Debug:   -> Not Found 
Debug:  Trying repository http://modules.ceylon-lang.org/test 
Debug: Got 200 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar 
Debug:   -> Found at /io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar 
Debug: Fetching resource: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar 
Debug: Got 200 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar 
Debug: Creating local copy of external node: RemoteNode for RemoteContentStore: http://modules.ceylon-lang.org/test -> [/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar] at repo: /home/tschotan/.ceylon/cache 
Debug: Got 200 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar.sha1 
Debug: Fetching resource: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar.sha1 
Debug: Got 200 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar.sha1 
Debug: Looking for org.codejive.ceylon.options-1.0.0.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Found at /org/codejive/ceylon/options/1.0.0/org.codejive.ceylon.options-1.0.0.car 
Debug: Looking for ceylon.interop.java-0.3.3.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository /home/tschotan/.ceylon/repo 
Debug:   -> Found at /ceylon/interop/java/0.3.3/ceylon.interop.java-0.3.3.car 
Debug: Looking for ceylon.collection-0.3.3.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository /home/tschotan/.ceylon/repo 
Debug:   -> Found at /ceylon/collection/0.3.3/ceylon.collection-0.3.3.car 

The thing that jumps out:

Debug:  Trying repository http://modules.ceylon-lang.org/test 
Debug: Got 404 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.car 
Debug:  Trying repository /home/tschotan/.ceylon/cache 
Debug:   -> Not Found

First checking the remote repository, getting a 404 and only afterwards checking the cache? That seems the wrong way around.

Which is probably the reason why once it gets to check the JAR version it finds it remotely each time and starts downloading it.

Member

quintesse commented Oct 27, 2012

I have a module that depends on a Java JAR module io.netty and this is the full CMR debug log:

$ ceylon run --verbose org.codejive.ceylon.httpd/1.0.2
Debug: Repository lookup order: 
Debug:  - /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:  - ./modules 
Debug:  - ./modules 
Debug:  - /home/tschotan/.ceylon/repo 
Debug:  - http://modules.ceylon-lang.org/test 
Debug: Looking for org.codejive.ceylon.httpd-1.0.2.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Found at /org/codejive/ceylon/httpd/1.0.2/org.codejive.ceylon.httpd-1.0.2.car 
Debug: Looking for io.netty-3.5.0.Final.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository /home/tschotan/.ceylon/repo 
Debug:   -> Not Found 
Debug:  Trying repository http://modules.ceylon-lang.org/test 
Debug: Got 404 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.car 
Debug:  Trying repository /home/tschotan/.ceylon/cache 
Debug:   -> Not Found 
Debug: Looking for io.netty-3.5.0.Final.jar 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository /home/tschotan/.ceylon/repo 
Debug:   -> Not Found 
Debug:  Trying repository http://modules.ceylon-lang.org/test 
Debug: Got 200 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar 
Debug:   -> Found at /io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar 
Debug: Fetching resource: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar 
Debug: Got 200 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar 
Debug: Creating local copy of external node: RemoteNode for RemoteContentStore: http://modules.ceylon-lang.org/test -> [/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar] at repo: /home/tschotan/.ceylon/cache 
Debug: Got 200 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar.sha1 
Debug: Fetching resource: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar.sha1 
Debug: Got 200 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.jar.sha1 
Debug: Looking for org.codejive.ceylon.options-1.0.0.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Found at /org/codejive/ceylon/options/1.0.0/org.codejive.ceylon.options-1.0.0.car 
Debug: Looking for ceylon.interop.java-0.3.3.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository /home/tschotan/.ceylon/repo 
Debug:   -> Found at /ceylon/interop/java/0.3.3/ceylon.interop.java-0.3.3.car 
Debug: Looking for ceylon.collection-0.3.3.car 
Debug:  Trying repository /home/tschotan/projects/ceylon/ceylon-dist/dist/bin/../repo 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository ./modules 
Debug:   -> Not Found 
Debug:  Trying repository /home/tschotan/.ceylon/repo 
Debug:   -> Found at /ceylon/collection/0.3.3/ceylon.collection-0.3.3.car 

The thing that jumps out:

Debug:  Trying repository http://modules.ceylon-lang.org/test 
Debug: Got 404 for url: http://modules.ceylon-lang.org/test/io/netty/3.5.0.Final/io.netty-3.5.0.Final.car 
Debug:  Trying repository /home/tschotan/.ceylon/cache 
Debug:   -> Not Found

First checking the remote repository, getting a 404 and only afterwards checking the cache? That seems the wrong way around.

Which is probably the reason why once it gets to check the JAR version it finds it remotely each time and starts downloading it.

@alesj

This comment has been minimized.

Show comment Hide comment
@alesj

alesj Oct 27, 2012

Member

First checking the remote repository, getting a 404 and only afterwards checking the cache?

That's how you wanted it.
But you're changing your mind every few days ...

Member

alesj commented Oct 27, 2012

First checking the remote repository, getting a 404 and only afterwards checking the cache?

That's how you wanted it.
But you're changing your mind every few days ...

@quintesse

This comment has been minimized.

Show comment Hide comment
@quintesse

quintesse Oct 27, 2012

Member

From #47:

Or as @quintesse suggested, the cache should not be a normal repo but sort of a delegate of remote repos, where they would first try the cache before trying the remote repo. This way local repos that are before remote repos would not use the cache.

Member

quintesse commented Oct 27, 2012

From #47:

Or as @quintesse suggested, the cache should not be a normal repo but sort of a delegate of remote repos, where they would first try the cache before trying the remote repo. This way local repos that are before remote repos would not use the cache.

@quintesse

This comment has been minimized.

Show comment Hide comment
@quintesse

quintesse Oct 27, 2012

Member

But I think I already see how to change it, so I can do it right now if you want. Seems like a tiny fix.

Member

quintesse commented Oct 27, 2012

But I think I already see how to change it, so I can do it right now if you want. Seems like a tiny fix.

@alesj

This comment has been minimized.

Show comment Hide comment
@alesj

alesj Oct 27, 2012

Member

Yeah, it should be an easy change.
Just move that post-lookup code into pre-lookup code for remote repos + other conditions (e.g. not cheched already, etc).

Member

alesj commented Oct 27, 2012

Yeah, it should be an easy change.
Just move that post-lookup code into pre-lookup code for remote repos + other conditions (e.g. not cheched already, etc).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment