Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxying private repository doesn`t work #29

Open
xrow opened this issue Aug 15, 2018 · 2 comments · May be fixed by #37
Open

Proxying private repository doesn`t work #29

xrow opened this issue Aug 15, 2018 · 2 comments · May be fixed by #37

Comments

@xrow
Copy link

xrow commented Aug 15, 2018 •
edited

What are you trying to do?
I am trying to proxy the private repo https://updates.ez.no/ttl

https://updates.ez.no/ttl/packages.json

{
    "packages": [],
    "includes": {
        "include/all$b48dd9ea8d0e45a2d2032c902f86ffbefd2fad9c.json": {
            "sha1": "b48dd9ea8d0e45a2d2032c902f86ffbefd2fad9c"
        }
    }
}

What feature or behavior is this required for?
proxy a private composer repo

How could we solve this issue? (Not knowing is okay!)
Do not know.

Anything else?
Yes .. Packagist works for me. It looks like that is a problem with repositories with http auth. Maybe the error is not with http auth maybe it is related to the json formatting and the keyword "includes"

I can provide the username and password if needed.

Also I had configured a group repository including this private one. When accessing the group one i show this error in the logs

2018-08-15 01:00:48,704+0000 INFO  [qtp1870042037-220]  admin org.sonatype.nexus.repository.httpclient.internal.HttpClientFacetImpl - Repository status for ezsystems-bul changed from READY to AVAILABLE - reason n/a for n/a
2018-08-15 01:00:49,094+0000 WARN  [qtp1870042037-220]  admin org.sonatype.nexus.repository.httpbridge.internal.ViewServlet - Failure servicing: GET /repository/composer-all/packages.json
java.lang.NullPointerException: null
	at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:882)
	at org.sonatype.nexus.repository.composer.internal.ComposerProxyFacetImpl.generatePackagesJson(ComposerProxyFacetImpl.java:164)
	at org.sonatype.nexus.repository.composer.internal.ComposerProxyFacetImpl.store(ComposerProxyFacetImpl.java:106)
	at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.doGet(ProxyFacetSupport.java:222)
	at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.lambda$1(ProxyFacetSupport.java:209)
	at org.sonatype.nexus.repository.proxy.Cooperation$CooperatingFuture.download(Cooperation.java:262)
	at org.sonatype.nexus.repository.proxy.Cooperation.download(Cooperation.java:194)
	at org.sonatype.nexus.repository.proxy.Cooperation.cooperate(Cooperation.java:93)
	at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.get(ProxyFacetSupport.java:201)
	at org.sonatype.nexus.repository.proxy.ProxyHandler.handle(ProxyHandler.java:49)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.handlers.ContentHeadersHandler.handle(ContentHeadersHandler.java:44)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.http.PartialFetchHandler.handle(PartialFetchHandler.java:59)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.handlers.ConditionalRequestHandler.handle(ConditionalRequestHandler.java:72)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.cache.NegativeCacheHandler.handle(NegativeCacheHandler.java:56)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.assetdownloadcount.internal.AssetDownloadCountContributedHandler.handle(AssetDownloadCountContributedHandler.java:53)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at com.sonatype.nexus.clm.internal.QuarantineContributedHandler.handle(QuarantineContributedHandler.java:69)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.handlers.HandlerContributor.handle(HandlerContributor.java:67)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:44)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:52)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.Context$proceed$0.call(Unknown Source)
	at org.sonatype.nexus.repository.composer.internal.ComposerRecipeSupport$_closure1.doCall(ComposerRecipeSupport.groovy:110)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1099)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
	at groovy.lang.Closure.call(Closure.java:418)
	at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
	at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
	at com.sun.proxy.$Proxy229.handle(Unknown Source)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:46)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
	at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:64)
	at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
	at org.sonatype.nexus.repository.group.GroupHandler.getAll(GroupHandler.java:181)
	at org.sonatype.nexus.repository.group.GroupHandler.getAll(GroupHandler.java:149)
	at org.sonatype.nexus.repository.composer.internal.ComposerGroupMergingHandler.doGet(ComposerGroupMergingHandler.java:52)
	at org.sonatype.nexus.repository.group.GroupHandler.handle(GroupHandler.java:93)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.assetdownloadcount.internal.AssetDownloadCountContributedHandler.handle(AssetDownloadCountContributedHandler.java:53)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at com.sonatype.nexus.clm.internal.QuarantineContributedHandler.handle(QuarantineContributedHandler.java:69)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.handlers.HandlerContributor.handle(HandlerContributor.java:67)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:44)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:52)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.Context$proceed$0.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
	at org.sonatype.nexus.repository.composer.internal.ComposerRecipeSupport$_closure1.doCall(ComposerRecipeSupport.groovy:110)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1099)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
	at groovy.lang.Closure.call(Closure.java:418)
	at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
	at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
	at com.sun.proxy.$Proxy229.handle(Unknown Source)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:46)
	at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
	at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
	at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:64)
	at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
	at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:43)
	at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.dispatchAndSend(ViewServlet.java:210)
	at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.doService(ViewServlet.java:172)
	at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.service(ViewServlet.java:126)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
	at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
	at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
	at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
	at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
	at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
	at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
	at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
	at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.sonatype.nexus.repository.httpbridge.internal.ExhaustRequestFilter.doFilter(ExhaustRequestFilter.java:80)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
	at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:531)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
	at java.lang.Thread.run(Thread.java:748)
@DarthHater
Copy link
Member

DarthHater commented Nov 2, 2018 •
edited

Well, it's certainly interesting:

https://github.com/sonatype-nexus-community/nexus-repository-composer/blob/master/src/main/java/org/sonatype/nexus/repository/composer/internal/ComposerProxyFacetImpl.java#L164

That seems to indicate there is no payload in the response.

I would set a breakpoint in there at that line and examine the response object to see what you are getting from the remote. My guess is a 403 or something akin.

NOTE: if you run Nexus Repo with the flag debug, you should be able to connect using IntelliJ, etc... and debug the plugin

Not sure how handy you are with Java, etc... but figured I'd pop some info in on how to debug this and get a bit more info.

@roodkcab
Copy link

did u create a private repo using satis? if that so then u have to parse include.json and transform it into /package/list.json for list all your projects, and a /p/%package%.json for a single one.

`location =/packages/list.json {
rewrite ^/(.*)$ /index.php?a=list last;
}

location ~ /p/ {
rewrite ^/p/(.*).json$ /index.php?a=p&p=$1 last;
}`

`<?php
$packages = json_decode(file_get_contents(dirname(FILE) . '/packages.json'), true);
foreach ($packages['includes'] as $k => $v) {
$includes = json_decode(file_get_contents(dirname(FILE) . "/{$k}"), true);
}

header('Content-Type: application/json');
switch($_GET['a']) {
case 'list':
echo json_encode(['packageNames' => array_keys($includes['packages'])]);
break;
case 'p':
$p = $_GET['p'];
echo json_encode(['packages' => [$p => $includes['packages'][$p]]]);
break;
}`

and since satis use tar instead of zip for package, u have to add tarball parser to nexus-composer
I upload such change to https://github.com/roodkcab/nexus-repository-composer.

@roodkcab roodkcab linked a pull request Apr 21, 2019 that will close this issue
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add support for satis roodkcab/nexus-repository-composer
3 participants
@xrow @roodkcab @DarthHater