Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash when two resources of the same kind with the same name exist in different namespaces. #33

Closed
luisdavim opened this issue Apr 15, 2019 · 4 comments
Labels
bug Something isn't working

Comments

@luisdavim
Copy link

Hi.

I think I found an issue:

2019/04/15 12:47:10 Checked prometheus endpoint: http://prom-op-prometheus-operato-prometheus.monitoring.svc.cluster.local:9090
2019/04/15 12:47:10 Found ProviderID starting with "azure", using Azure Provider
2019/04/15 12:47:10 Recording prices...
2019/04/15 12:47:10 Interface map[error:found duplicate series for the match group {persistentvolumeclaim="data-vol-kube-opex-analytics-0"} on the left hand-side of the operation: [{persistentvolumeclaim="data-vol-kube-opex-analytics-0", storageclass="standard-disk"}, {persistentvolumeclaim="data-vol-kube-opex-analytics-0", storageclass="<none>"}];many-to-many matching not allowed: matching labels must be unique on one side errorType:execution status:error]. If the interface is nil, prometheus is not running!
panic: interface conversion: interface {} is nil, not map[string]interface {}

goroutine 85 [running]:
github.com/kubecost/cost-model/costmodel.getPVInfoVector(0x1852100, 0xc00058cba0, 0xc0006e2480, 0x0, 0x0)
        /app/costmodel/costmodel.go:642 +0xb20
github.com/kubecost/cost-model/costmodel.ComputeCostData(0x1e088c0, 0xc000424000, 0xc000297680, 0x1e4cca0, 0xc000195110, 0x1b29747, 0x2, 0x0, 0x0, 0x0)
        /app/costmodel/costmodel.go:130 +0xe8b
main.(*Accesses).recordPrices.func1(0xc000424d00)
        /app/main.go:104 +0x476
created by main.(*Accesses).recordPrices
        /app/main.go:101 +0x3f
@dwbrown2
Copy link
Collaborator

Good catch, Luis. This is definitely an issue. It actually looks like you were for some reason provisioned a claim that has two storageclasses: standard-disk and <none>. I haven't seen this yet.

Is there any chance you could share the result to this Prometheus query? kube_persistentvolumeclaim_info This should help us pinpoint the issue a little faster.

An easy fix would be to exclude storageclass =<none> but it's not clear this addresses the underlying problem.

@dwbrown2 dwbrown2 added the bug Something isn't working label Apr 15, 2019
@luisdavim
Copy link
Author

sorry, I've ended up deleting the extra pvc that had been created by mistake in the default namespce but I can try to reproduce this with some other resource type.
What I had was two pvcs one in the kube-opex-analytics namespace with storage class set to standard-disk and another one with the same name on the default namespace with no storage class set.

@dwbrown2
Copy link
Collaborator

Got it, this is helpful. We should now be able to recreate but will follow up if for some reason we can't!

@dwbrown2
Copy link
Collaborator

I was able to recreate! PR out for review.

AjayTripathy added a commit that referenced this issue Apr 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants