Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Oracle Cloud Infrastructure (OCI) Provider #2367

Conversation

anders-swanson
Copy link
Contributor

@anders-swanson anders-swanson commented Dec 5, 2023

Oracle Cloud Infrastructure (OCI) provider

What does this PR change?

  • Adds a new cloud service provider implementation for OCI.

Does this PR relate to any other PRs?

  • N/A

How will this PR impact users?

  • Users running clusters on OCI will be able to access pricing data.

Does this PR address any GitHub or Zendesk issues?

How was this PR tested?

  • Tested on OCI with OKE and self-managed clusters.

Does this PR require changes to documentation?

Have you labeled this PR and its corresponding Issue as "next release" if it should be part of the next OpenCost release? If not, why not?

  • Release eligibility has not yet been determined.

Copy link

vercel bot commented Dec 5, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
opencost ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 22, 2024 6:32pm

Copy link
Collaborator

@mattray mattray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions related to maintainability going forward, but it looks good to me.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to be updated frequently? Is there somewhere we could retrieve this if we needed to recreate/update it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's compiled from the cost estimator APIs, which are subject to change and could not be directly queried. When new shapes are added, this data may need to be recompiled.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we document recompiling that data? I just worry that this will go stale and we won't know how to fix it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we could comment in here the source of the part numbers and how we're going to maintain them?

pkg/cloud/oracle/provider.go Show resolved Hide resolved
@@ -0,0 +1,39 @@
package oracle

var oracleRegions = []string{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

source of the list?

@mattray
Copy link
Collaborator

mattray commented Dec 6, 2023

I'm not very familiar with adding a provider, @AjayTripathy or @Sean-Holcomb, could either of you review this please?

"storage": "0.00005479452",
"defaultLBPrice": "0.0113",
"internetNetworkEgress": "0.0085",
"oraclePricingEndpoint":"https://apexapps.oracle.com/pls/apex/cetools/api/v1/products"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be configurable? When would you want to change it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple situations:

  • You have custom pricing data you want to serve
  • Your cluster is disconnected from the internet, or otherwise doesn't have access to this URI

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Oracle provide an endpoint with custom pricing in the same format or would the user set it up themselves, If it is Oracle provided is there authorization associated or would the expectation be that they are using a kind of workload identity auth?

For the air gapped example, is the thought that they would have an alternate endpoint available on a vpn or it would just be disable?

As an explanation, I am hammering on this point because that config file is a bit of a mess and I don't want to increase dependencies on it. In the short term I would adding that URL as a const

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Oracle has custom pricing, which requires authentication to access, as it is from your tenancy. Until this integration is done, custom pricing would have to be from a user-supplied URI.
  2. For scenarios where the default URI is not accessible, users could provide their own endpoint.

Couple thoughts on the config file:
If the config file is too large, what would another means of providing external configurations?
OR, If there's another parameter we can reuse, that might also work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anders-swanson sorry for the slow responses here, we've been on a holiday schedule:)

Many providers have huge config files-- we tokenize them and scan them to extract pricing for just the resources you're running. We should definitely not block this PR on that, though. Would be a great todo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Ajay,
I can move this to an envrionment variable. Is there any concern with overloading environment variables with config values?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overloaded env variables with default is preferable to this config file having additional fields added to it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! It may be some time until I can implement the update due to holiday break, but know that it will be done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @anders-swanson checking back in. Still on your radar? I'm hopeful this should be really quick!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, sorry for the delay, I've just returned from winter holidays. I plan to push out these changes tomorrow 🤞

@mattray mattray added opencost OpenCost issues vs. external/downstream P2 Estimated Priority (P0 is highest, P4 is lowest) kubecost Relevant to Kubecost's downstream project E2 Estimated level of Effort (1 is easiest, 4 is hardest) and removed needs-follow-up labels Dec 11, 2023
@mattray
Copy link
Collaborator

mattray commented Dec 18, 2023

@anders-swanson OpenCost 1.108 is shipping with Cloud Costs, do you want (in a separate PR) to add Oracle support for this too?

Once @Sean-Holcomb or @AjayTripathy or one of the other @opencost/opencost-maintainers signs off on a review we'll merge for the next release

@AjayTripathy
Copy link
Contributor

This basically LGTM @anders-swanson -- just see my note about swapping the configuration of the pricing information to an environment variable

@anders-swanson
Copy link
Contributor Author

@anders-swanson OpenCost 1.108 is shipping with Cloud Costs, do you want (in a separate PR) to add Oracle support for this too?

I may be able to integrate cloud costs in a follow-up PR.

@AjayTripathy
Copy link
Contributor

@anders-swanson OpenCost 1.108 is shipping with Cloud Costs, do you want (in a separate PR) to add Oracle support for this too?

I may be able to integrate cloud costs in a follow-up PR.

Yep, happy to not block on that!

@mattray
Copy link
Collaborator

mattray commented Jan 19, 2024

@anders-swanson you mentioned in Slack you're still working on getting approval on this. In the meantime could you get the DCO added to your commits?

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
@anders-swanson
Copy link
Contributor Author

@anders-swanson you mentioned in Slack you're still working on getting approval on this. In the meantime could you get the DCO added to your commits?

Rebased for DCO, thanks!

return err
}
if o.RateCardStore == nil {
url := os.Getenv("OCI_PRICING_URL")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you instead do this in https://github.com/opencost/opencost/blob/develop/pkg/env/costmodelenv.go and supply a default for pricing data that would work for most users?

@AjayTripathy
Copy link
Contributor

One quick note @anders-swanson then we should be good.

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
@AjayTripathy AjayTripathy merged commit 9b43b49 into opencost:develop Jan 22, 2024
5 checks passed
Copy link

sonarcloud bot commented Jan 22, 2024

Quality Gate Passed Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

3 New issues
0 Security Hotspots
18.4% Coverage on New Code
1.4% Duplication on New Code

See analysis details on SonarCloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E2 Estimated level of Effort (1 is easiest, 4 is hardest) kubecost Relevant to Kubecost's downstream project needs-follow-up opencost OpenCost issues vs. external/downstream P2 Estimated Priority (P0 is highest, P4 is lowest)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Oracle Cloud Infrastructure (OCI) Provider implementation
4 participants