- High Level Update from respective working group participants about progress made over the last month.
- Shimpa to discuss thoughts on Persistent Storage Classes from discussion in discord
- Participants to update sig on [Content Moderation Working Group]
- Discussion of Support issue #3
- Review any new pulls in Akash Provider Repo
- Praetor team to give an update on their work.
- Open Discussion
- Date: Wednesday, February 22, 2023
- Time: 08:00 AM PT (Pacific Time)
- Recording
- Transcript
- Adam Bozanich
- Andrey Arapov
- Anil Murty
- Artur Troian
- Damir Simpovic
- Deval Patel
- Jigar Patel
- Joao Luna
- Scott Carruthers
- Confusing storage classes: Multiple people brought up issues regarding the naming of the storage classes (beta1, beta2, and beta3) and how it is confusing. The open question that was discussed during different times at this call was whether or not the storage classes should be based off speed (verifiable by benchmarks), or through explaining what hardware is used. Artur Troian specifically mentioned how different drives can be as performant as better drives if configured correctly RAID, and that benchmarks could show whether or not providers should "be allowed" to use the storage class they're advertising.
- Template SDLs leading to low usage of some storage classes: Damir Simpovic mentioned that most people who use persistent storage use the "default values" found in template SDLs (most commonly beta2) which leads to fewer tenants renting other storage classes (e.g., beta3) leading to a low number of leases.
- Not enough standardization within CPU-related attributes: Artur Troian mentioned that CPUs need attributes as tenants may need access to specific CPU specifications (i.e., models, brands, architectures, instructions, etc).
- Template SDLs leading to low usage of some storage classes: Adam Bozanich suggested a solution where SDLs can specify "anyOf" storage classes instead of a specific storage class, giving tenants who don't care which storage class they are renting more choices to pick from. Artur Troian added that single deployments could hog the speeds from any storage class, making the storage class be misleading for people who expect a certain speed.
- Benchmarking: Adam Bozanich talked about how benchmarks could show performance rather than relying only on storage classes. On the topic of benchmarking, Damir Simpovic mentioned that hardware can change over time on provider rigs in case of upgrades. Andrey Arapov mentioned briefly that providers could self-report in the form of attributes how updated their hardware is similar to how "hotel star's" work. Adam Bozanich mentioned concerns with explaining the hardware the provider rigs are running due to there being no mechanisms in place for ensuring that your deployment lands on the "correct rig" in case of multi-node clusters (e.g., two different processors, which one will you get?). He mentions that benchmarks and "hotel star's" could both work if it can be standardized.
- Confusing storage classes / Template SDLs leading to low usage of some storage classes: The provider attributes need to be improved, mainly for storage classes, where they currently do not mean much to the tenant. The names of the storage classes should also be reconsidered to be changed to something more understandable, as beta1-3 does not say much to a tenant.
- Benchmarking: Benchmarking and standardization within it must be improved and trackable. This will help tenants pick providers from performance and not only attributes.
- Adam Bozanich mentioned that the working group related to content moderation is almost finished.
- Damir Simpovic mentioned that he might create another YouTube video on the topic of how to build provider rigs, consisting of used parts, for reasonable prices.
- Jigar Patel mentioned how the Praetor team was able to create a workaround for the dropping leases issue from the last monthly call.
- Adam Bozanich quickly talked about Luna's PRs for the cert manager which could turn into SSL for all host names.
This editable transcript was computer generated and might contain errors. People can also change the text after it was created.
Scott Carruthers: Okay, so welcome everyone. To I believe this is our Second or Third, Sig-providers Session, As mentioned, Tyler our our normal MC is not able to make it today. So I will run through the agenda before running through the agenda, any Anything pressing that? We Want to discuss before getting started.
Scott Carruthers: okay, let me Take a look at the agenda. So, the first thing that again, this is a the agenda that was posted on the The Github repository and the agenda for this specific succession.
Adam Bozanich: and,
Scott Carruthers: So to start off with the the agenda has A discussion item for any updates on working groups. So, content, provider moderation, or any of the working groups. And this is kind of a, just a Open discussion for the group. Is there anything going on and they provider working groups that this group should be made aware of
Adam Bozanich: We basically finished up the Content moderation.
Adam Bozanich: Working group is very close to being done. There's a couple Pieces of feedback that I have to incorporate into one of the PRS. But aside from that my understanding is that we're we're quite done with it and we're ready to start moving on. To implementation.
Scott Carruthers: Okay. Yeah, I don't think there's any additional content. Moderation, workgroup sessions, currently scheduled. So yeah. That would make sense that working group is essentially finishing up and now's being passed back to the SIG for PR days and natural work.
Scott Carruthers: Okay, in Anil, maybe you may be able to help here as well. I'd have to go back to the list. I'm not sure if there's any other ongoing working groups that are associated with this SIG that have met since we met last month that we might want to touch on.
Anil Murty: No, I think it's just a content moderation and…
Scott Carruthers: Okay. Okay.
Anil Murty: GPU, those little thing.
Scott Carruthers: Pretty, pretty short and simple. So if there are no other and…
Adam Bozanich: Here.
Scott Carruthers: it sounds like content, moderation is the only one in Adam. Give us an update on that. So I don't think there's any other working groups that need coverage. so, I think that most of the items in this agenda, we're going to be able to Check off pretty quickly and then we can just get into open discussion and some interesting conversation. so the next thing that was listed in the agenda in Damir, I not I don't want to Catch you off, guard on this and your call if there's anything to discuss. But in the agenda, you had raised recently that
Scott Carruthers: the suggestion and I think this is always the intense and and Artur could definitely speak on this, but you made a suggestion and discord that for persistent storage. We get away from the nondescripts beta 1 Beta 2 Beta 3 storage classes. So again, I think we probably all agree with that and eventually want to come up with more Discrimp names near anything that you wanted to or Artur. Did you have something before?
Artur Troian: Notice that? Yeah, a few words about the presentation.
Scott Carruthers: I'm sorry. Artura, I didn't catch the
Artur Troian: Okay, yeah, yeah. So the better names. They just pretty much sort of suggestion of how many storage classes we have.
Artur Troian: I remember there was a suggestion and blimp from the last seed being about getting rid of pretty much all of them. Accept metal three. Which are not quite agree on that. And I think the main reason for the those nightings where to
Artur Troian: Show how like, what speeds and latencies can be, you know, provider for different various, for example, of laws. And before we rename you them, that needs to be sort of assessed. how amount of replicas for example a fact latency or how amount of replicas affects speed along these, for example, we have, you know, SSD obviously affects speed and then be Which is, you know, you can get them. Give me rice like eight, he gives per second. But once again if the files are are small, the IOP didn't really all. So, yeah, there are a bunch of things that needs to be soldiers and I would suggest you start actually doing the PRD for that.
00:05:00
Artur Troian: And starts only discussing on what the Hollywood name, those social classes.
Scott Carruthers: Okay. Yeah, they certainly makes sense to create a PRD for that. So again, Damir I think we all agree that we need to get away from the The Beta names eventually. So Damir wrote a pretty extensive post, and And discord with some suggestions. So, yeah, go ahead Damir.
Damir Simpovic: Um, okay. Thank you. And hello everyone. the point that I was trying to make, I might not have actually hit the point but The point is that the the biggest modeling is the network speed and not the drive speed or the Bus that they drive is connected to. So, as Artur has pointed pointed out the nvme drives, the modern nvme drives can hit eight gigabytes per second, which is roughly roughly just roughly speaking 80 gigabits per second. Those those are PCI Express, 4 Generation, 4 speeds, you cannot achieve those speeds with Gen 3 drives.
Damir Simpovic: and you will most certainly not hit those speeds, even if you have a system that's PCI Express Generation 4 and you have PCI gene for nvme drives that are capable of hitting those speeds. Those speeds will not be.
Damir Simpovic: There will be no impact. Basically, if you're running this on a Gigabit network because the Gigabit network will get set, saturated immediately like this so, and the point of persistent storage is That it's distributed among multiple nodes, which automatically means. Network. Yeah, it will hit the network. It has to hit the network because it's a safe cluster. I mean I'm running a single node set cluster but I'm my case might be a bit. Different than what's suggested and I would agree. Definitely. But in my case, for me, it made sense to make a single make just a single load. With all the persistent storage drives are basically concentrating in this single node and everything else. All the compute nodes are
Damir Simpovic: our separate. Kind of machines. So yeah, let's the network speed is also included in the kind of in the description. Of the storage class. The storage classes have basically very little point because the only really perceivable difference is when you have a spinning hard drive and everything else. Basically, and the difference is not in the In the throughput, but in Iops the number of iops. Jumps dramatically. when you use a spinning hard drive and then you switch to any type of flash drive, Whether it's
Damir Simpovic: it doesn't matter on which bus it's connected to. Is it a sada? Or is it sass? Or is it nvme or whatever it is?
Damir Simpovic: So the IOPS in the Flash group is will be way, way higher. We're talking hundreds of thousands of Iops versus a few hundred. so, it's kind of like, Three orders of magnitude higher or even higher, or even form orders of magnitude high on, on very, very modern hard. Flash drives, nvme drives, the IPS can go over a million and on top and spinning hard drive, you will maybe hit, I don't know, 200 300 So that's that's really the biggest difference. and yeah, unless the underlying network is not up to par So at least 10 Gigaby, that's the minimum. And I would suggest 40 Gigabit.
00:10:00
Damir Simpovic: For most providers big. I mean they could I use use hardware in this case Yeah I don't buy a thousand dollar Nick. I can I cannot afford it.
Artur Troian: Yeah, let me just. Yeah, this question just shows that the You need to put. A lot of description into the storage classes, but the first of all, networking speed, as ingress does not pretty much correlate to the speed of the storage. and I want to point a couple examples with that so you might have
Artur Troian: For the dots, a lot of processing and writing to the geese and have as little as just, you know, maybe of the traffic and It and the same as well. It is opposite side. You can have a workload that needs you know thank you bits of bandwidth but it doesn't do pretty much a lot of writing to the storage. So what I'm trying to say is I still think we need to have the storage passes because they show not only.
Artur Troian: Provider on what they can provide, right? Because removing that that simply means someone can say Hey I have this storage class, let's say, just call the storage and it just being rising that and most will actually what the spirit is. So that's that's the first part of the story. The second part of the story. Violence have doesn't necessarily be.
Scott Carruthers: and,
Artur Troian: It has to be network, right? You can have one rank one unit, Let's say four units in the rack that has like 40 and gaming drives in that and that that like is two huge epic CPUs. Couple turbines from And you're gonna have a really pleasantly fast persistent storage. There's no network at all. So, that's what I'm just trying to show is There, a huge amount of cases. And we cannot limit the case to the resistance storage solidly on the ingress network speed. Yeah. Well
Damir Simpovic: Sorry, I'll just jump in. I wasn't referring to the ingress network speed, I was referring to the interconnection. So the The internal. Network speed, you cannot.
Damir Simpovic: The limit of your network.
Artur Troian: Yeah, but once again, what I'm trying to show is it doesn't necessarily mean you have internal network for your provider. You may have provider, this is just one single unit and…
Damir Simpovic: Yes, of course, definitely.
Artur Troian: there's no matter and you can also have like, I mean easily right now. You can have me. As long to beats per second, right? And then you can go easily aggregation, is the full of them and you simply get programming per second. That's actually what the centers do. I mean, we're not saying we're gonna interpret those cases, But we are saying that the networking speed is one subject and it should not be considered as a requirement for the persistent storage, right? It's just the boundary that can be applied, so close, that's
Damir Simpovic: well yes, definitely the this the stories of course broader than than just, What we have touched here? I definitely agree.
Damir Simpovic: I'm just saying this from the from, for instance, from my point of view.
Damir Simpovic: I mean, the biggest reason why I actually started this is that I've noticed there's practically zero leases using Beta 3. I personally don't have anything against storage classes.
Damir Simpovic: I actually, I welcome them. personally, but there is what I'm trying to say is that the most of our users will just use the default values. And currently. Correct me if I'm wrong, but in awesome Akash or cloud, most or wherever you can find a generic SDL for your deployment. You will find Beta 2. That's what I'm saying. So, Yeah.
00:15:00
Artur Troian: Yeah, that's good.
Adam Bozanich: it's like,…
Artur Troian: Oh yeah.
Adam Bozanich: Yeah, let me just get some thoughts out of real quick here. So,
Adam Bozanich: let's see, first of all, you think on the default the issue of defaults? They maybe not expect, you know, basically splitting up the the possible number of the possible providers, right? If we haven't defaults of beta 2, There's a bunch of beta three's people that don't care. Who won't see this beta 3s. So I think we can change the defaults to be one of, right? Instead of Just beta 2. We can have it. Just be all of them later on beta 2 Beta 32. I have to so that that might be something we could do, you know, we should consider that with kind of everything, really, but I think that we archer trick me from around but isn't that it's possible to express that in the current Yeah.
Artur Troian: Yeah, so the default is, I know, I totally agree that default is meaning of that is controversial. however, like initial idea what you've done is that we said like, Hey default doesn't mean beta 1 or beta 3, The fold must be declared explicitly, while it wider. So, again, a couple examples, So let's say we have a provider recently me, right? And they won't and declare it as a bite of three, they may not declare the fall it as the default and that simply means if someone said meets world that has the full storage class, this provider will not be those claws, right? There might be another provider that has beta 2 and Beta 3, for whatever reason, and they declare Beta 2 as a default. So treat the full as an alias.
Artur Troian: And what that area says for the provider. I don't care what this workload does. If workload owner doesn't care, either, right? So I'm happy to declare my beta 2 As the fault as well, right? So you cannot have this default class and don't have waited one way to work. Three. So the default is just an alias that applicable to particular provider, we should probably put more description and explanation on that subject. That.
Adam Bozanich: So that providers say, like, for people that don't care, I want to offer them services for people that do care. I have beta 3.
Artur Troian: yeah, and we have actually those people, I give you just one particular example is osmosis right now, so if you're on RPC you know As possible on a car. The true part like is around 100 megabits per second, that needs to be written in storage right now. Imagine, so having beta 3, it doesn't necessarily mean that this entire throughput and latency. I hope gonna be occupied by one report, right? So we want to have to use this storage class by other reports. Otherwise just doesn't make sense and
Artur Troian: So for example, if I run osmosis on provider, the provider doesn't want that. My report just simply occupies entire by the storage plus and we cannot pretty much anyone cannot deploy anymore. And once again the the storage classes are more commitment of the provider,…
Adam Bozanich: You.
Artur Troian: right? No, the requirements for workloads, that's why the food the full thing and the fourth class is the full for every claw that other pointed out. That's just they don't care if you want to run website or some small thing, you don't care. Kind of speeds you need on the story side. Why you should bother if you know what you're doing on that and that same applies from if you do yes, right? Great, your instance, you select what storage sizes? You want storage speeds, you want along the Iops. So that's I On the subject of making it more. You know, clearly described we haven't just recommendation and make sure that the limitation covers also all the aspects
Adam Bozanich: Yeah, I think it'd be worth spending some time to document. I will address them like these concerns that Damir brought up because the totally legitimate. You know,…
Artur Troian: Yeah. Yeah.
Adam Bozanich: you don't go to the hassle of setting up like, you know, high-end A high-end provider and then have everybody not even see that you exist. You know that's up over. That's kind of not what we want to happen. second point I second thing I thought I was you guys were having this discussion is that like so also like the Declaration of hardware that a provider has is like Like, it's something. I feel like we kind of have to do. We have to tell people? Yes, SSD, this is used. What have you? But yes, it
00:20:00
Adam Bozanich: Depend like actual performance. Depends on so many factors that it ends up kind of just being a rabbit hole trying to like, Supply enough information such that people can infer. What the performance would be. So, like, you know, I, I was strong opponent of like using benchmarking instead of not it's not totally instead of, but spending more time on like having you know, benchmark, benchmarking results. over like going deeper and deeper into specifying what the actual hardware is because at the end of the day, what people care about is what the performance is Third thing, just real quick. I'll say I'm doing here before it showed up. You mentioned that you buy.
Adam Bozanich: Used hardware. It'd be great if we could like start documenting, how people can do that. So that I mean maybe it's your secret sauce and, you know, want to give it up like, that's fine too. But you know, I think that it it is something that maybe it's like providers can oversee. This is that we start documenting how people can build providers, basically. And,…
Damir Simpovic: Sure.
Adam Bozanich: you know, and what options they have for getting, you know, hardware, you know.
Damir Simpovic: I actually, Sorry. Yeah, I actually did.
Adam Bozanich: Yeah.
Adam Bozanich: Okay.
Damir Simpovic: In my YouTube videos, I did show what hardware I was using,…
Adam Bozanich: Oh awesome.
Damir Simpovic: and most of it was used. not all of it but most of it is used and…
Adam Bozanich: Good.
Damir Simpovic: but yeah yeah sure why not I might make another video and explain my process how I do things and How I get things done for well,…
Adam Bozanich: Awesome. Okay. Cool.
Damir Simpovic: maybe not cheap, but for a reasonable price. And yeah, the performance is, you know, blows everything out of the water. Sure, definitely.
Adam Bozanich: I'll be awesome.
Damir Simpovic: Especially the network stuff I think that's really, it's a shame because the, for instance, the 40 Gigabit mix are like 50 bucks. You know and two ports so you basically get two 40 Gigabit ports for $50. I mean that's I mean I'm in my opinion,…
Adam Bozanich: Mmm. Wow.
Damir Simpovic: everyone can afford that the more expensive
Adam Bozanich: Yeah.
Damir Simpovic: And I I even I have another YouTube channel that's for fun. That I've explained how to make a, you know, a switch a network switch with you…
Adam Bozanich: The Nixon Mix.
Damir Simpovic: you don't
Damir Simpovic: yeah, you can you can basically Create Your Own Network, Switch with Four Ports. If you need four ports, you don't have to go ahead and buy a 2000 or $3,000 switch. You can just go ahead and buy three nicks and you know, but anyway well we were the digressing. I just wanted to comment, Sorry on this what you said, I also think that some kind of benchmarking would be great. there's another reason, one of those is that the availability of the hardware for upgrades Is not always there.
Damir Simpovic: So, we cannot if you specify for instance I have specified that I'm using APEX the AMD epic processors for instance, but that's a very broad term. A new generation comes out, and I buy a new generation CPU and I spray it into my existing cluster. And, you know, who knows which node will get deployment will actually get assigned to, you know? So that's also not consistent. So,
Adam Bozanich: Right. Yeah.
Adam Bozanich: No, thank you.
Andrey Arapov: Hey, I just want to comment about this a bit one. Beta, 2 Beta 3. Actually was thinking, like, it looks more to me, like a hotel rating system. Like, you know, have like one star to start three stars, why? Because is a generally, it's hard to promise, exactly. The iops people gonna have because you may have tons of workloads that gonna write gonna be writing on the discs.
00:25:00
Andrey Arapov: And it will be always impacting. So we cannot like really give a good promise that. Hey, that's the minimum. I hope you're gonna get. Of course, we can embroider can have some tests run, lots of workloads and see what the real minimum. It can give if like every single be writing to the disk but it's another thing and maybe you should even add like, you know not bitter but more like a rating. Like in star-storms like one means for sure you do something because they're really slow data to means like SSD without this precise. Loads. Oh, version 4. I mean, like just basically this is these result in me connection. A bit of three would be like in DME maybe a bit of four like four stars. Let's say, yeah, we can have like up to five classes would mean that provider has done lots of testing and has very good underlying systems. Like in terms of networking and stuff, work for data for and for a bit of five, that would mean that Prov.
Andrey Arapov: Past latest generation, Adapters, Generation 4, and the best networking. So something like that, what they think.
Adam Bozanich: That's it kind of goes down this, this concept. It gets into this concern. I have like
Adam Bozanich: f****** knows, you know, like That it really, I think. It's cool to. I mean, I think we need to keep both options open. I do think that there's like, like limited like this is a decreasing return on time. Investment, as you start going down, like, Getting super, super granular with how you specify what hardware you're running. I mean so that's just my thoughts on it. It's open, you know, to basically
Adam Bozanich: You know that's just my just not just might be you know if people make they're good arguments for having more specificity than great.
Adam Bozanich: I think that we have a prefer to see like real demand for that, not people just wanting to know but like actually how I didn't like people wanted to know because it impacts their purchasing decisions.
Adam Bozanich: and also also say that the way that we describe, Um, Characteristics of a provider. You know, it's kind of like a V1 thing, right? It's like a bucket of attributes. That doesn't lend itself well to like heterogeneous environments, for instance, like, Oh yeah, like to me was just saying You have, you know, you have AMD epic and that you upgrade to the new epic. And now half your clusters, writing one, half your clusters, running the other, How do you really describe that? And even if you do describe that and people choose like, Hey, I want this guy because he has, you know, super epic.
Adam Bozanich: There are no mechanisms in place to ensure that that workload runs on the super epic half of the cluster. so you know there's there's some issues with like heterogeneous environment so we need to deal with This, there's some issues with like, how we
Adam Bozanich: Express what the cluster is made of. I think that could be a lot. That could be more organized and easier to query. That's one another. One of the reasons why I like let's just get some benchmarks out. The star system is great. I think if we quickly, if we standardize like a bench like a star system for you know, I was hoping that
Adam Bozanich: the network auditors would kind of start. sorry using a star system, so like People could just make decisions they so on top of that. So, like, you know, network auditor, like, Hey, ends up becoming quite prominent, they, they do benchmarking and all this stuff for the providers that they, that they can monitor, and they issue a star rating on for those for this providers, maybe on and a number of different metrics.
Deval Patel: Yeah.
Andrey Arapov: Yeah, or just based on the auditing key like five keys, and five stars. And if like some providers you got audited by like this specific emails like it's five star and…
00:30:00
Adam Bozanich: Yeah.
Andrey Arapov: we all can always like removal red this stuff based on the feedback on the users.
Adam Bozanich: Yeah, one last thing I would say about the specifying hardware and the is that people to say, whatever the hell they want, and that's another reason. Why actual real benchmarks from like a third party, like, the network auditor is a little bit more valuable than just like reading off the Attributes from a provider. Deval, did you have something to say?
Deval Patel: yeah, like when we met last time in Colombia, there was a front project was wanted to like I mean, Yeah, Pause wanted to integrate with which was observatory who used to do observatory of validators. Like, they used to provide a validators writings, right out of hundred based on the location based on the speed, based on the like, you know, other attributes. So, we can do something like that, as well. We can integrate them. Or, you know, find out like what we can do in terms of time.
Adam Bozanich: Yeah, totally. I don't remember recall the exact the project. In particular, but Yes, absolutely. Yes.
Andrey Arapov: It's a,…
Deval Patel: I think it's Observatory God's own.
Andrey Arapov: it's also.
Deval Patel: I think it's observatory dots on. I'll share the link here.
Adam Bozanich: Cool.
Andrey Arapov: I was thinking like, you know, we are like airbnb of compute. So why not have our hotel rating system and stars? It's really cool.
Adam Bozanich: Totally. Yeah. No. Absolutely So in other issue with the stars, this has been something that we kind of been talking about for a couple years now. But Another issue with stars and the way the attributes work right now is that you can't really express like, Hey I want three and above stars when you're doing an appointment, like we we need some, we need to be the kind of the querying to be more expressive.
Deval Patel: and,
Adam Bozanich: So right now, if you want, you know, three, four or five stars, I think that you can hack it together,…
Andrey Arapov: Right.
Adam Bozanich: such that you can get, three, four, or five, but you have to like, write it all out. And I don't even think we have any examples of that because like a random it's kind of like a hack but
Andrey Arapov: Yeah, we should think a little bit on this like, because I was thinking, like, maybe 0.5 for something good. Like for say, let's say you have storage the better, it is the more power you can have just for the storage and say up to five and…
Adam Bozanich: Yeah.
Andrey Arapov: the same for memory, the same for CPU quality and network. And then you can like, you know the minimum denomination is like 0.5 and then you can combine all of them. And yeah it's good point to make sure that nobody can hack out. I think it's okay. Like if somebody puts five parts to their network and they have like this for sure like one star and think that's, you know, and then we can have a composite of this whole stars and the final one will be like either three or four or five something like that.
Artur Troian: It's important not to include cleaning fee. Price of deployment of thing to the attributes. So it is happens that You know, he really great example is the CPU, right? So how the right, so, right now treated as No matter what kind of the CPU is. It is just set by default and there is no way for tenant to choose. Right? What the CPU if you have, like, multirac, multiple dispute types within the provider?
Artur Troian: They pretty much all set to default, but yeah, you're thinking on, you know, extending for example CPU attributes, same way as the precision storage. So they're going to be ability for the tenant to specify, You know? Yeah, I want to have and leave it to the different levels right now, or I want to have Awx 512 included, right? So this, not every CPU has it. So that's it, yet another story. And actually on the system storage, this default setting shows really good use case. It is, unfortunately probably, it's not yet. Well, Described and shown. So it once we have same for other for the CPU,
00:35:00
Artur Troian: And, you know, providers, you'll be able to specify what CPU types they have after social details then. They can mark those CPUs as the fault as well. Why not? However, they may not to do so and then simply means that provider. So returns that want to deploy just to put that don't care what you type is that Does it have Abx 512 or even like, you know, some other things or new agents?
Artur Troian: They just deploy and they just do normal claws. If you want some specifics, you don't have to filter it in to the specific things you want to have and they're actually same applies to networking. I mean, the bandwidth channels. So right now it's complete and there's no guarantee on the you know how much traffic can be allocated for each department force. There will be also said things related to the network. and yes, so as I say that there should be grades of it's what they have qualities on feature is it's basically saying how capable this feature is within this provider.
Artur Troian: And default is capability. I just means like whatever you want. Um, whatever. Sorry I have and great. Like, wait a long, wait a little bit three. They just say, Hey, yeah, this this storage class is actually guaranteed to have Um, you know, certain speeds, certainly iops certain storage amount, and other things, and you mentioned that the networking speeds are limited. This should not be a concern for the end user and intended that how the storage actually deals on the provider side. So we provider has
Artur Troian: Only NBA news storage, it has multiple racks that they all connected through networking basically using anemia or fiber. For some reason that storage is really slow. They should not declare it actually, as a, like a three for a number of reasons and that's where actually our little coming in. So just like, Hey, yeah, validated that. It's not able to do the Beta 3 class. It has to be by three Beta 2 and I'm going on assignment one is better too. Pretty much yeah.
Adam Bozanich: It and…
Artur Troian: Oh yeah.
Adam Bozanich: I don't think it's anybody. So like I think there's something there like, we're all using this thing. We're talking about it, but we don't even without needs, right? Like, I think like the NBA Me SSD. In and spinning disc. But it's something but I think we should consider renaming the what with the storage classes are? I don't know.
Artur Troian: Yeah. That's like we had been sort of confusion and while the reasons we chose to name it this way for the insiders testing and he just didn't remind it but we Yeah.
Adam Bozanich: Hours into the fire drill, and we never forget.
Artur Troian: And so the biggest problem was the staff itself, right? So let's say, you know, we have the storage spin up on the provider and there's one spinning disks, right? But then if the spinning disks are in a raid mode and that pretty much Michael corresponds in speed as an SSD. So we decide like, okay, how let's leave it and think like maybe we should describe storage class based on the, you know, certain. Abilities, like a speed.
Adam Bozanich: It's a really? Yeah, it's a hard. It's a hard problem again. It's this problem is specificity even like You…
Artur Troian: Yeah.
Adam Bozanich: for, for people approaching, this is the, Oh gosh, right? Like, Yeah, I want to use, you know, they know they're so much technical at the very least and like I won't use, you know, fast. I'm running a note, I need to use best storage. And like, okay, well here's your options for storage better one. Better too, baby, three, you know, it's like it's an approachable and then Even if we try to explain it, we're like Well there's this I could be a really, it could be this or that. So you know, maybe it should be
00:40:00
Adam Bozanich: metric like a benchmark based or maybe we just say, Hey, this is either speedy disk SD or nvme And yes, there are more things that impact actual real world Io. Like to me, I was talking about now work or this, there's a raid all that stuff but Yeah,…
Artur Troian: and,
Adam Bozanich: the thing that your data, your bytes are getting written to, is it either on a spinning disk because they're not SSD or is it about Mdme? You know, I don't know.
Artur Troian: Yeah, that's that's actually a very important to that.
Scott Carruthers: Take care.
Artur Troian: Um, I think one of the, what, for more, so in the, you know, different clouds providers, none of that. None of them do that on the virtual instances. They specified disks, If you for example, you know, please Burma hardware. However, what I noticed is, what they all do is. Yeah. You they pretty much name on the metric based And we could say, Just, I'm just right now to, you know, give an idea and then think about that impurity. So we named the storage classes within the window of Iops. So let's say, you know,
Artur Troian: Are your 1000 just and pretending, right? So, that's simply means that the this provider currency is iops up to 1000 on this storage class, right? And then we have, you know, why your 1 million, or whatever. Just once again I'm just looking on that but I totally agree. Yeah, that bite and I mean is that was pretty. I was truly. Testing phase and we do that moment,…
Adam Bozanich: And then it gets released.
Artur Troian: you know.
Adam Bozanich: I know I get it. It's really still wild. This is…
Artur Troian: Yeah.
Adam Bozanich: what we live. That's part of running at real world system. It's all good. That's Happened. I think it's something that this thing should consider is like At the very, you know, just for usability, just maybe we look. Are we creating standards for for attributes? I think Anil is working on that document and maybe that can be part of it.
Adam Bozanich: Scott, did you have something you want to mention related to this?
Scott Carruthers: Last week, I'm an entirely sure what this was referencing my I believe this was just referencing. Documentation updates surrounding provider builds. So I'm just going to paste into the Chatham and Andy. If you have anything to, I believe this is a exhaustive list, but just so that everyone in the SIG is aware. Not any real dramatic, documentation changes. But as always, our documentation is evolving. So Andy and Andy basically authored, and then I massage them into the, the documentation over the last We can half, we've added documentation around provider rotation.
Scott Carruthers: We've expiring Certs on a provider and ensuring, that users, have clear instructions, and those regards, also updated the metal will be docs for IP leases. So metal will be as shifted from config map. You usage to crds and custom controllers. So we have a migration from the legacy config maps to um to the new Crds and also instructions. If someone's building a new just to use the Crds and then also added in this kind of a provider sending a check at the end of a provider build. Some SCD database health checks So I think that's what that, that agenda item was referencing. Just the documentation updates. Any questions on? Those documentation matters.
Scott Carruthers: Okay, great. The next item on the agenda was I think we just want to give some air time for the Prater team. So, Jigarh, and devalum, anything that you wanted to discuss updates on on your side, and I think you may also be presenting during the steering community. The steering committee for for funding. So I guess just general updates. And if you want to talk about that at all.
00:45:00
Jigar Patel: Yes, yes, it can hear me right? Awesome. So yeah,…
Scott Carruthers: Yes.
Jigar Patel: from the last meeting, we raise stylish issue on the restart happening. Like we have to restart our providers. So, and at that time, Delicious was dropping, right? So, basically we did a workaround. So now we are waiting for CID to come up and then we to restart our providers. And also there was an issue where we So our upgrade path from like credit provider, if they are running, let's say 0.8 in our cars version. So before we introduce the provided service. So there was some providers that that's running the older version. So we give them part to upgrade now. So they can go to the latest providers and like 0.2.1.
Jigar Patel: And we also work on PR days for management and moderation API. So basically will be presenting tomorrow or to stay in community. Like the, the work that of your proposed to do on those two more Jason, and Management APIs. And we also have some updates for you guys for tomorrow.
Scott Carruthers: All…
Adam Bozanich: That's awesome.
Scott Carruthers: Yeah, yeah,…
Scott Carruthers: awesome. Updates you there.
Adam Bozanich: Yeah, cool.
Scott Carruthers: Any any questions or thoughts for the Prater team?
Adam Bozanich: I think I can think of, you know,
Scott Carruthers: Okay. All right. Yeah, look forward to your presentation's tomorrow and Jigar in the Prater team. So yeah, I've no questions on the from the prayer team. The our last item on the agenda was just some open discussions and we're probably just doing somewhat someone naturally but any Topics that we'd like to cover as a group that haven't been discussed yet.
Adam Bozanich: but I'll just mention real quick because I just saw this PR, but Luna has some PRS open for server manager. I don't know if we talked about this last time or not. But it's great. It's really cool. I think that it looks like it uses one search for the entire cluster. Possibly with this star things all the host names can be SSL you know go over HTTPS. Super cool. And
Adam Bozanich: Yeah, I don't know, I'm excited about it, you know, I think we can, we can get do. This is like a milestone and then you know in the future possibly at support for her deployment certificates. That could be the next step.
Adam Bozanich: It's super cool. So, thank you Luna for doing that work.
Adam Bozanich: That's it for me.
Scott Carruthers: Okay, I thought I saw someone so yeah, that awesome updates. Luna is very exciting. I thought I also saw someone have their end up. Was it devalt? Did you have a?
Scott Carruthers: Question on your hand up. Oh, Maybe it was a thumbs up. Okay. Anybody else in the room? Have any Thoughts or topics.
Scott Carruthers: All right, sounds good. Well Yeah if there's there's nothing else for open discussion. I'll go ahead and and the recording. And I think that's a, yeah. Yeah.
Adam Bozanich: So, thank you.
Scott Carruthers: Awesome discussion on the, on the storage classes. And another items. Yeah, I think. And that's it for this week.
Scott Carruthers: Okay. Thanks everyone.
Jigar Patel: Right. Thanks guys.
Andrey Arapov: Thank you.
Damir Simpovic: Thank you.
Deval Patel: Thank.
Meeting ended after 00:49:08 👋