-
Notifications
You must be signed in to change notification settings - Fork 76
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pure api based truenas driver (ssh dependency removed) #101
Comments
Hey, |
Yeah you don’t even need to use another namespace. Just a new helm release name and unique storage class names etc |
In order to test this out use the existing helm chart with values to mimic nearly exactly the existing freenas drivers with a few changes:
Remove the Ensure
|
First test result : Test environment:
Deployment Status:
In order to avoid an extremely long post, here are the files with more information. (values, description, logs) I guess you have tested the deployment, so I must have something wrong in my value file. |
I forgot to mention the most important detail :( the |
And it may not work on the beta, it might need nightlies…a lot of collaborative work has been ongoing and I don’t think it all landed in the beta. |
Indeed it works way better with this modification ! And the results of testing : Test NFS (quota enabled, reservation disabled):create pvc: ok Test NFS (quota enabled, reservation enabled):create pvc: ok
delete pvc: ok Test NFS (quota disabled, reservation enabled):create pvc: ok
Is it the responsibility of the csi driver to start and enable the service when it creates the first share? same for stopping and disabling the service when deleting the last share? I'll check out the iscsi and smb drivers later. |
They are not manually exclusive. That’s a good find that I’ll need to ask the ix team how it’s handled on the backend. I think that combination works fine in the ssh scenario but I’d have to double check that. Thanks for the feedback! |
@travisghansen FYI, this does not appear to work in the latest TrueNAS CORE (12.0 U4.1) because the
|
Thanks for testing! That’s expected atm unfortunately. This is currently targeted at TrueNAS scale (we’re looking to make it more official for the 21.08 beta release) but I’m fairly confident all the middleware updates will be pulled into core at some point. |
Thanks @travisghansen, makes sense, looking forward to breaking the SSH dependency 😄. |
I have reproduced the expand volume error and submitted an issue with ix. Things are looking really good to have this all ready in time to release with the scale beta release. |
OK, expand volume bug has been fixed in latest nightlies and several other things buttoned up. I'm getting ready to snap a massive new release so any testing that you can do beforehand would be great. Make sure you have the latest nightly of scale and that your cluster has the most recent |
Also note there are fixes in place for the nfs server issue you mentioned (not starting when no shares) and various other things. 21.08 + |
Ok, I’ve released |
Known issued that were NOT included in 21.08: https://jira.ixsystems.com/browse/NAS-111870 In particular, if the system boots without any targets/luns clients will not be able to connect. A workaround is to |
I just ran into the "create_ancestors" issue on TrueNAS Core 12.0-U5.1. Hope it's fixed soon. Not looking to moving to SCALE any time soon but I can set up a virtual test instance if needed to move this forward. |
@mister2d thanks for trying it out! There are no definitive plans yet for pulling all the new api calls back into CORE, but I'm guessing they won't land until 13 :( In the mean time I have have 0 intention of removing the ssh-based drivers ever. I would love to have additional eyes on it! I'm currently working towards passing the full suite of csi compliance test (don't be alarmed, anything that's currently failing are generally cosmetic-level issues). I'm very close and actually working on the final failing test right this minute. After I have that fully passing I'm working with iX to provision some resources to run the full test suite across several different configurations and I hope to add 13 to that as soon as possible. ATM the test suite running again 12 CORE pretty much fails everything (with iscsi anyway) because I can't make the tests run without using datasets that don't put me over the 63 char limit for zvol names. In BSD 13 that limit has been bumped to 255 so it will make running the test suite there more feasibly anyway. |
There's always sadness when projects move on from BSD. Anyway, I'll be moving my ZFS pools over to SCALE by this weekend. Is there anything specifically that needs testing on CORE? |
Not as it relates to this ticket no. The api-only drivers are currently limited in scope to SCALE only (21.08+). When CORE 13 hits I'm relatively confident it will support all the endpoints etc so until then, SCALE it is :) I will be snapping a new release shortly with all the 'fixes' to make the drivers fully compliant with the csi test suite. |
@travisghansen Any plans on SCALE integration with iX? |
@Ornias1993 there's nothing really to setup on the NAS itself, it uses the built-in api entirely. What's the use-case you have in mind? |
I was refering to integration with the SCALE kubernetes system. Unless iX told you they are going to do so (which I don't expect), i'm open to adding it as an app :) |
Is the use case for a cluster of scale nodes trying to use storage from only 1 of the nodes? Or a single node cluster trying to access storage from the server hosting the cluster? or something else? |
Currently SCALE does not support kubernetes clustering at all. So we did get requests if we could help out in finding a solution where people have 1 compute and 1 storage node ,and some requests for having SCALE connect to core stored files. So the usecase would, basically, be any case a normal k8s user would utilise democratic-CSI. (as it's basically stock k8s what this is concerned) |
Ok yeah, then just wrapping the chart sounds ideal. I’m not aware of any work to include it atm in the upstream repo but I can ask if you’d like to prevent double work! |
@travisghansen Awesome, in that case i'll safe you the trouble and throw Kris a bone right away myself :) |
And indeed a "wrap" like we did with metallb should be enough here as well and give the most stability guarantees 👍 |
@Ornias1993 @travisghansen just checking -- did this ever get a truecharts wrapper? Couldn't find it in the |
Appears it hasn’t been done yet no :( |
Okey, if we go all "lets delete comments because I don't like them". The simple fact is that if something isn't showing in the catalog (or the website), it isn't in it. It's not because I want to be rude, but because that's simply how catalogs work. |
You are welcome to not include it in your catalog if you do not wish it to be there. I’m not interested in your negativity (and now childish threats about not adding it to the catalog) in this space however (if I didn’t have experience with the toxic behavior previously I would have never deleted the comment, but I have seen it in other places and know your history and I won’t allow it here). So add it or don’t, but don’t come here with ridiculous comments to users sincerely seeking answers. There are far better ways to relay the message you wished to relay, I would ask you to consider those ‘better ways’ in further communication in this space. |
That was never what I wrote or intended, I actually did plan to work on it, as you're well aware. But sadly enough I don't have endless amounts of time. But I think you very well understand that, considering you're maintaining this repo and are sometimes hitting the same issue.
I'm not making a threat, i'm stating the obvious fact that I need to prioritise my work. This includes that communities I don't feel like I have "synergy" with personally, obviously get less development time.
I don't think the answer is resorting to deletions and/or personal attacks. A friendly message with "I cannot allow that, etcetc. please make it sound a bit nicer" would've sufficed. If you don't want harsh direct comments, that is 100% your call as maintainer.
They are not seeking answers in this case, that was the whole point I tried to make. The whole point about a catalog is that if it's not in the catalog it's not in the catalog. If you open the fisherprice catalog looking for a blue-toycar and it's not there, it's not there. You can ask random other communities on the internet if the blue car is there, but that doesn't change the fact "it's not there if it's not there".
You are in your right to request that and It's my responsibility to stick to the limits within your community. i'll leave it at this, as i'm not likely to spend time on this due to the way priorities within TrueCharts have shifted. We basically already have 2022 completely layed out by now and supporting external PVC-backed storage is not really something we actively aim towards for BlueFin. (mostly because BlueFin itself already will have quite some PVC and storage related changes when it comes to kubernetes) |
Now that TrueNAS Core 13 is out does that mean this CSI driver should work without ssh? |
Unfortunately not :( I would suggest opening a ticket on the ix jira issue tracker to request support in future releases. |
When trying to connect to TrueNAS Core (13.0-U2) i still get an error: GRPC error: rpc error: code = FailedPrecondition desc = driver is only availalbe with TrueNAS SCALE with API Key and SSH :-( |
The iX team has no immediate plans to add support for CORE :( |
Oh, thank you for your Feedback. Then the iX team should update your website |
I think there may be some confusion. You can use this project with CORE just fine, but you need to use the ssh-based driver vs the api-only driver. Maybe open up another issue for help getting it going with ssh and we’ll get it working for you :) |
thank you, i try ssh connection |
Are there any examples for the api-only driver for iSCSI and Helm? I'm trying to test this with SCALE 22.12 but Helm is rejecting my attempt to install the driver, presumably because of a syntax error on my part! This is what I have come up with so far, as a hybrid between the community examples for the ssh driver, and the NFS example provided in this issue.
When I attempt to execute that, this is what I get:
|
What you have is pretty good but the scope of the driver config is off as it relates to a helm values file. Something like this..
|
I’m working on a purely api based driver for truenas (currently requires latest/greatest scale). Any takers to do some early testing?
SCALE 21.08 epic: https://jira.ixsystems.com/browse/NAS-110637
SCALE 21.10 epic: https://jira.ixsystems.com/browse/NAS-111870
The text was updated successfully, but these errors were encountered: