New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

single to raid1 pool resize not reflected in Web-UI #1406

Closed
phillxnet opened this Issue Jul 23, 2016 · 8 comments

Comments

Projects
None yet
3 participants
@phillxnet
Member

phillxnet commented Jul 23, 2016

Thanks to @grebnek in the following forum thread for reporting this bug.
https://forum.rockstor.com/t/finally-build-my-own-nas/1811

Steps to reproduce (Rockstor version 3.8-14.07):-

Had a single disk pool with single raid level.

Resize Pool - Add Disk - Yes to change RAID level - selected raid1 from drop down and ticked additional disk - Next

Waited until balance complete.

Pools - rock-pool - indicates "Raid Configuration: single"
as does the pools column in the Pools page overview.

Expected result = indication of raid1 for this pool.

Confirmation of correct balance from single to raid1:

btrfs fi df /mnt2/rock-pool/
Data, RAID1: total=112.00GiB, used=110.03GiB
Data, single: total=1.00GiB, used=0.00B
System, RAID1: total=32.00MiB, used=48.00KiB
Metadata, RAID1: total=1.00GiB, used=390.86MiB
GlobalReserve, single: total=144.00MiB, used=0.00B

This replicates @grebnek findings.

@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Jul 23, 2016

Member

Please update the indicated forum thread with significant development on this issue.

Member

phillxnet commented Jul 23, 2016

Please update the indicated forum thread with significant development on this issue.

@grebnek

This comment has been minimized.

Show comment
Hide comment
@grebnek

grebnek Jul 23, 2016

Contributor

Maybe I can share the results of my research with you, to help a little bit.
The Problem seems to be the second Data block in the output of btrfs fi df /mnt2/pool.

The function pool_raid(mnt_pt) in fs/btrfs.py iterates over this output
and only use the last value of the data block.

These single data chunks seem to be normal, according to btrfs FAQ and I could get rid of them with another balance. I still have to try it. see "Why do I have "single" chunks in my RAID filesystem?" in FAQ

Contributor

grebnek commented Jul 23, 2016

Maybe I can share the results of my research with you, to help a little bit.
The Problem seems to be the second Data block in the output of btrfs fi df /mnt2/pool.

The function pool_raid(mnt_pt) in fs/btrfs.py iterates over this output
and only use the last value of the data block.

These single data chunks seem to be normal, according to btrfs FAQ and I could get rid of them with another balance. I still have to try it. see "Why do I have "single" chunks in my RAID filesystem?" in FAQ

@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Jul 23, 2016

Member

@grebnek Well done. I was about to try another balance here on the system I setup to replicate your findings so I will report back shortly. My current task is actually to take a closer look at all the functions within btrfs.py so your report is actually very timely. I'll post here if it worked for me, obviously we need to parse better and I will keep this in mind when I get to that function in the unit testing I am ear marked to do there.

Member

phillxnet commented Jul 23, 2016

@grebnek Well done. I was about to try another balance here on the system I setup to replicate your findings so I will report back shortly. My current task is actually to take a closer look at all the functions within btrfs.py so your report is actually very timely. I'll post here if it worked for me, obviously we need to parse better and I will keep this in mind when I get to that function in the unit testing I am ear marked to do there.

@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Jul 23, 2016

Member

@grebnek while I await the outcome of the re-balance via Web-UI do please consider submitting a pr to address this if you fancy. My next task is predominantly unit testing but this fix can always slip in under that easily enough. As you say these single lines can creep in but we simply haven't had the single to other raid level as an option until quite recently so it's quite fortunate that you drive purchasing habits coincided with this added feature (71% left of my test system run so far)

Nice find.

Member

phillxnet commented Jul 23, 2016

@grebnek while I await the outcome of the re-balance via Web-UI do please consider submitting a pr to address this if you fancy. My next task is predominantly unit testing but this fix can always slip in under that easily enough. As you say these single lines can creep in but we simply haven't had the single to other raid level as an option until quite recently so it's quite fortunate that you drive purchasing habits coincided with this added feature (71% left of my test system run so far)

Nice find.

@grebnek

This comment has been minimized.

Show comment
Hide comment
@grebnek

grebnek Jul 23, 2016

Contributor

Good to see that my accident helps to improve Rockstor.
Because I have all my data still save on the old HDs, I could not resist an started another raid change from single to raid1. After the balancing was done the use less data chunks in single raid were gone and also the WebUI show raid1 correctly.

Im not were familiar with github and working on such big project like this, but maybe this is a good time to start. I will try my best.

Contributor

grebnek commented Jul 23, 2016

Good to see that my accident helps to improve Rockstor.
Because I have all my data still save on the old HDs, I could not resist an started another raid change from single to raid1. After the balancing was done the use less data chunks in single raid were gone and also the WebUI show raid1 correctly.

Im not were familiar with github and working on such big project like this, but maybe this is a good time to start. I will try my best.

@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Jul 23, 2016

Member

@grebnek Great, go for it. As is you have narrowed the problem down to a single function so really all you need do is replicate the issue in a vm and set about improving the parser to be wise some how to these single lines. Easier said than done but still doable, just make sure to test on when they arn't there as well of course.

There is some pretty approachable documentation on contributing to Rockstor within the official documentation:
http://rockstor.com/docs/
and more specifically the section on Contributing to Rockstor - Overview and the developers section within that. Covers setting up the local git to github links and everything. Including the naming of branches to match the issue they reference (this on in that case).
http://rockstor.com/docs/contribute.html

You can of course always ask stuff on the forum where there are more eyes.
For the time being I will leave this issue myself for your attention as now that you have identified it and proved a fix everyone is in a better position already. It's kind of your right to at least have a stab at it. @schakrava is project lead and we have several other active contributors all of whom are on the forum as well. Plus you are now the most knowledgeable on this particular issue. Shining start I say.

Thanks for your contributions thus far and I look forward to your first pull request, no stress as this is, in my experience, a very relaxed and capable team (bar me of course as I just have to wing it from day to day) :) .

Member

phillxnet commented Jul 23, 2016

@grebnek Great, go for it. As is you have narrowed the problem down to a single function so really all you need do is replicate the issue in a vm and set about improving the parser to be wise some how to these single lines. Easier said than done but still doable, just make sure to test on when they arn't there as well of course.

There is some pretty approachable documentation on contributing to Rockstor within the official documentation:
http://rockstor.com/docs/
and more specifically the section on Contributing to Rockstor - Overview and the developers section within that. Covers setting up the local git to github links and everything. Including the naming of branches to match the issue they reference (this on in that case).
http://rockstor.com/docs/contribute.html

You can of course always ask stuff on the forum where there are more eyes.
For the time being I will leave this issue myself for your attention as now that you have identified it and proved a fix everyone is in a better position already. It's kind of your right to at least have a stab at it. @schakrava is project lead and we have several other active contributors all of whom are on the forum as well. Plus you are now the most knowledgeable on this particular issue. Shining start I say.

Thanks for your contributions thus far and I look forward to your first pull request, no stress as this is, in my experience, a very relaxed and capable team (bar me of course as I just have to wing it from day to day) :) .

@phillxnet

This comment has been minimized.

Show comment
Hide comment
@phillxnet

phillxnet Jul 23, 2016

Member

@grebnek I can confirm you findings for a work around ie on my raid1 that was showing incorrectly as single in the Web-UI (after above single to raid1 conversion), I did the following:

Resize Pool (button) - Modify RAID level - selected raid1.
Awaited it's completion and then the pool was correctly identified within the Web-UI as raid1.

And the same btrfs command as before now gives the following output:

btrfs fi df /mnt2/rock-pool/
Data, RAID1: total=113.00GiB, used=110.02GiB
System, RAID1: total=32.00MiB, used=16.00KiB
Metadata, RAID1: total=1.00GiB, used=391.44MiB
GlobalReserve, single: total=144.00MiB, used=0.00B

Hence also confirming you findings of the "Data, single: total=1.00GiB, used=0.00B" line no longer appearing.

Maybe we could just take the first raid value of Data rather than the last? Haven't look yet so your call on this. We would need to check if the rogue single line is always listed second for this to cut it of course.

Member

phillxnet commented Jul 23, 2016

@grebnek I can confirm you findings for a work around ie on my raid1 that was showing incorrectly as single in the Web-UI (after above single to raid1 conversion), I did the following:

Resize Pool (button) - Modify RAID level - selected raid1.
Awaited it's completion and then the pool was correctly identified within the Web-UI as raid1.

And the same btrfs command as before now gives the following output:

btrfs fi df /mnt2/rock-pool/
Data, RAID1: total=113.00GiB, used=110.02GiB
System, RAID1: total=32.00MiB, used=16.00KiB
Metadata, RAID1: total=1.00GiB, used=391.44MiB
GlobalReserve, single: total=144.00MiB, used=0.00B

Hence also confirming you findings of the "Data, single: total=1.00GiB, used=0.00B" line no longer appearing.

Maybe we could just take the first raid value of Data rather than the last? Haven't look yet so your call on this. We would need to check if the rogue single line is always listed second for this to cut it of course.

@schakrava

This comment has been minimized.

Show comment
Hide comment
@schakrava

schakrava Aug 2, 2016

Member

Fixed by #1408

Member

schakrava commented Aug 2, 2016

Fixed by #1408

@schakrava schakrava closed this Aug 2, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment