Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Strange behaviour: a write trough volume can be created, a write back can't - same device #103

Closed
matteotenca opened this issue Jun 12, 2022 · 5 comments
Closed

Strange behaviour: a write trough volume can be created, a write back can't - same device #103

matteotenca opened this issue Jun 12, 2022 · 5 comments

Comments

@matteotenca
Copy link
Collaborator

matteotenca commented Jun 12, 2022 •
edited
Loading

Hello,
I noticed this strange problem - I don't know if is a rapiddisk's one or not.
I create a ramdisk, I try to attach it to a partition in wt/wa mode, all is ok.
But if I try to attach it to the same partition in wb mode, I got an error (rapiddisk exit with 0 though).

shub@efikiller:~$ sudo rapiddisk -l
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

List of RapidDisk device(s):

 RapidDisk Device 1: rd0	Size (KB): 131072

List of RapidDisk-Cache mapping(s):

  None

shub@efikiller:~$ sudo rapiddisk -m rd0 -b /dev/sda5 -p wt
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

Command to map rc-wt_sda5 with rd0 and /dev/sda5 has been sent.
Verify with "-l"

shub@efikiller:~$ sudo rapiddisk -l
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

List of RapidDisk device(s):

 RapidDisk Device 1: rd0	Size (KB): 131072

List of RapidDisk-Cache mapping(s):

 RapidDisk-Cache Target 1: rc-wt_sda5	Cache: rd0  Target: sda5 (WRITE THROUGH)

shub@efikiller:~$ sudo rapiddisk -u rc-wt_sda5
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

Command to unmap rc-wt_sda5 has been sent
Verify with "-l"

shub@efikiller:~$ sudo rapiddisk -l
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

List of RapidDisk device(s):

 RapidDisk Device 1: rd0	Size (KB): 131072

List of RapidDisk-Cache mapping(s):

  None

shub@efikiller:~$ sudo rapiddisk -m rd0 -b /dev/sda5 -p wb
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

device-mapper: reload ioctl on rc-wb_sda5  failed: Invalid argument
Command failed.
Error. Unable to create map. Please verify all input values are correct.

shub@efikiller:~$ echo $?
0
shub@efikiller:~$

From dmesg:

[ 2009.238044] device-mapper: rapiddisk-cache: Allocate 256KB (8B per) mem for 32768-entry cache(capacity:128MB, associativity:512, block size:8 sectors(4KB))
[ 2155.369951] device-mapper: rapiddisk-cache: stats:
               	reads(299), writes(0)

[ 2155.369953] device-mapper: rapiddisk-cache: 	cache hits(28), replacement(0), write replacement(0)
               	read invalidates(0), write invalidates(0)

[ 2155.369954] device-mapper: rapiddisk-cache: conf:
               	capacity(128M), associativity(512), block size(4K)
               	total blocks(32768), cached blocks(270)

[ 2182.144049] device-mapper: table: 253:1: len=210745823 not aligned to h/w logical block size 4096 of sda5
[ 2182.144052] device-mapper: core: Cannot calculate initial queue limits
[ 2182.144054] device-mapper: ioctl: unable to set up device queue for new table.

From lsblk:

shub@efikiller:~$ sudo lsblk --json --bytes -O /dev/sda5|jq
{
  "blockdevices": [
    {
      "name": "sda5",
      "kname": "sda5",
      "path": "/dev/sda5",
      "maj:min": "8:5",
      "fsavail": null,
      "fssize": null,
      "fstype": "ext4",
      "fsused": null,
      "fsuse%": null,
      "mountpoint": null,
      "label": "bubuntu",
      "uuid": "829e1c81-820b-4feb-a70c-c01898f1f4e7",
      "ptuuid": "107c0ddb-5858-9090-8081-828310111213",
      "pttype": "gpt",
      "parttype": "0fc63daf-8483-4772-8e79-3d69d8477de4",
      "partlabel": "bubuntu",
      "partuuid": "67c2eb08-51e7-40cf-bd11-d87b7be25d0c",
      "partflags": null,
      "ra": 128,
      "ro": false,
      "rm": false,
      "hotplug": false,
      "model": null,
      "serial": null,
      "size": 107901861376,
      "state": null,
      "owner": "root",
      "group": "disk",
      "mode": "brw-rw----",
      "alignment": 0,
      "min-io": 4096,
      "opt-io": 0,
      "phy-sec": 4096,
      "log-sec": 512,
      "rota": true,
      "sched": "mq-deadline",
      "rq-size": 64,
      "type": "part",
      "disc-aln": 0,
      "disc-gran": 0,
      "disc-max": 0,
      "disc-zero": false,
      "wsame": 0,
      "wwn": "0x5000c50086cf3ad0",
      "rand": true,
      "pkname": "sda",
      "hctl": null,
      "tran": null,
      "subsystems": "block:scsi:pci",
      "rev": null,
      "vendor": null,
      "zoned": "none"
    }
  ]
}

The size of the partion in bytes divided by the physical sector size:

shub@efikiller:~$ echo 107901861376/4096 | bc -l
26343227.87500000000000000000

The size of the partion in bytes divided by the logical sector size:

shub@efikiller:~$ echo 107901861376/512 | bc -l
210745823.00000000000000000000

shub@efikiller:~$ uname -a
Linux efikiller 5.13.0-44-generic #49~20.04.1-Ubuntu SMP Wed May 18 18:44:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Regards
@matteotenca
Copy link
Collaborator Author

@pkoutoupis
I extended the partition with gparted, adding 1 MB, and now everything works:

shub@efikiller:~$ sudo lsblk --json --bytes -O /dev/sda5|jq
{
  "blockdevices": [
    {
      "name": "sda5",
      "kname": "sda5",
      "path": "/dev/sda5",
      "maj:min": "8:5",
      "fsavail": null,
      "fssize": null,
      "fstype": "ext4",
      "fsused": null,
      "fsuse%": null,
      "mountpoint": null,
      "label": "bubuntu",
      "uuid": "829e1c81-820b-4feb-a70c-c01898f1f4e7",
      "ptuuid": "107c0ddb-5858-9090-8081-828310111213",
      "pttype": "gpt",
      "parttype": "0fc63daf-8483-4772-8e79-3d69d8477de4",
      "partlabel": "bubuntu",
      "partuuid": "67c2eb08-51e7-40cf-bd11-d87b7be25d0c",
      "partflags": null,
      "ra": 128,
      "ro": false,
      "rm": false,
      "hotplug": false,
      "model": null,
      "serial": null,
      "size": 107902664704,
      "state": null,
      "owner": "root",
      "group": "disk",
      "mode": "brw-rw----",
      "alignment": 0,
      "min-io": 4096,
      "opt-io": 0,
      "phy-sec": 4096,
      "log-sec": 512,
      "rota": true,
      "sched": "mq-deadline",
      "rq-size": 64,
      "type": "part",
      "disc-aln": 0,
      "disc-gran": 0,
      "disc-max": 0,
      "disc-zero": false,
      "wsame": 0,
      "wwn": "0x5000c50086cf3ad0",
      "rand": true,
      "pkname": "sda",
      "hctl": null,
      "tran": null,
      "subsystems": "block:scsi:pci",
      "rev": null,
      "vendor": null,
      "zoned": "none"
    }
  ]
}
shub@efikiller:~$ echo 107902664704/4096|bc -l
26343424.00000000000000000000
shub@efikiller:~$ sudo rapiddisk -m rd0 -b /dev/sda5 -p wb
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

Command to map rc-wb_sda5 with rd0 and /dev/sda5 has been sent.
Verify with "-l"

hub@efikiller:~$ sudo rapiddisk -l
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

List of RapidDisk device(s):

 RapidDisk Device 1: rd0	Size (KB): 131072

List of RapidDisk-Cache mapping(s):

 dm-writecache Target   1: rc-wb_sda5	Cache: rd0  Target: sda5 (WRITEBACK)

shub@efikiller:~$

@pkoutoupis
Copy link
Owner

I am very limited with what I can do with dm-writecache since it isn't my driver or project. With regards to this:

I got an error (rapiddisk exit with 0 though)

The dmsetup technically did execute without error but driver did not like the input. This was always a problem when relying on the device-mapper framework since day one. That is why I write:

Command to map rc-wt_sda5 with rd0 and /dev/sda5 has been sent.
Verify with "-l"

That the command was sent and to verify that it was created.

@matteotenca
Copy link
Collaborator Author

The dmsetup technically did execute without error but driver did not like the input. This was always a problem when relying on the device-mapper framework since day one. That is why I write:

Command to map rc-wt_sda5 with rd0 and /dev/sda5 has been sent.
Verify with "-l"

That the command was sent and to verify that it was created.

@pkoutoupis Thank you, this is an important information... maybe I should arrange the scripts in such a way they don't rely on the return code only. Thanks for the explanation!

I have a question, not strictly related to this (non existing) issue: in the help there is this example:

rapiddisk -m rd3 -b /dev/mapper/rc-wa_sdb -p wb

This should create something like this (on my system):

shub@ubuserver:~$ sudo rapiddisk -l
rapiddisk 8.1.0
Copyright 2011 - 2022 Petros Koutoupis

List of RapidDisk device(s):

 RapidDisk Device 1: rd1        Size (KB): 131072
 RapidDisk Device 2: rd0        Size (KB): 131072

List of RapidDisk-Cache mapping(s):

 RapidDisk-Cache Target 1: rc-wa_sda2   Cache: rd0  Target: sda2 (WRITE AROUND)
 dm-writecache Target   2: rc-wb_rc-wa_sda2     Cache: rd1  Target: dm-0 (WRITEBACK)

shub@ubuserver:~$

The device mounted is /dev/mapper/rc-wb_rc-wa_sda2.

I suppose what happens is this: the application writes to Target 2 which caches data and sends it to Target 1 after a while. When this appens, Target 1 does not caches the writes (cause it is write-around mode) and put data on the disk.

While reading, the application reads from Target 2, which does not cache reads; Target 2 reads from from Target 1 which reads from disk and caches data.

Is this correct? Is the goal to take advantage of both rapiddisk's read-only cache and rapiddisk/dm-writeback's write-only cache at the same time?

I do apologize if I missed somethnig in the documentation; I don't know anything of this kind of low-level programming!

Regards

@pkoutoupis
Copy link
Owner

Yes. You are correct in your understanding of the above setup.

@matteotenca
Copy link
Collaborator Author

Thank you. Maybe you can add this infos in the documentation, since it's a nice and interesting approach to rapiddisk! I believe this issue can be closed!

Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pkoutoupis @matteotenca