Skip to content

Add zdb -r <dataset> <object-id | file> <output> #11027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 28, 2021
Merged

Add zdb -r <dataset> <object-id | file> <output> #11027

merged 1 commit into from
Jan 28, 2021

Conversation

allanjude
Copy link
Contributor

zdb will copy the specified object to the output file

Signed-off-by: Allan Jude allan@klarasystems.com

Motivation and Context

While you can use zdb -R poolname vdev:offset:[<lsize>/]<psize>[:flags] to extract individual DVAs from a vdev, it would be handy for be able copy an entire file

Description

Given a file or object number, copy the contents to a file. Useful for debugging and recovery.

How Has This Been Tested?

Demoed live at the OpenZFS Developer Summit 2020

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (a change to man pages or other documentation)

Checklist:

@allanjude
Copy link
Contributor Author

A suggested future improvement discussed at the hangout after the dev summit:
Read the file in chunks of the recordsize (rather than the current MIN(size, 1 MiB), and on error, print a warning, write a block of zeros to the output, then continue.

This would allow extracting the undamaged parts of a file that has some unreadable records.

lundman
lundman reviewed Oct 8, 2020
View reviewed changes
lundman
lundman reviewed Oct 8, 2020
View reviewed changes
@lundman
Copy link
Contributor

lundman commented Oct 8, 2020

It seems quite keen to run all the time:

# Lets just run zdb on a soft-import (no cache file)

./scripts/cmd-macos.sh zdb -e BOOM/ -p /dev/
failed to hold dataset 'BOOM/': No such file or directory

Configuration for import:
        vdev_children: 1
        version: 5000
        pool_guid: 2616708186120125697
        name: 'BOOM'
[snip]

        load-policy:
            load-request-txg: 18446744073709551615
            load-rewind-policy: 2
Copy Object 0 to file /dev/
Failed to get handle for SA znode

Did not specify -r just yet...

@allanjude
Copy link
Contributor Author

It seems quite keen to run all the time:

It seems I needed to explicitly add it to the exclude list for 'dump_all'

@behlendorf behlendorf added the Status: Code Review Needed Ready for review and testing label Oct 8, 2020
behlendorf
behlendorf reviewed Oct 8, 2020
View reviewed changes
Copy link
Contributor

@behlendorf behlendorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will definitely come in handy. Why don't we also add a simple test case to tests/zfs-tests/tests/functional/cli_root/zdb/.

@behlendorf behlendorf added Status: Work in Progress Not yet ready for general review and removed Status: Code Review Needed Ready for review and testing labels Oct 22, 2020
@behlendorf
Copy link
Contributor

@allanjude I've seen a couple cases now where this would be really helpful for debugging. If by chance you have some time to dust it off and tackle the remaining feedback I'd love to get it in.

@allanjude allanjude force-pushed the zdb_cp branch 2 times, most recently from 91a0915 to 4001892 Compare January 27, 2021 02:37
@allanjude
Copy link
Contributor Author

I've applied the review feedback, and created a very simple test case (it turned out to need a zpool sync, otherwise the file was not entirely written yet)

behlendorf
behlendorf reviewed Jan 27, 2021
View reviewed changes
Copy link
Contributor

@behlendorf behlendorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few little things then I'm good with this. Thanks for the quick refresh.

@allanjude
Copy link
Contributor Author

I think that should do it

@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Work in Progress Not yet ready for general review labels Jan 27, 2021
behlendorf
behlendorf reviewed Jan 27, 2021
View reviewed changes
Copy link
Contributor

@behlendorf behlendorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there's one last thing, the new test case needs to be added to tests/runfiles/common.run and tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am.

@allanjude
Add zdb -r <dataset> <object-id | file> <output>
98a9de3 
zdb will copy the specified object to the output file

Signed-off-by: Allan Jude <allan@klarasystems.com>
@allanjude
Copy link
Contributor Author

I also added a 2nd test that tests a file with an odd length, as there used to be a problem where it'd pad the extracted file to the record size with garbled data

@behlendorf behlendorf merged commit 393e692 into openzfs:master Jan 28, 2021
jsai20 pushed a commit to jsai20/zfs that referenced this pull request Mar 30, 2021
@allanjude @jsai20
Add zdb -r <dataset> <object-id | file> <output>
0771a8b 
While you can use zdb -R poolname vdev:offset:[<lsize>/]<psize>[:flags] 
to extract individual DVAs from a vdev, it would be handy for be able 
copy an entire file out of the pool.

Given a file or object number, add support to copy the contents to a 
file. Useful for debugging and recovery.

Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Closes openzfs#11027
sempervictus pushed a commit to sempervictus/zfs that referenced this pull request May 31, 2021
@allanjude
Add zdb -r <dataset> <object-id | file> <output>
eee0ec2 
While you can use zdb -R poolname vdev:offset:[<lsize>/]<psize>[:flags] 
to extract individual DVAs from a vdev, it would be handy for be able 
copy an entire file out of the pool.

Given a file or object number, add support to copy the contents to a 
file. Useful for debugging and recovery.

Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Closes openzfs#11027
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@behlendorf behlendorf behlendorf approved these changes

@lundman lundman lundman approved these changes

Assignees
No one assigned
Labels
Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@allanjude @lundman @behlendorf