New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update dcmp interface #228

Merged
merged 1 commit into from Feb 8, 2019

Conversation

Projects
None yet
3 participants
@dsikich
Copy link
Member

dsikich commented Feb 6, 2019

Change the way differences between the source and
destination directories in dcmp are reported.

@dsikich dsikich requested review from adammoody and gonsie Feb 6, 2019

@gonsie

This comment has been minimized.

Copy link
Contributor

gonsie commented Feb 6, 2019

this addresses #217

@dsikich

This comment has been minimized.

Copy link
Member Author

dsikich commented Feb 6, 2019

@gonsie @adammoody Here is new summary output from my testing:

quartz1148{sikich1}193: ~/install/bin/dcmp -v ~/dirA ~/dirB
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:2108] Walking source path
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/common/mfu_flist_walk.c:740] Walking /g/g0/sikich1/dirA
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/common/mfu_flist_walk.c:813] Walked 3 items in 0.007563 seconds (396.662001 files/sec)
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:2113] Walking destination path
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/common/mfu_flist_walk.c:740] Walking /g/g0/sikich1/dirB
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/common/mfu_flist_walk.c:813] Walked 1 items in 0.005715 seconds (174.974094 files/sec)
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:854] Started   : Feb-05-2019, 17:10:57
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:855] Completed : Feb-05-2019, 17:10:57
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:856] Seconds   : 0.005
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:857] Items     : 3
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:859] Item Rate : 3 items in 0.004815 seconds (623.101515 items/sec)
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:861] Bytes read: 0.000 B (0 bytes)
[2019-02-05T17:10:57] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:863] Byte Rate : 0.000 B/s (000 bytes in 0.005 seconds)
Number of files that exist in both directories: 1
	Source: 1	Destination: 1
Number of files that exist only in one directory  --  Source: 2  Destination: 0
Number of files that exist in both directories and have the same type: 1
	Source: 1	Destination: 1
Number of files that exist in both directories and have different types: 0
	Source: 0	Destination: 0
Number of files that exist in both directories and have the same content: 1
	Source: 1	Destination: 1
Number of files that exist in both directories and have different contents: 0
	Source: 0	Destination: 0

All of the dcmp code right now assumes each line in the summary output will have the same text/print line based on an enum field that is looped over with a macro function, but different numbers to report. Let me know if this is easy enough to understand. If not, we will have to break many of the original assumptions in the dcmp code.

@gonsie

This comment has been minimized.

Copy link
Contributor

gonsie commented Feb 6, 2019

If I’m interpreting this correctly, then there are 2 files in the source dir which don’t exist in the destination. There is 1 file that exists in both that has the same name and content. Is that correct?

I will also show this to @cmharr to see if it makes sense.

@dsikich

This comment has been minimized.

Copy link
Member Author

dsikich commented Feb 6, 2019

@gonsie yes, two files in source dir which do not exist in the destination directory. And yes to the second question as well.

@cmharr

This comment has been minimized.

Copy link

cmharr commented Feb 6, 2019

This looks clearer to me. I think it might look a bit cleaner if the source/destination were on the same line like the listing # files existing in one directory. For instance, something like:
Number of files that exist in both directories and have different types: 0 (Src: 0 Dest: 0)

@dsikich dsikich force-pushed the dcmp-interface branch from 82b647b to 5157c54 Feb 6, 2019

@dsikich

This comment has been minimized.

Copy link
Member Author

dsikich commented Feb 6, 2019

@cmharr @adammoody @gonsie Yeah, I like that better. Here is the updated output:

~/install/bin/dcmp -v ~/dirA ~/dirB
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:2108] Walking source path
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/common/mfu_flist_walk.c:740] Walking /g/g0/sikich1/dirA
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/common/mfu_flist_walk.c:813] Walked 3 items in 0.008902 seconds (337.000161 files/sec)
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:2113] Walking destination path
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/common/mfu_flist_walk.c:740] Walking /g/g0/sikich1/dirB
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/common/mfu_flist_walk.c:813] Walked 1 items in 0.007134 seconds (140.169903 files/sec)
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:854] Started   : Feb-06-2019, 10:17:35
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:855] Completed : Feb-06-2019, 10:17:35
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:856] Seconds   : 0.014
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:857] Items     : 3
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:859] Item Rate : 3 items in 0.013958 seconds (214.927184 items/sec)
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:861] Bytes read: 0.000 B (0 bytes)
[2019-02-06T10:17:35] [0] [/g/g0/sikich1/mpifileutils/src/dcmp/dcmp.c:863] Byte Rate : 0.000 B/s (000 bytes in 0.014 seconds)
Number of files that exist in both directories: 1 (Src: 1 Dest: 1)
Number of files that exist only in one directory: N/A (Src: 2 Dest: 0)
Number of files that exist in both directories and have the same type: 1 (Src: 1 Dest: 1)
Number of files that exist in both directories and have different types: 0 (Src: 0 Dest: 0)
Number of files that exist in both directories and have the same content: 1 (Src: 1 Dest: 1)
Number of files that exist in both directories and have different contents: 0 (Src: 0 Dest: 0)
update dcmp interface
Change the way differences between the source and
destination directories in dcmp are reported.

@dsikich dsikich force-pushed the dcmp-interface branch from 5157c54 to 44c1b11 Feb 6, 2019

@gonsie

gonsie approved these changes Feb 7, 2019

@dsikich dsikich merged commit 7d2ca2f into master Feb 8, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment