"pin verify". #3843

kevina · 2017-03-29T19:33:47Z

No description provided.

kevina · 2017-03-29T19:35:47Z

This needs work, but want to ge tthe U.I down first. Here is the current usage:

$ ipfs pin verify
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn ok
QmVLDAhCY3X9P2uRudKAryuQFPM5zqA3Yij1dY8FpGbL7T ok
QmQZ1WVkddYPpRPm6vPpemhNacjwhupzSMzFEcYMisVpuY broken
  QmSijovevteoY63Uj1uC5b8pkpDU5Jgyk2dYBqz3sMJUPc: merkledag: not found
  QmTbPEyrA1JyGUHFvmtx1FNZVzdBreMv8Hc8jV9sBRWhNA: The block referred to by 'QmTbPEyrA1JyGUHFvmtx1FNZVzdBreMv8Hc8jV9sBRWhNA' was not a vald merkledag node

whyrusleeping · 2017-03-29T19:43:35Z

core/commands/pin.go

@@ -414,6 +436,90 @@ func pinLsAll(typeStr string, ctx context.Context, n *core.IpfsNode) (map[string
 	return keys, nil
 }

+type pinStatus struct {
+	// use pointer to array slice to reduce memory usage


How does this reduce memory usage?

Just make it a normal slice. Slices are already pointers

A slice is the width of three pointers. There is a node for every indirect pin. By using an extra lawyer of indirection I reduce the size by two pointers for each node. The actual array slice will be shared many times in the common case.

whyrusleeping · 2017-03-29T19:45:02Z

core/commands/pin.go

+	return out
+}
+
+func (r pinVerifyRes) Format(out io.Writer) {


Use a marshaler for this.

whyrusleeping · 2017-03-29T19:45:23Z

core/commands/pin.go

+			}
+		}()
+
+		res.SetOutput(rdr)


try not to ever set the output to a reader, that makes the http api realllllly hard to deal with

@whyrusleeping this is a WIP, more concerned with the U.I. will fix up this later.

kevina · 2017-03-30T04:58:13Z

Here is the current usage:

USAGE
  ipfs pin verify - Verify that recursive pins are complete.

SYNOPSIS
  ipfs pin verify [--verbose] [--quiet | -q]

OPTIONS

  --verbose        bool - Also write the hashes of non-broken pins.
  -q,      --quiet bool - Write just hashes of broken pins.

Here as some sample outputs:

$ ipfs pin verify
QmQZ1WVkddYPpRPm6vPpemhNacjwhupzSMzFEcYMisVpuY broken
  QmSijovevteoY63Uj1uC5b8pkpDU5Jgyk2dYBqz3sMJUPc: merkledag: not found
  QmTbPEyrA1JyGUHFvmtx1FNZVzdBreMv8Hc8jV9sBRWhNA: The block referred to by 'QmTbPEyrA1JyGUHFvmtx1FNZVzdBreMv8Hc8jV9sBRWhNA' was not a vald merkledag node

$ ipfs pin verify -q
QmQZ1WVkddYPpRPm6vPpemhNacjwhupzSMzFEcYMisVpuY

$ ipfs pin verify --verbose
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn ok
QmVLDAhCY3X9P2uRudKAryuQFPM5zqA3Yij1dY8FpGbL7T ok
QmQZ1WVkddYPpRPm6vPpemhNacjwhupzSMzFEcYMisVpuY broken
  QmSijovevteoY63Uj1uC5b8pkpDU5Jgyk2dYBqz3sMJUPc: merkledag: not found
  QmTbPEyrA1JyGUHFvmtx1FNZVzdBreMv8Hc8jV9sBRWhNA: The block referred to by 'QmTbPEyrA1JyGUHFvmtx1FNZVzdBreMv8Hc8jV9sBRWhNA' was not a vald merkledag node

@whyrusleeping @Kubuxu (and others) let me know if you think the usage and output is reasonable and I will work on adding test cases and fixing the API.

kevina · 2017-04-26T01:04:03Z

@whyrusleeping @Kubuxu (and others) let me know if you think the usage and output is reasonable and I will work on adding test cases and fixing the API.

hsanjuan · 2017-04-28T18:49:20Z

Is this a client side operation only, not exposed through http api?

kevina · 2017-04-28T18:55:21Z

@hsanjuan what makes you say that?

hsanjuan · 2017-04-28T18:57:48Z

maybe just ignorance @kevina. Just not clear to me what the output would be in JSON since structs have private fields.

kevina · 2017-04-28T18:59:35Z

Just not clear to me what the output would be in JSON since structs have private fields.
@hsanjuan that part is not designed yet

At this point I just want to know "if you think the usage and output is reasonable".

This is not ready for a full review yet.

hsanjuan · 2017-04-28T19:16:38Z

@kevina yeah, so this would involve knowing if the JSON output is reasonable, and how it matches the CLI-formatted output etc..

kevina · 2017-04-28T19:25:24Z

I was hoping for some feedback on the U.I. before I concern myself with the API and the JSON output.

whyrusleeping · 2017-04-29T06:16:51Z

Yeah, given that the output looks nontrivial (nested things and different sorts of behaviour) it would probably be good to spec out what the api looks like here as that may be just as important as the output.

That said, I think the given output looks good to me. It provides decent enough information to the end user

kevina · 2017-05-02T21:43:15Z

@hsanjuan @whyrusleeping I fixed the API.

hsanjuan · 2017-05-04T21:34:56Z

@kevina thanks, can you post the json output for the examples above?

kevina · 2017-05-04T21:55:49Z

@hsanjuan I would if I knew an easy easy way to get at it. I will hand construct it later today if necessary.

Kubuxu · 2017-05-08T16:48:35Z

@kevina ipfs --enc=json pin verify.

kevina · 2017-05-08T18:47:34Z

Standard output:

> ipfs --enc=json pin verify
{"Cid":"zdvgq3v8bUfT3EskMzK7FqzpMiymfmTLfZA8JZh43wr9RMk64","Ok":false,"BadNodes":[{"Cid":"zdvgq3v8bUfT3EskMzK7FqzpMiymfmTLfZA8JZh43wr9RMk64","Err":"unrecognized object type: %!s(uint64=114)"}]}
...
{"Cid":"QmRKmNZqwSxcPs6HjEN3JiAKxjEJMByYpgr5ykwKuqNahP","Ok":false,"BadNodes":[{"Cid":"zdvgqAiZHaP4GiRxrhecMdw6rb2wBMrP7r2CGbicCznebfLTH","Err":"unrecognized object type: %!s(uint64=114)"},{"Cid":"zdvgqAiZHaP4GiRxrhecMdw6rb2wBMrP7r2CGbicCznebfLTH","Err":"unrecognized object type: %!s(uint64=114)"},{"Cid":"zdvgqAiZHaP4GiRxrhecMdw6rb2wBMrP7r2CGbicCznebfLTH","Err":"unrecognized object type: %!s(uint64=114)"},{"Cid":"zdvgqAiZHaP4GiRxrhecMdw6rb2wBMrP7r2CGbicCznebfLTH","Err":"unrecognized object type: %!s(uint64=114)"}]}

With --quiet option.

> ipfs --enc=json pin verify -q
{"Cid":"zdvgqDYeRDUp9EFkTwt4MDH4GFsZKu5wAA2x5zxcPYxGNZks6","Ok":false}
{"Cid":"QmRKmNZqwSxcPs6HjEN3JiAKxjEJMByYpgr5ykwKuqNahP","Ok":false}

With `--verbose option.

{"Cid":"QmYAUTWVubcjipibdrwkJPbH61XNDrERHyAABRqwWU9K46","Ok":true}
...
{"Cid":"zdvgq3v8bUfT3EskMzK7FqzpMiymfmTLfZA8JZh43wr9RMk64","Ok":false,"BadNodes":[{"Cid":"zdvgq3v8bUfT3EskMzK7FqzpMiymfmTLfZA8JZh43wr9RMk64","Err":"unrecognized object type: %!s(uint64=114)"}]}
...
{"Cid":"QmRKmNZqwSxcPs6HjEN3JiAKxjEJMByYpgr5ykwKuqNahP","Ok":false,"BadNodes":[{"Cid":"zdvgqAiZHaP4GiRxrhecMdw6rb2wBMrP7r2CGbicCznebfLTH","Err":"unrecognized object type: %!s(uint64=114)"},{"Cid":"zdvgqAiZHaP4GiRxrhecMdw6rb2wBMrP7r2CGbicCznebfLTH","Err":"unrecognized object type: %!s(uint64=114)"},{"Cid":"zdvgqAiZHaP4GiRxrhecMdw6rb2wBMrP7r2CGbicCznebfLTH","Err":"unrecognized object type: %!s(uint64=114)"},{"Cid":"zdvgqAiZHaP4GiRxrhecMdw6rb2wBMrP7r2CGbicCznebfLTH","Err":"unrecognized object type: %!s(uint64=114)"}]}

whyrusleeping · 2017-05-25T17:29:03Z

@kevina that api output looks good to me

License: MIT Signed-off-by: Kevin Atkinson <k@kevina.org>

kevina · 2017-05-25T17:53:12Z

@whyrusleeping rebased

whyrusleeping

This LGTM, thanks @kevina !

kevina added the status/in-progress In progress label Mar 29, 2017

whyrusleeping reviewed Mar 29, 2017

View reviewed changes

kevina force-pushed the kevina/pin-verify branch from 00ceb4b to fda389b Compare March 29, 2017 19:44

whyrusleeping reviewed Mar 29, 2017

View reviewed changes

core/commands/pin.go Outdated

return out

}

func (r pinVerifyRes) Format(out io.Writer) {

Copy link

Member

whyrusleeping Mar 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a marshaler for this.

whyrusleeping reviewed Mar 29, 2017

View reviewed changes

kevina force-pushed the kevina/pin-verify branch 2 times, most recently from 5e08cbb to 0a6d496 Compare March 29, 2017 21:55

kevina requested review from whyrusleeping and Kubuxu April 26, 2017 01:02

whyrusleeping requested review from a user and hsanjuan April 28, 2017 00:42

kevina force-pushed the kevina/pin-verify branch from abed668 to 6ea4b6c Compare May 2, 2017 21:40

kevina added 2 commits May 25, 2017 13:40

"pin verify": basic implementation

325e1c0

License: MIT Signed-off-by: Kevin Atkinson <k@kevina.org>

"pin verify": don't use a pointer to a slice.

6e898ea

License: MIT Signed-off-by: Kevin Atkinson <k@kevina.org>

"pin verify": add --verbose and --quiet options

7a4031d

License: MIT Signed-off-by: Kevin Atkinson <k@kevina.org>

kevina force-pushed the kevina/pin-verify branch from 6ea4b6c to ec30b38 Compare May 25, 2017 17:41

kevina changed the title ~~WIP "pin verify".~~ "pin verify". May 25, 2017

"pin verify": fix API

b4a0087

License: MIT Signed-off-by: Kevin Atkinson <k@kevina.org>

kevina force-pushed the kevina/pin-verify branch from ec30b38 to b4a0087 Compare May 25, 2017 17:52

kevina mentioned this pull request May 25, 2017

"pin verify" kevina/todo#3

Closed

whyrusleeping approved these changes May 25, 2017

View reviewed changes

whyrusleeping merged commit c7462ac into master May 25, 2017

whyrusleeping added this to the Ipfs 0.4.10 milestone May 25, 2017

whyrusleeping removed status/in-progress In progress labels May 25, 2017

whyrusleeping deleted the kevina/pin-verify branch May 25, 2017 21:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"pin verify". #3843

"pin verify". #3843

kevina commented Mar 29, 2017

kevina commented Mar 29, 2017

whyrusleeping Mar 29, 2017

whyrusleeping Mar 29, 2017

kevina Mar 29, 2017

whyrusleeping Mar 29, 2017

whyrusleeping Mar 29, 2017

kevina Mar 29, 2017

kevina commented Mar 30, 2017 •

edited

Loading

kevina commented Apr 26, 2017

hsanjuan commented Apr 28, 2017

kevina commented Apr 28, 2017

hsanjuan commented Apr 28, 2017

kevina commented Apr 28, 2017

hsanjuan commented Apr 28, 2017

kevina commented Apr 28, 2017

whyrusleeping commented Apr 29, 2017

kevina commented May 2, 2017

hsanjuan commented May 4, 2017

kevina commented May 4, 2017 •

edited

Loading

Kubuxu commented May 8, 2017

kevina commented May 8, 2017

whyrusleeping commented May 25, 2017

kevina commented May 25, 2017

whyrusleeping left a comment

"pin verify". #3843

"pin verify". #3843

Conversation

kevina commented Mar 29, 2017

kevina commented Mar 29, 2017

whyrusleeping Mar 29, 2017

Choose a reason for hiding this comment

whyrusleeping Mar 29, 2017

Choose a reason for hiding this comment

kevina Mar 29, 2017

Choose a reason for hiding this comment

whyrusleeping Mar 29, 2017

Choose a reason for hiding this comment

whyrusleeping Mar 29, 2017

Choose a reason for hiding this comment

kevina Mar 29, 2017

Choose a reason for hiding this comment

kevina commented Mar 30, 2017 • edited Loading

kevina commented Apr 26, 2017

hsanjuan commented Apr 28, 2017

kevina commented Apr 28, 2017

hsanjuan commented Apr 28, 2017

kevina commented Apr 28, 2017

hsanjuan commented Apr 28, 2017

kevina commented Apr 28, 2017

whyrusleeping commented Apr 29, 2017

kevina commented May 2, 2017

hsanjuan commented May 4, 2017

kevina commented May 4, 2017 • edited Loading

Kubuxu commented May 8, 2017

kevina commented May 8, 2017

whyrusleeping commented May 25, 2017

kevina commented May 25, 2017

whyrusleeping left a comment

Choose a reason for hiding this comment

kevina commented Mar 30, 2017 •

edited

Loading

kevina commented May 4, 2017 •

edited

Loading