Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add 'ipfs multibase' commands #8180

Merged
merged 9 commits into from Aug 18, 2021
Merged

feat: add 'ipfs multibase' commands #8180

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c41b68a
add multibase helper command
Jun 8, 2021
82baaa3
handle files
Jun 8, 2021
8d09010
descriptions
Jun 8, 2021
36e60d2
test
Jun 8, 2021
0ccf9b2
files
Jun 9, 2021
cb997aa
refactor: use base64url as default multibase
lidel Jul 1, 2021
509b2a0
docs(cli): add multibase next to cid command
lidel Jul 1, 2021
27bf745
refactor: improved ux of multibase command
lidel Jul 1, 2021
1c8c234
test: sharness for multibase command
lidel Jul 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
29 changes: 16 additions & 13 deletions core/commands/commands_test.go
View file
Open in desktop
Expand Up @@ -88,18 +88,24 @@ func TestCommands(t *testing.T) {
"/bootstrap/rm",
"/bootstrap/rm/all",
"/cat",
"/cid",
"/cid/base32",
"/cid/bases",
"/cid/codecs",
"/cid/format",
"/cid/hashes",
"/commands",
"/config",
"/config/edit",
"/config/replace",
"/config/show",
"/config/profile",
"/config/profile/apply",
"/config/replace",
"/config/show",
"/dag",
"/dag/get",
"/dag/export",
"/dag/put",
"/dag/get",
"/dag/import",
"/dag/put",
"/dag/resolve",
"/dag/stat",
"/dht",
Expand Down Expand Up @@ -127,16 +133,16 @@ func TestCommands(t *testing.T) {
"/files/read",
"/files/rm",
"/files/stat",
"/files/write",
"/filestore",
"/filestore/dups",
"/filestore/ls",
"/filestore/verify",
"/files/write",
"/get",
"/id",
"/key",
"/key/gen",
"/key/export",
"/key/gen",
"/key/import",
"/key/list",
"/key/rename",
Expand All @@ -148,12 +154,15 @@ func TestCommands(t *testing.T) {
"/log/tail",
"/ls",
"/mount",
"/multibase",
"/multibase/decode",
"/multibase/encode",
"/name",
"/name/publish",
"/name/pubsub",
"/name/pubsub/cancel",
"/name/pubsub/state",
"/name/pubsub/subs",
"/name/pubsub/cancel",
"/name/resolve",
"/object",
"/object/data",
Expand Down Expand Up @@ -230,12 +239,6 @@ func TestCommands(t *testing.T) {
"/urlstore/add",
"/version",
"/version/deps",
"/cid",
"/cid/format",
"/cid/base32",
"/cid/codecs",
"/cid/bases",
"/cid/hashes",
}

cmdSet := make(map[string]struct{})
Expand Down
86 changes: 86 additions & 0 deletions core/commands/multibase.go
View file
Open in desktop
@@ -0,0 +1,86 @@
package commands

import (
"bytes"
"io/ioutil"
"strings"

cmds "github.com/ipfs/go-ipfs-cmds"
"github.com/ipfs/go-ipfs/core/commands/cmdenv"
mbase "github.com/multiformats/go-multibase"
)

var MbaseCmd = &cmds.Command{
Helptext: cmds.HelpText{
Tagline: "Encode and decode files with multibase format",
lidel marked this conversation as resolved.
Show resolved Hide resolved
},
Subcommands: map[string]*cmds.Command{
"encode": mbaseEncodeCmd,
"decode": mbaseDecodeCmd,
},
Extra: CreateCmdExtras(SetDoesNotUseRepo(true)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to merge for now as this seems like it wouldn't hurt and matches what ipfs cid does.

However, I don't think this really works properly without applying the flag to each subcommand.

@lidel let's revisit during the RC process what we want to do here and figure out if ipfs cid has similar problems.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Continued in #8375 (comment)

}

const (
mbaseOptionName = "b"
)

var mbaseEncodeCmd = &cmds.Command{
Helptext: cmds.HelpText{
Tagline: "Encode file or stdin into multibase string",
},
Arguments: []cmds.Argument{
cmds.FileArg("file", true, false, "data to encode").EnableStdin(),
},
Options: []cmds.Option{
cmds.StringOption(mbaseOptionName, "multibase encoding").WithDefault("identity"),
lidel marked this conversation as resolved.
Show resolved Hide resolved
},
Run: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {
if err := req.ParseBodyArgs(); err != nil {
return err
}
encoderName, _ := req.Options[mbaseOptionName].(string)
encoder, err := mbase.EncoderByName(encoderName)
if err != nil {
return err
}
files := req.Files.Entries()
file, err := cmdenv.GetFileArg(files)
if err != nil {
return err
}
buf, err := ioutil.ReadAll(file)
if err != nil {
return err
}
encoded := encoder.Encode(buf)
reader := strings.NewReader(encoded)
return resp.Emit(reader)
},
}

var mbaseDecodeCmd = &cmds.Command{
Helptext: cmds.HelpText{
Tagline: "Decode multibase string to stdout",
},
Arguments: []cmds.Argument{
cmds.FileArg("encoded_file", true, false, "encoded data to decode").EnableStdin(),
},
Run: func(req *cmds.Request, resp cmds.ResponseEmitter, env cmds.Environment) error {
if err := req.ParseBodyArgs(); err != nil {
return err
}
files := req.Files.Entries()
file, err := cmdenv.GetFileArg(files)
if err != nil {
return err
}
encoded_data, err := ioutil.ReadAll(file)
Copy link
Contributor

@gammazero gammazero Jul 16, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be worth implementing a multibase Decoder that would take an io.Reader and return one as well, with a signature like NewDecoder(io.Reader) io.Reader

Then this could all be done without having to read the entire file into memory.

decReader := mbase.NewDecoder(file)
return resp.emit(decReader)

@lidel Seems like this is something that should exist, or am I missing an obvious reason it does not? Seems like a NewDecode function could start reading the stream, and then construct the correct decoder and return it as an io.Reader.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! I filled upstream issue multiformats/go-multibase#44 to discuss this, but out of scope for this PR.

_, data, err := mbase.Decode(string(encoded_data))
if err != nil {
return err
}
reader := bytes.NewReader(data)
return resp.Emit(reader)
},
}
4 changes: 4 additions & 0 deletions core/commands/root.go
View file
Open in desktop
Expand Up @@ -75,6 +75,9 @@ TOOL COMMANDS
commands List all available commands
log Manage and show logs of running daemon

UTILITIES
multibase Encode and decode files with multibase format
lidel marked this conversation as resolved.
Show resolved Hide resolved

Use 'ipfs <command> --help' to learn more about each command.

ipfs uses a repository in the local file system. By default, the repo is
Expand Down Expand Up @@ -151,6 +154,7 @@ var rootSubcommands = map[string]*cmds.Command{
"version": VersionCmd,
"shutdown": daemonShutdownCmd,
"cid": CidCmd,
"multibase": MbaseCmd,
}

// RootRO is the readonly version of Root
Expand Down