Does data need to have a datapackage.json? If yes, infer one.

[pwalsh]

Description

I'm a first time user of the current CLI. After login, I want to just push something.

mkdir test
cd test
touch test.txt
data push
> Error! ENOENT: no such file or directory, open '/Users/pwalsh/test/datapackage.json'

I'm confused because the CLI tells me the following about data push:

[path] Push data at path to the DataHub

I could be a user who does not know what datapackage.json is.

As someone involved in the development of our Frictionless Data software, I know we have an infer method for Data Packages, so I wonder why we are not using it.

[pwalsh]

Above I linked to infer in Python. Here is the same in JavaScript.

[Mikanebu]

@pwalsh data push checks for datapackage.json file in given directory. If you run data push fileName (you need to provide full path to a file, otherwise it would count it as a Data Package) it will publish by generating datapackage.json automatically by our system. This is good point.

[anuveyatsu]

@pwalsh we expect users to have a full path to a file because a directory can have any number of files/dirs in it. So we assume there is a datapackage.json in the given dir and raise an error if not. As mentioned above by @Mikanebu you can push a single files and we do infer descriptor for it.

[pwalsh]

Ok. It feels a bit arbitrary, but good to know 👍

[rufuspollock]

@pwalsh what do you think is the expected behaviour. There are a few options when we do data push in a directory:

1. Check for a datapackage.json.
   • If found use that.
   • If no datapackage.json escape out
2. Check for a datapackage.json.
   • If found use that
   • Else: Search the directory (and subdirectories) and create a data package based on that (in some interactive way)
3. Present a help message and ask them to do data push . or something else

[pwalsh]

Depends: do we definitely intend to only push files referenced in a datapackage.json? Or, files found at location?

[rufuspollock]

@pwalsh we want people to be able to push files really easily.

The question here is not whether we allow them to push files but what is the UX of the CLI. Consider git:

Doing git push in a non git repo will result in:

fatal: Not a git repository (or any of the parent directories): .git

You then then need to do:

git init
git add ...
git commit
git push

Whilst git is maybe not the best example of a great cli experience 😉 it illustrates that it has the user perform several steps rather than trying to do too much magic in one.

Similarly, here. We could have data push try to do some magic for people and guess what they want (or prompt them) or we could be strict that data push does not work if no datapackage.json created and prompt them to run data init in the directory to auto-create the datapackage.json for them.

My instinct, having thought about it is that this latter, less magical, approach is better but i'm keen to get your (and others) views.

[pwalsh]

That is fine, and I agree on less magic, but it means that we must educate the user as to what a Data Package is, i.e.: Data Package is a concept that users must grok in order to push data.

[rufuspollock]

@pwalsh good point. They only need to grok if they don't push a single file but want to push multiple files.

[AcckiyGerman]

We extended the error message and it now suggest to:

• create a new datapackage with data init
• read more help with data help push.

Related issue #63
Fix PR: frictionlessdata/frictionless-js#29

Closing this issue in an advance as DUPLICATE of #63
@pwalsh Thanks for appreciate reports ;-)