Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Editable Metadata for file (whilst uploading) and default values are inferred from supplied file (e.g. title ...) #12

Closed
24 of 27 tasks
lgilies opened this issue Aug 11, 2020 · 13 comments
Assignees

Comments

@lgilies
Copy link

lgilies commented Aug 11, 2020

When the user selects a file and starts uploading they are presented with key metadata fields (e.g.title, description, encode, and format) that they can edit. These metadata fields should be auto-populated where possible. The user can then explicitly save (once the file has finished uploading).

Bonus would be extracting the table schema info: I want to extract the table schema info from the file and save it in a JSON object. See #13

Acceptance

Given a file by a user

  • Present a form with key metadata fields:
  • Auto-populate with the information extracted from the file where possible
    • path: from file name
    • title: either empty OR (tbd) a human version of name (e.g. _ => " ")
    • description - should be an empty and editable field
    • format - should be automatically filled with the extension of the file. The user can edit and select a new format from a format list (format list is passed to the app/component on initialisation)
    • encoding: should be empty and the user can select an option from a list of encodings (this encoding list should be passed to the app/component on initlisation)
  • This information should be serialized internally to a frictionless data Data Resource based structure
  • On Save this information will be used to update CKAN MetaStore
    • This assumes package id for this resource has been passed in.

Tasks

  • UI - Create metadata fields for (title, description, encode, and format) with a unique ID for which field.
  • JS - Get the ID's and auto-populate the metadata fields with the file information.
    • I think we have all info we need from HTML5 FileReader ...
    • If we are doing table schema we can/should use data.js https://github.com/datopian/data.js to load the file and infer information
  • TODO: saving process
    • need to check we don't save before file has finished uploading

Bonus

  • check and test more how the import-ui works
  • install and test react-table
  • create a variable "schema" type object
  • iterable the data(table schema) from the file and save in the variable.
  • Show this data with react table
@lgilies lgilies changed the title Metadata fields are inferred from supplied file Metadata fields are inferred from supplied file (e.g. title ...) Aug 11, 2020
@mariorodeghiero mariorodeghiero self-assigned this Aug 11, 2020
@rufuspollock rufuspollock changed the title Metadata fields are inferred from supplied file (e.g. title ...) Editable Metadata for file (whilst uploading) and default values are inferred from supplied file (e.g. title ...) Aug 17, 2020
mariorodeghiero added a commit that referenced this issue Aug 18, 2020
@mariorodeghiero
Copy link
Contributor

mariorodeghiero commented Aug 19, 2020

@rufuspollock please take a look at this gif below.

  1. I am adding a file and auto-populate with the file information
  2. I edited the fields and I clicked in "save"(just an event to show a console.log with the metadata object)
  3. I started to upload the file
  4. I edited the fields with new information and I clicked in "save" again while the upload is happening.

Object example:

{
    bytes: 3862044
    description: "new description"
    encoding: "windows_1256"
    format: "HTML"
    hash: "SHA256:8173a1ce8be87c72ecb7a74ec26c9b994d641c0b2fbafb1aa4f5e5ca63d023b5"
    mediatype: "text/csv"
    name: "Data-Prices"
    path: "Data-Prices.csv"
    title: "Data Prices Test"
}

My question is: each key of this object above needs to have a label and description?
e.g:

name: {
    label: "name"
    description: "name of the dataset"
    value: "Data-Prices"
}

fields

mariorodeghiero added a commit that referenced this issue Aug 19, 2020
- auto-populate with the information of the file
- render a list of format
- render a list of encode
- be able to edit the metadata information when uploading the file
@rufuspollock
Copy link
Member

rufuspollock commented Aug 19, 2020

@mariorodeghiero great job 👏

My question is: each key of this object above needs to have a label and description?

No, you don't. If you want information about fields that should go in the profile / json schema which describes the form. For now I would be using an enhanced profile based of data resource json schema but with additional enums for e.g. format field. (if that does not make sense let's chat).

Other comments

  • I think we want probably disable saving metadata until upload is complete (in general i think we want to think about ux here e.g. do we want to hide upload button once upload has started etc).

@mariorodeghiero
Copy link
Contributor

@rufuspollock ok, I will get a list of format and licenses and render it in select option.

yes, I agree with you, we need to think about UX. I will try to add something like you said.

@mariorodeghiero
Copy link
Contributor

@rufuspollock take a look.

ux

@rufuspollock
Copy link
Member

@mariorodeghiero looks good 👏 . I think we could auto start upload once file is selected (so no need for upload button) but that is very minor.

Can we get this deployed at a staging site (maybe using storybook) so i can play around - this is #5

@mariorodeghiero
Copy link
Contributor

@rufuspollock right, I'm adding the licenses and I'll remove to use auto-upload and start to create the new stories.

mariorodeghiero added a commit that referenced this issue Aug 20, 2020
- add licenses json
- add formats json
- removed button upload
- add auto-upload when selecet a file
- enable save metadata button when the upload is done
- render list of licenses
- render list of formats
- change the UI
mariorodeghiero added a commit that referenced this issue Aug 20, 2020
@rufuspollock
Copy link
Member

rufuspollock commented Aug 21, 2020 via email

@anuveyatsu
Copy link
Member

@rufuspollock we're looking at "Bonus" part of this issue's Acceptance Criteria and it looks like it is related to #13

  • extract the table schema info from a file and store Table Schema structured json object
  • Present that table schema with a preview of the data so the user can edit / adjust iit

Could you confirm?

cc @mariorodeghiero

@rufuspollock
Copy link
Member

@anuveyatsu confirmed and i've removed it.

@mariorodeghiero
Copy link
Contributor

Fixed.

@rufuspollock
Copy link
Member

rufuspollock commented Aug 25, 2020

@mariorodeghiero there was one item in the acceptance not ticked re labels. Can you comment on that in the acceptance section. Also in general can you update the acceptance section with brief comments on each item see https://playbook.datopian.com/issues-tasks/#completing-tasks for notes about that. (so not just ticking the checkbox but a short comment too in bold)

Also are we running off json schema atm or is it hard-coded (the form)?

@mariorodeghiero
Copy link
Contributor

The item below was not ticked. You can check why in this conversation

  • each field may have a short label/description
  • UI - Create metadata fields for (title, description, encode, and format) Done -- check here
  • JS - Get the ID's and auto-populate the metadata fields **Done -- check [https://datopian.github.io/datapub/?path=/story/components-metadata--idle]
  • The user can't save before file has finished uploading Done -- you can see in the gif below

upload-new

@rufuspollock
Copy link
Member

rufuspollock commented Aug 25, 2020

@mariorodeghiero i think we misunderstood each other in that discussion. I meant you don't need a label in what you save but you want a label / description in the form which come from the json schema (if we had that).

Anyway at this point no big deal and we can leave this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants