Skip to content

Conversation

@jsam
Copy link
Contributor

@jsam jsam commented Oct 31, 2019

closes: #781

@jsam jsam requested a review from a team as a code owner October 31, 2019 10:45
Copy link
Contributor Author

@jsam jsam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a test which caused this problem here as well, but I have no idea which DOI caused it, so any help with searching of that DOI is more than welcome. 🙉

Copy link
Contributor

@mohammad-alisafaee mohammad-alisafaee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a test which caused this problem here as well, but I have no idea which DOI caused it, so any help with searching of that DOI is more than welcome. 🙉

git grep path .renku/datasets shows that the empty path is in .renku/datasets/dd38afca-0e96-4034-a2e5-dc7505c0af30/metadata.yml so I guess the DOI is https://doi.org/10.5281/zenodo.3348115 (https://zenodo.org/record/3348115).

if p and len(p) >= 1:
return Path(p).name


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please explain how this fix works? If filename is None then no DatasetFile object will be created!?

Copy link
Contributor Author

@jsam jsam Oct 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be created, but filename field will remain None within the instantiated object.

The problem of why raise was happening is because this Path(None) results in TypeError. Now why that is None is probably during import phase something went wrong or we didn't properly validate.

@jsam
Copy link
Contributor Author

jsam commented Oct 31, 2019

I would add a test which caused this problem here as well, but I have no idea which DOI caused it, so any help with searching of that DOI is more than welcome. hear_no_evil

git grep path .renku/datasets shows that the empty path is in .renku/datasets/dd38afca-0e96-4034-a2e5-dc7505c0af30/metadata.yml so I guess the DOI is https://doi.org/10.5281/zenodo.3348115 (https://zenodo.org/record/3348115).

Hmm, this is interesting. Seems like migration dumped that dataset since I don't even have it. 🤔 Thanks! I'll look into this and add the test.

@jsam jsam changed the title fix: adds simple check on empty filename [WIP] fix: adds simple check on empty filename Oct 31, 2019
@jsam jsam changed the title [WIP] fix: adds simple check on empty filename fix: adds simple check on empty filename Nov 5, 2019
@jsam jsam force-pushed the 781_bugfix branch 4 times, most recently from 30a391f to 5b1b7f8 Compare November 6, 2019 16:13
Copy link
Member

@rokroskar rokroskar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just looking at what happens to the DatasetFile objects that have name: null and they seem to also have a very strange url:

- '@type':
  - prov:Entity
  - schema:DigitalDocument
  - wfprov:Artifact
  _id: blob/c6a0aa165dcab4ca7f5c45a935f1d8e1a1eab52b/
  _label: c6a0aa165dcab4ca7f5c45a935f1d8e1a1eab52b
  _project: null
  added: '2019-07-30T09:21:18.055457+01:00'
  based_on: null
  creator: []
  dataset: Binary black-hole surrogate waveform catalog
  name: null
  path: ''
  url:
  - https
  - zenodo.org
  - /api/files/f6c6c4f3-f63e-43fd-a16b-5629af025bac/remnant_fits/fit_3dq8.h5
  - ''
  - ''
  - ''

Perhaps this gets corrupted somewhere?

@rokroskar
Copy link
Member

I still see the same issue with the url field of the files with a blank path.

Copy link
Member

@rokroskar rokroskar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, seems to work perfectly now!

@jsam jsam merged commit 8cd061b into master Nov 14, 2019
@jsam jsam deleted the 781_bugfix branch November 14, 2019 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cli: renku migrate datasets fails

4 participants