New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation instructions are included in manifold/manifold-docs #245

Closed
atiro opened this Issue Apr 12, 2017 · 5 comments

Comments

Projects
None yet
2 participants
@atiro

atiro commented Apr 12, 2017

Following the build instructions in the README (running ruby 2.3.3) I am trying to ingest the specpubs and get:

spring stop && rake ingest:specpubs

Spring is not running
{:EPUB=>Ingestor::Strategy::EPUB::Strategy, :Word=>Ingestor::Strategy::Word::Strategy, :Gitbook=>Ingestor::Strategy::Gitbook::Strategy, :GoogleDoc=>Ingestor::Strategy::GoogleDoc::Strategy}
I, [2017-04-12T13:57:38.691737 #18354] INFO -- : Ingesting wasteland.epub!
I, [2017-04-12T13:57:38.802123 #18354] INFO -- : Ingestor::Strategy::EPUB::Strategy can ingest? true
I, [2017-04-12T13:57:38.802393 #18354] INFO -- : Using strategy Ingestor::Strategy::EPUB::Strategy
I, [2017-04-12T13:57:38.810508 #18354] INFO -- : No existing text model found for source ID code.google.com.epub-samples.wasteland-basic
I, [2017-04-12T13:57:38.812503 #18354] INFO -- : EPUB version is 3.0
I, [2017-04-12T13:57:38.848429 #18354] INFO -- : New title: "The Waste Land"
I, [2017-04-12T13:57:38.906481 #18354] INFO -- : New creator: "T.S. Eliot"
I, [2017-04-12T13:57:38.910900 #18354] INFO -- : Setting language to "en-US"
I, [2017-04-12T13:57:38.912128 #18354] INFO -- : Setting publication date to "2011-09-01"
I, [2017-04-12T13:57:38.913257 #18354] INFO -- : Setting rights to "This work is shared with the public using the Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) license."
I, [2017-04-12T13:57:38.914326 #18354] INFO -- : Setting description to ""
I, [2017-04-12T13:57:38.914485 #18354] INFO -- : Attempting to save the text
E, [2017-04-12T13:57:38.922525 #18354] ERROR -- : Text : Collaborators is invalid
E, [2017-04-12T13:57:38.922651 #18354] ERROR -- : Text : Creator must exist
E, [2017-04-12T13:57:38.944193 #18354] ERROR -- : Unable to save text due to ingestion errors.
E, [2017-04-12T13:57:38.946794 #18354] ERROR -- : Ingestor::IngestionFailed
... [same for all EPUBs]..

Seems like the creator & collaborator models are not being created even though it discovers them?

@zdavis

This comment has been minimized.

Show comment
Hide comment
@zdavis

zdavis Apr 12, 2017

Member

Hi Atiro. Our docs are a little bit out of date — we'll be circling in the next few days to get these updated! The particular rake task you're using is one we haven't used for a long time, and it should be removed (or updated). Some of these rake tasks were built early on as temporary stop-gaps because we needed to be able to ingest texts before we'd built a backend to handle it.

If you want to ingest a text through the CLI, the best way to do it is to use the import:project rake task. This task expects a directory (we keep ours in /import in the project root) with a project.json file that defines the project and references a text to be ingested. With this approach, the project record will also be created and the text will be nested within the project.

There's a little issue in that Wasteland EPUB — it has some CSS in it's stylesheet that our parser takes issue with. We'll catch the exception there but for now, here's a sample Wasteland project with a corrected EPUB that you can use to test out Manifold's ingestion:

the_wasteland.zip

To get it into Manifold, follow these steps:

  1. Make sure you've seeded the database with rails db:seed. Doing this creates a CLI user, which will be the "creator" of any texts and projects imported through the CLI (I think this was the cause of your error above)

  2. Make a directory in the root of the repo called import and unzip that file above in there. Manifold will expect to find the project.json file at import/the_wasteland/project.json.

  3. From the API directory, execute rails import:project["../import/the_wasteland","debug"]

On the frontend, you should see a new wasteland project with the imported EPUB.

Going forward, the ideal way for people to ingest texts is through the Manifold backend. We haven't documented this yet, but there are mechanisms for creating projects and texts through the backend rather than the CLI. We'll get those docs up soon!

Hope this helps, and thanks for bearing with us while we're still in beta and working out a number of kinks!

--Zach

Member

zdavis commented Apr 12, 2017

Hi Atiro. Our docs are a little bit out of date — we'll be circling in the next few days to get these updated! The particular rake task you're using is one we haven't used for a long time, and it should be removed (or updated). Some of these rake tasks were built early on as temporary stop-gaps because we needed to be able to ingest texts before we'd built a backend to handle it.

If you want to ingest a text through the CLI, the best way to do it is to use the import:project rake task. This task expects a directory (we keep ours in /import in the project root) with a project.json file that defines the project and references a text to be ingested. With this approach, the project record will also be created and the text will be nested within the project.

There's a little issue in that Wasteland EPUB — it has some CSS in it's stylesheet that our parser takes issue with. We'll catch the exception there but for now, here's a sample Wasteland project with a corrected EPUB that you can use to test out Manifold's ingestion:

the_wasteland.zip

To get it into Manifold, follow these steps:

  1. Make sure you've seeded the database with rails db:seed. Doing this creates a CLI user, which will be the "creator" of any texts and projects imported through the CLI (I think this was the cause of your error above)

  2. Make a directory in the root of the repo called import and unzip that file above in there. Manifold will expect to find the project.json file at import/the_wasteland/project.json.

  3. From the API directory, execute rails import:project["../import/the_wasteland","debug"]

On the frontend, you should see a new wasteland project with the imported EPUB.

Going forward, the ideal way for people to ingest texts is through the Manifold backend. We haven't documented this yet, but there are mechanisms for creating projects and texts through the backend rather than the CLI. We'll get those docs up soon!

Hope this helps, and thanks for bearing with us while we're still in beta and working out a number of kinks!

--Zach

@atiro

This comment has been minimized.

Show comment
Hide comment
@atiro

atiro Apr 12, 2017

Thanks Zach, sorry I'm probably jumping the gun a little as keen to play with the system! I'll let you catch your breath first.

(I did get the import to work with rake import:project["../import/the_wastleland", "debug"] but then
got stuck on running the cable service (with a URI parse error). Skipping that, I couldn't then work out what I actually needed to connect to to see the frontend.)

atiro commented Apr 12, 2017

Thanks Zach, sorry I'm probably jumping the gun a little as keen to play with the system! I'll let you catch your breath first.

(I did get the import to work with rake import:project["../import/the_wastleland", "debug"] but then
got stuck on running the cable service (with a URI parse error). Skipping that, I couldn't then work out what I actually needed to connect to to see the frontend.)

@zdavis

This comment has been minimized.

Show comment
Hide comment
@zdavis

zdavis Apr 12, 2017

Member

No worries — we're thankful for anyone who wants to experiment with our work! I'm working through some of these install / setup issues now and am working through the docs. I expect we'll have an update with better docs in the next few days. I'll update this issue when I there's demonstrable progress.

Member

zdavis commented Apr 12, 2017

No worries — we're thankful for anyone who wants to experiment with our work! I'm working through some of these install / setup issues now and am working through the docs. I expect we'll have an update with better docs in the next few days. I'll update this issue when I there's demonstrable progress.

@zdavis

This comment has been minimized.

Show comment
Hide comment
@zdavis

zdavis Apr 13, 2017

Member

Hi Atiro,

Take a look at these updated docs and see if you're able to get up and running by following the steps there:

https://github.com/ManifoldScholar/manifold-docs/blob/master/developers/installation/development.md

Member

zdavis commented Apr 13, 2017

Hi Atiro,

Take a look at these updated docs and see if you're able to get up and running by following the steps there:

https://github.com/ManifoldScholar/manifold-docs/blob/master/developers/installation/development.md

@zdavis

This comment has been minimized.

Show comment
Hide comment
@zdavis

zdavis Apr 14, 2017

Member

I'm closing this issue for now, but please don't hesitate to continue the conversation and we can reopen if necessary.

Member

zdavis commented Apr 14, 2017

I'm closing this issue for now, but please don't hesitate to continue the conversation and we can reopen if necessary.

@zdavis zdavis closed this Apr 14, 2017

@zdavis zdavis added the bug label Jun 20, 2017

@zdavis zdavis changed the title from EPUB ingest fails with collaborator error to Add Installation Documentation to manifold/manifold-docs Jun 20, 2017

@zdavis zdavis added feature and removed bug under discussion labels Jun 20, 2017

@zdavis zdavis changed the title from Add Installation Documentation to manifold/manifold-docs to Installation instructions are included in manifold/manifold-docs Jun 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment