Extract metadata for ims scorm cp #4258

manavagr1108 · 2023-08-21T22:27:20Z

This code aims to extract metadata from IMS content package.
This Pr is a part of GSoC Project linked with the issue #4081.

Summary

Description of the change(s) you made

Check if imsmanifest.xml is present in file or not
Metadata may also be present in imsmetadata.xml we need to check this file as well
Extract metadata through extractIMSMetadata

Manual verification steps performed

IMS content package is a zip file and these format presents contains .zip format:
- QTI
- HTML5_DEPENDENCY
- HTML5_ZIP
Use jszip to load the uploaded ZIP file asynchronously.
Once loaded, parse imsmanifest.xmland imsmetadata.xml file and read the file as text.
Then parse the text to json so that we can extract the data.
Example of metadata:

{
  title,
  description,
  language,
  folders: [
    {
      title,
      files: [
        {
          identifierref,
          resourceHref,
          title,
        },
        {
          identifierref,
          resourceHref,
          title,
        }
      ]
    },
    {
      title,
      files: [
        {
          identifierref,
          resourceHref,
          title,
        },
      ],
    },
  ],
}

Once metadata is extracted, map the topic node to the resource node in frontend.
We use extra_fields to map topic node with resource node and render in frontend.

Comments

Checking for sub manifest files present in content packe is yet to be implemented.

Contributor's Checklist

PR process:

If this is an important user-facing change, PR or related issue the CHANGELOG label been added to this PR. Note: items with this label will be added to the CHANGELOG at a later time
If this includes an internal dependency change, a link to the diff is provided
The docs label has been added if this introduces a change that needs to be updated in the user docs?
If any Python requirements have changed, the updated requirements.txt files also included in this PR
Opportunities for using Google Analytics here are noted
Migrations are safe for a large db

Studio-specifc:

All user-facing strings are translated properly
The notranslate class been added to elements that shouldn't be translated by Google Chrome's automatic translation feature (e.g. icons, user-generated text)
All UI components are LTR and RTL compliant
Views are organized into pages, components, and layouts directories as described in the docs
Users' storage used is recalculated properly on any changes to main tree files
If there new ways this uses user data that needs to be factored into our Privacy Policy, it has been noted.

Testing:

Code is clean and well-commented
Contributor has fully tested the PR manually
If there are any front-end changes, before/after screenshots are included
Critical user journeys are covered by Gherkin stories
Any new interactions have been added to the QA Sheet
Critical and brittle code paths are covered by unit tests

Reviewer's Checklist

This section is for reviewers to fill out.

Automated test coverage is satisfactory
PR is fully functional
PR has been tested for accessibility regressions
External dependency files were updated if necessary (yarn and pip)
Documentation is updated
Contributor is in AUTHORS.md

- Create topic and resouce node for the metadata

- adding test cases for extractIMSMetadata

rtibbles

Some minor tweaks needed, but this is looking very good!

Note: we will hold off merge until unstable has been released, so that we can release the H5P metadata extraction sooner, then this will be merged and released in a later release!

rtibbles · 2023-09-05T17:06:57Z

contentcuration/contentcuration/frontend/channelEdit/components/edit/EditListItems.vue

+      @input="trackSelect"
+      @removed="handleRemoved"
+    />
+    <div v-if="getChildren !== undefined">


Should change this to a length check on the array!

rtibbles · 2023-09-05T17:08:31Z

contentcuration/contentcuration/frontend/channelEdit/components/edit/EditModal.vue

-          });
+          } else if (file.metadata.folders) {
+            this.createNode('topic', file.metadata).then(newNodeId => {
+              file.metadata.folders.forEach(org => {


Update org here to folder.

rtibbles · 2023-09-05T17:08:44Z

contentcuration/contentcuration/frontend/channelEdit/components/edit/EditModal.vue

+            this.createNode('topic', file.metadata).then(newNodeId => {
+              file.metadata.folders.forEach(org => {
+                this.createNode('topic', org, newNodeId).then(topicNodeId => {
+                  org.files.forEach(orgFile => {


orgFile to folderFile

rtibbles · 2023-09-05T17:10:18Z

contentcuration/contentcuration/frontend/channelEdit/components/edit/EditModal.vue

+                      return File.uploadUrl({
+                        checksum: file.checksum,
+                        size: file.file_size,
+                        type: 'application/zip',


Let's double check if we need to specify this explicitly, or if it should be inferred by existing functionality.

rtibbles · 2023-09-05T17:15:23Z

contentcuration/contentcuration/frontend/channelEdit/components/edit/EditModal.vue

+                          total: file.size,
+                        };
+                        if (index === 0) {
+                          this.selected = [resourceNodeId];


Let's change this to set this.selected if we haven't already set this.selected to something - so only the first finalized node gets selected.

rtibbles · 2023-09-05T17:15:58Z

contentcuration/contentcuration/frontend/shared/vuex/file/__tests__/module.spec.js

+        });
+      });
+      it('extractIMSMetadata should extract metadata from imsmanifest.xml', async () => {
+        // const manifestFile = get_imsmanifest_file({ title: 'Test file' });


Clean up comments!

rtibbles · 2023-09-05T17:18:48Z

contentcuration/contentcuration/frontend/shared/vuex/file/__tests__/module.spec.js

+            <imsmd:lom>
+              <imsmd:general>
+                <imsmd:title>
+                  <imsmd:langstring xml:lang="en">Test File</imsmd:langstring>


lang should be und here.

rtibbles · 2023-09-05T17:21:17Z

contentcuration/contentcuration/frontend/shared/vuex/file/utils.js

+      ) {
+        metadata.language = xmlDoc
+          .getElementsByTagName('lomes:idiom')[0]
+          .children[0].textContent.replace(/ {2}|\r\n|\n|\r/gm, '');


Just double check whether trim will do the same job here!

The assertions in the tests should validate that this is working properly!

rtibbles · 2023-09-05T17:24:43Z

contentcuration/contentcuration/frontend/shared/vuex/file/utils.js

+          .getElementsByTagName('lomes:idiom')[0]
+          .textContent.replace(/ {2}|\r\n|\n|\r/gm, '') !== 'und'
+      ) {
+        metadata.language = xmlDoc


For language extraction let's replicate the validation we are doing in H5P to check this is a supported language code.

Can also add some more tests for unhappy paths!

rtibbles · 2023-09-05T17:31:40Z

contentcuration/contentcuration/frontend/shared/vuex/file/utils.js

+const IMS_PRESETS = [
+  FormatPresetsNames.QTI,
+  FormatPresetsNames.HTML5_DEPENDENCY,
+  FormatPresetsNames.HTML5_ZIP,


Note for me - I may need to consider an IMSCP preset format.

…ith trim

rtibbles · 2023-09-27T15:05:21Z

contentcuration/contentcuration/frontend/shared/vuex/file/utils.js

-        xmlDoc
-          .getElementsByTagName('lomes:idiom')[0]
-          .textContent.replace(/ {2}|\r\n|\n|\r/gm, '') !== 'und'
+        LanguagesMap.has(xmlDoc.getElementsByTagName('lomes:idiom')[0].textContent.trim()) &&


Could avoid a bit of repetition here and define outside of the if statement:

const language = xmlDoc.getElementsByTagName('lomes:idiom').length ? xmlDoc.getElementsByTagName('lomes:idiom')[0].textContent.trim() : 'und';

(defaulting to the disallowed und)

then you can do the checks and assignment against this value instead.

- extract metadata from IMS cp

ba1e0d4

- Create topic and resouce node for the metadata

manavagr1108 force-pushed the extract-metadata-for-IMS-SCORM-cp branch 3 times, most recently from cb939ec to 33b1680 Compare August 22, 2023 23:33

- Enhance IMS Content Package Import UI

90102de

manavagr1108 force-pushed the extract-metadata-for-IMS-SCORM-cp branch from 33b1680 to 90102de Compare August 22, 2023 23:36

- adding code to extract metadata from submanifest file

606b300

- adding test cases for extractIMSMetadata

rtibbles requested changes Sep 5, 2023

View reviewed changes

manavagr1108 mentioned this pull request Sep 10, 2023

Facilitating H5P and SCORM import in Kolibri Studio #4081

Open

- changing metadata key names, removing comments, replacing replace w…

6c94668

…ith trim

manavagr1108 force-pushed the extract-metadata-for-IMS-SCORM-cp branch from 2db6a02 to 6c94668 Compare September 19, 2023 12:46

rtibbles reviewed Sep 27, 2023

View reviewed changes

rtibbles mentioned this pull request Oct 20, 2023

Add IMS Content Package support to the HTML5 Viewer learningequality/kolibri#11436

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extract metadata for ims scorm cp #4258

Extract metadata for ims scorm cp #4258

manavagr1108 commented Aug 21, 2023 •

edited

rtibbles left a comment

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 5, 2023

rtibbles Sep 27, 2023

Extract metadata for ims scorm cp #4258

Are you sure you want to change the base?

Extract metadata for ims scorm cp #4258

Conversation

manavagr1108 commented Aug 21, 2023 • edited

Summary

Description of the change(s) you made

Manual verification steps performed

Comments

Contributor's Checklist

Reviewer's Checklist

This section is for reviewers to fill out.

rtibbles left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

manavagr1108 commented Aug 21, 2023 •

edited