adding support for XML parsing and shale type mapping #48

HassanAkbar · 2024-01-30T12:14:43Z

Adding support for XML parsing and Shale-type mapping.

Fixes #47

HassanAkbar · 2024-01-30T12:49:51Z

@ronaldtse I've tried to make the structure for parsing XML as close to other structures as possible. is this correct or do we need to change something here?

ronaldtse · 2024-01-31T04:39:46Z

@HassanAkbar actually this is not quite what I wanted. We need a way to allow users to "access XML element and attributes".

So the user should provide an XML schema, and also an XML document, like this, and allow the user to access elements:

https://www.shalerb.org/#compiling-json-and-xml-schema

HassanAkbar · 2024-02-06T06:33:36Z

@ronaldtse I don't fully understand what you mean, But I will look into it and get back to you with questions related to this after some research.

HassanAkbar · 2024-02-27T11:20:13Z

@ronaldtse I've looked at https://www.shalerb.org/#compiling-json-and-xml-schema, and figured how shale gem can generate ruby code from schemas provided in xml or json format.

From what I understand the flow is something like,

User provides an xml document. (this point is clear)
User provides the xml schema. (this is unclear, I have few questions related to this)
- How will the user provide the schema? will he pass it as a string parameter or as a file or folder name?
- Are we going to support both xml and json schemas or only xml schemas?
We will return the shale object for the given xml_document.

ronaldtse · 2024-04-02T14:22:06Z

Are we going to support both xml and json schemas or only xml schemas?

We will return the shale object for the given xml_document.

The user will provide both:

XML document
XML schema that the document complies to

Steps:

We pass the XML schema to Shale to obtain a Ruby object that can access the contents of the XML document.
We pass the XML document to the step 1 Shale instance so we can access the content of the document.

How will the user provide the schema? will he pass it as a string parameter or as a file or folder name?

An XSD can be standalone (if it's complete), or it may include other XSDs via includes. So we have to take a file (if it's standalone) and also a folder (if it includes other files in the same folder).

Are we going to support both xml and json schemas or only xml schemas?

We want to support both, but let's finish XML first and do JSON next?

We will return the shale object for the given xml_document.

Yes.

hound · 2024-04-23T12:20:18Z

lib/lutaml/xml/lutaml_path/document_wrapper.rb

+                hash[mapping.name] = content.strip
+              end
+            else
+              if object.class.attributes[mapping.attribute].collection?


Convert if nested inside else to elsif.

hound · 2024-04-23T12:20:18Z

lib/lutaml/xml/lutaml_path/document_wrapper.rb

+          serialize_to_hash(document)
+        end
+
+        def serialize_to_hash(object)


Assignment Branch Condition size for serialize_to_hash is too high. [<11, 45, 16> 49.01/15]
Cyclomatic complexity for serialize_to_hash is too high. [13/6]
Method has too many lines. [25/10]
Perceived complexity for serialize_to_hash is too high. [16/7]

hound · 2024-04-23T12:20:18Z

lib/lutaml/xml/mapper.rb

+          element
+        end
+
+        def get_content_value(content_mapping, element, only, except, delegates, instance)


Avoid parameter lists longer than 5 parameters. [6/5]
Line is too long. [90/80]

hound · 2024-04-23T12:20:18Z

lib/lutaml/xml/mapper.rb

+                    mapper.send(content_mapping.method_to, instance, element,
+                                doc, context)
+                  else
+                    mapper.send(content_mapping.method_to, instance, element, doc)


Line is too long. [82/80]

hound · 2024-04-23T12:20:19Z

lib/lutaml/xml/mapper.rb

+        # @return [::REXML::Document, ::Nokogiri::Document, ::Ox::Document]
+        #
+        # @api public
+        def as_xml(


Avoid parameter lists longer than 5 parameters. [8/5]

hound · 2024-04-23T12:20:19Z

lib/lutaml/xml/parsers/xml.rb

+        def load_schema(schema)
+          result = Shale::Schema.from_xml([schema])
+
+          result.values.each do |klass|


Use each_value instead of values.each.

hound · 2024-04-23T12:20:19Z

lib/lutaml/xml/parsers/xml.rb

+            klass = klass.gsub(/^require.*?\n/, "")
+            klass = klass.gsub(/< Shale::Mapper/, "< Lutaml::Xml::Mapper")
+
+            eval(klass, TOPLEVEL_BINDING)


The use of eval is a serious security risk.

hound · 2024-04-23T12:20:19Z

lib/lutaml/xml/parsers/xml.rb

+        def self.load_schema(schema, root_schema)
+          result = Shale::Schema.from_xml([schema])
+
+          result.values.each do |klass|


Use each_value instead of values.each.

hound · 2024-04-23T12:20:19Z

spec/lutaml/xml/lutaml_path/document_wrapper_spec.rb

+          "Address" => {
+            "City" => "London",
+            "ZIP" => "E1 6AN",
+            "content" => ["Oxford Street"]


Put a comma after the last item of a multiline hash.

hound · 2024-04-23T12:20:19Z

spec/lutaml/xml/lutaml_path/document_wrapper_spec.rb

+
+RSpec.describe Lutaml::Xml::LutamlPath::DocumentWrapper do
+  describe ".parse" do
+    subject(:lutaml_path) { described_class.new(Lutaml::Xml::Parsers::Xml.parse(xml_file_path)) }


Line is too long. [97/80]

hound · 2024-04-23T12:35:54Z

lib/lutaml/xml/lutaml_path/document_wrapper.rb

+            if mapping == "content"
+              attribute_name = object.class.xml_mapping.content.attribute.to_s
+              hash[attribute_name] ||= []
+              hash[attribute_name] << content.strip unless content.strip&.empty?


Avoid calling empty? with the safe navigation operator in conditionals.

hound · 2024-04-23T12:35:54Z

lib/lutaml/xml/lutaml_path/document_wrapper.rb

+          serialize_to_hash(document)
+        end
+
+        def serialize_to_hash(object)


Assignment Branch Condition size for serialize_to_hash is too high. [<11, 45, 15> 48.69/15]
Cyclomatic complexity for serialize_to_hash is too high. [12/6]
Method has too many lines. [25/10]
Perceived complexity for serialize_to_hash is too high. [15/7]

HassanAkbar · 2024-04-23T14:48:17Z

@ronaldtse I have updated this PR. Is this what you were looking for ?

ronaldtse · 2024-05-14T14:21:48Z

lib/lutaml/xml/parsers/xml.rb

+          result.values.each do |klass|
+            # Temporary solution will update in the parser
+            klass = klass.gsub(/^require.*?\n/, "")
+            klass = klass.gsub(/< Shale::Mapper/, "< Lutaml::Xml::Mapper")


We need to fork the gem

@ronaldtse Sure, I'll fork this and open a PR in that repo.

@ronaldtse Should I fork the gem and move everything there or only update this part of the code in the forked gem and keep everything else here?

@ronaldtse Forked the gem at lutaml/shale -> https://github.com/lutaml/shale

adding support for XML parsing and shale type mapping

198e028

HassanAkbar changed the title ~~[WIP] adding support for XML parsing and shale type mapping~~ adding support for XML parsing and shale type mapping Jan 30, 2024

updated to use schema for reading xml data

bec7c04

hound bot reviewed Apr 23, 2024

View reviewed changes

HassanAkbar added 2 commits April 23, 2024 17:24

fixed failing test cases

009dce6

fixing failing tests on ruby >= 3.2

b73b011

hound bot reviewed Apr 23, 2024

View reviewed changes

HassanAkbar added 2 commits April 23, 2024 19:01

fixed failing test cases on github actions

abc01c3

updating rake to latest for ruby 3.2

2a9f38b

HassanAkbar requested a review from ronaldtse April 23, 2024 14:47

ronaldtse reviewed May 14, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

adding support for XML parsing and shale type mapping #48

adding support for XML parsing and shale type mapping #48

HassanAkbar commented Jan 30, 2024

HassanAkbar commented Jan 30, 2024

ronaldtse commented Jan 31, 2024

HassanAkbar commented Feb 6, 2024

HassanAkbar commented Feb 27, 2024

ronaldtse commented Apr 2, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

hound bot Apr 23, 2024

HassanAkbar commented Apr 23, 2024

ronaldtse May 14, 2024

HassanAkbar May 15, 2024

HassanAkbar May 16, 2024

HassanAkbar May 20, 2024

adding support for XML parsing and shale type mapping #48

Are you sure you want to change the base?

adding support for XML parsing and shale type mapping #48

Conversation

HassanAkbar commented Jan 30, 2024

HassanAkbar commented Jan 30, 2024

ronaldtse commented Jan 31, 2024

HassanAkbar commented Feb 6, 2024

HassanAkbar commented Feb 27, 2024

ronaldtse commented Apr 2, 2024

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

hound bot Apr 23, 2024

Choose a reason for hiding this comment

HassanAkbar commented Apr 23, 2024

ronaldtse May 14, 2024

Choose a reason for hiding this comment

HassanAkbar May 15, 2024

Choose a reason for hiding this comment

HassanAkbar May 16, 2024

Choose a reason for hiding this comment

HassanAkbar May 20, 2024

Choose a reason for hiding this comment