Towards handling Import[xxx, "ZIP", member-name]
#1846
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,40 @@ | ||
| r"""Import/Export File Formats, Importers and Exporters | ||
|
|
||
| The data of files on a filesystem or retrieved from the Internet often are structured \ | ||
| according to a specific structures and rules. For example, consider different kinds of \ | ||
| structuring used in a JSON file, versus an HTML files, or a compressed GZIP file. | ||
|
|
||
| In some cases, such as archive files, e.g., ZIP, TAR, and JAR, the file contains component parts, \ | ||
| which in WMA terminology are called "members" which is part of the broader metadata items \ | ||
| called "elements". | ||
|
|
||
| A MIME type is typically associated with each kind of format. \Mathics3, following WMA, \ | ||
| uses a shortend name for this MIME type. For example \Mathics3 uses "HTML" as a shorthand \ | ||
| for the MIME type "text/html". | ||
|
|
||
| Below is a list of file supported file types that we have builtin importers or exporters written \ | ||
| in Python. Other importers, however, are written in \Mathics3. | ||
|
|
||
| Variable <url> | ||
| :\$ExportFormats: | ||
| /doc/reference-of-built-in-symbols/inputoutput-files-and-filesystem/importing-and-exporting/\$exportformats</url> \ | ||
| contains a list of file formats that are supported by <url> | ||
| :Export: | ||
| /doc/reference-of-built-in-symbols/inputoutput-files-and-filesystem/importing-and-exporting/export</url>, \ | ||
| while <url> | ||
| :\$ImportFormats: | ||
| /doc/reference-of-built-in-symbols/inputoutput-files-and-filesystem/importing-and-exporting/\$importformats</url> \ | ||
| does the corresponding thing for <url> | ||
| :Import: | ||
| /doc/reference-of-built-in-symbols/inputoutput-files-and-filesystem/importing-and-exporting/import</url>. | ||
|
|
||
| Many Import/Export functions are registered in SystemFiles/Formats/*.wl which is \ | ||
| autoloaded on startup. | ||
|
|
||
| The Built-in Functions are defined in a separate context. | ||
| For example, HTML` or Compress`. This is done to not pollute the System` namespace. | ||
| """ | ||
|
|
||
| # This tells documentation how to sort this module | ||
| # Here we are also hiding "file_io" since this can erroneously appear at the top level. | ||
| sort_order = "mathics.builtin.importing-export-file-formats" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| """ | ||
| Compression & Archive Formats | ||
| """ | ||
|
|
||
| from mathics.core.builtin import Builtin, String | ||
| from mathics.core.evaluation import Evaluation | ||
| from mathics.eval.fileformats.compression import eval_ImportZIP | ||
|
|
||
| # See commit in __init__.py regarding the whacky way this gets called | ||
|
|
||
|
|
||
| class ImportZIP(Builtin): | ||
| """ | ||
| <url>:WMA link:https://reference.wolfram.com/language/ref/format/ZIP.html</url> | ||
|
|
||
| <dl> | ||
| <dt>'Compress`ImportZIP[path]' | ||
| <dd>Run zip for archive file $path$ | ||
| </dl> | ||
|
|
||
| """ | ||
|
|
||
| context = "Compress`" | ||
| summary_text = "import a ZIP file" | ||
|
|
||
| def eval(self, path: String, evaluation: Evaluation): | ||
| "Compress`ImportZIP[path_String]" | ||
| return eval_ImportZIP(path, evaluation) | ||
|
|
||
| def eval_with_elements(self, path: String, elements, evaluation: Evaluation): | ||
| "Compress`ImportZIP[path_String, elements_]" | ||
| return eval_ImportZIP(path, evaluation, elements) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,32 +1,29 @@ | ||
| """ | ||
| JSON File Format | ||
|
|
||
| JSON importer (via Python's "json" module). | ||
| """ | ||
|
|
||
| from mathics.core.builtin import Builtin, String | ||
| from mathics.core.evaluation import Evaluation | ||
| from mathics.eval.fileformats.jsonformat import eval_JSONImport | ||
|
|
||
|
|
||
| class ImportJSON(Builtin): | ||
| """ | ||
| <url>:WMA link:https://reference.wolfram.com/language/ref/format/JSON.html</url> | ||
|
|
||
| <dl> | ||
| <dt>'JSON`ImportJSON[path]' | ||
| <dd>Read $path$ as JSON and convert that to its corresponding Mathics3 equivalent. | ||
| </dl> | ||
|
|
||
| """ | ||
|
|
||
| context = "JSON`" | ||
| messages = {"dec": "Decoding Error at `1`"} | ||
| summary_text = "import JSON file" | ||
|
|
||
| def eval(self, path: String, evaluation: Evaluation): | ||
| "JSON`ImportJSON[path_String]" | ||
| return eval_JSONImport(path, evaluation) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mmatera Is there a better way to combine this with the
def eval()above?Even if that is the case, it might be useful to have this broken out as a stub for when this is revised to be able to handle the element passed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not using the parameter
element. In a proper implementation, this function should be more general thanself.eval. Also,parse_htmlshould have an extra attribute to filter a given element.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is exactly what FIXME says.
Yep. Revising HTML and XML is left for later. I will be happy when we are able to "Import" and extract a JSON file from a ZIP import which is needed for being able to install paclets from the public paclet server.
This is the main reason why any work on this is currently being done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, but the output with the second element produce something different.
In the Mathics3 master branch, this seems to work:
So I do not see what this new method provides.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see that. Below, I see lots of output, and I believe this is the same that you'd get from Wolframscript. The formatting of the text is different, but that's to be expected until we match up StandardForm output better.
If there is a specific difference, exactly what's different?
(Please try to give a small example of a difference.)
It is a placeholder function (that indicates FIXME) and it is there to indicate that it should be filled out to remove gross inefficiency that can arise by reading in lots of stuff and then throwing away or filtering most of it.
Instead, that code should be filled out to pass information to other eval routines that handle element retrieval in a better way.