NullPointerException when trying to convert MyBibleZone bible that contains footnotes

[Juzujka]

Trying to convert MyBibleZone module to OSIS
with BibleMultiConverter-AllInOneEdition-0.0.7

BibleMultiConverter-AllInOneEdition-0.0.7$ java -jar BibleMultiConverter-AllInOneEdition.jar MyBibleZone ./BTI\'15.SQLite3 OSIS

WARNING: Rebuilding index verses_index on verses
WARNING: Rebuilding index stories_index on stories
WARNING: Unusual footnote mark:  [1]
Exception in thread "main" java.lang.RuntimeException: В НАЧАЛЕ сотворил Бог небо и землю. [1]
at biblemulticonverter.sqlite.format.MyBibleZone.doImport(MyBibleZone.java:292)
at biblemulticonverter.Main.main(Main.java:66)
Caused by: java.lang.NullPointerException
at biblemulticonverter.sqlite.format.MyBibleZone.convertFromVerse(MyBibleZone.java:545)
at biblemulticonverter.sqlite.format.MyBibleZone.doImport(MyBibleZone.java:286)
... 1 more

[schierlm]

Can you please attach both BTI'15.SQLite3 and BTI'15.commentaries.SQLite3 to this ticket? It seems that the latter file, or its index commentaries_index is corrupted, as the SQLite library throws internal errors when trying to access this index. I'd like to have a look at the file, whether it is possible to implement a workaround like try to force a rebuild of the index in this case.

In case you do not need the footnotes, you can work around accessing the -commentaries.SQLite3 file by calling java -Dmybiblezone.footnotes.raw=true -jar BibleMultiConverter-AllInOneEdition.jar MyBibleZone ./BTI\'15.SQLite3 OSIS That way, footnote contents will not be parsed by the MyBibleZone import filter and the OSIS export filter will discard the empty footnotes.

[Juzujka]

yes, of course!
Here is the link
BTI 2015 Современный русский перевод Библии

and attached files too
BTI'15.zip

Thank you for the hint!
But there is another exception:

$ java -Dmybiblezone.footnotes.raw=true -jar BibleMultiConverter-AllInOneEdition.jar MyBibleZone ./BTI\'15/BTI\'15.SQLite3 OSIS > report.txt

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at biblemulticonverter.format.OSIS.doExport(OSIS.java:970)
at biblemulticonverter.Main.main(Main.java:67)

РБО 2015[РБО2] | ★Библия. Современный русский перевод is similar

РСП 2014 | ★Святая Библия: Современный перевод - says

Exception in thread "main" java.lang.IllegalArgumentException: text is invalid: Иисус
говорит о Своей смерти
at biblemulticonverter.data.Utils.validateString(Utils.java:32)
at biblemulticonverter.data.FormattedText$Text.(FormattedText.java:206)
at biblemulticonverter.data.FormattedText$Text.(FormattedText.java:202)
at biblemulticonverter.data.FormattedText$AppendVisitor.visitText(FormattedText.java:681)
at biblemulticonverter.sqlite.format.MyBibleZone.doImport(MyBibleZone.java:346)
at biblemulticonverter.Main.main(Main.java:66)

And lists with warnings in all cases
Warnings like

WARNING: Footnote text [2] not found in 10 1:3
WARNING: Footnote text [2] not found in 10 4:4
WARNING: Footnote text [3] not found in 10 4:4

I try module with English text
the same issue
for NKJV:

...
WARNING: Unusual footnote mark: [34†]
WARNING: Unusual footnote mark: [35†]
WARNING: Unusual footnote mark: [36†]
WARNING: Unusual footnote mark: [37†]
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at biblemulticonverter.format.OSIS.doExport(OSIS.java:970)
at biblemulticonverter.Main.main(Main.java:67)

For NIV 2011★New International Version the result is the same

I run this on Ubuntu 18.04.4 LTS
BibleMultiConverter-AllInOneEdition-0.0.7

[schierlm]

Hello,

Thanks for the details.

Sorry I did not notice you forgot the output file name at the end of the command line. When I add it, BTI'15 converts fine. РСП indeed does not convert, seem to be some control characters amids the Cyrillic letters. Will have to fix properly filtering them.

I can also reproduce the original NullPointerException, when the .commentaries.SQLite3 file is missing or inaccessible. Will add a better error message for that.

[Juzujka]

Thank you!
Now it works!

I think it would be better to change the help message:

java -Dmybiblezone.footnotes.raw=true -jar BibleMultiConverter-AllInOneEdition.jar
Usage:
java -jar BibleMultiConverter.jar [...] there is no output file name, it is not obvious it must be here
java -jar BibleMultiConverter.jar [...]

[schierlm]

Whehter an ExportFormat requires a filename or not depends on the format.

Quoting the full help message:

Usage:
java -jar BibleMultiConverter.jar <ImportFormat> <ImportFile> <ExportFormat> [<ExportArgs>...]
java -jar BibleMultiConverter.jar <Tool> [<ToolArgs>...]

To list the arguments supported/required by a tool or export format, use the help tool:
java -jar BibleMultiConverter.jar help <ExportFormat/Tool>

In particular:

To list the arguments supported/required by a tool or export format, [...]

If you have a concrete suggestion how to make this clearer (without confusing people who use export formats that do not need args), I'm open to suggestions.

On the other hand, the help text for OSIS really needs improving:

OSIS - Very rudimentary OSIS (Open Scripture Information Standard) import/export.

Very rudimentary OSIS (Open Scripture Information Standard) import/export.

When exporting to OSIS, you can pass a comma separated list of tag names you
like to have milestoned in the output file as second parameter. The default
is 'verse', the maximum is 'div,chapter,verse,q'.

It documents the second export parameter (but does not tell what to pass if you do not want any milestoned elements at all), but not the first one. Will fix this too:

Usage (export): OSIS <OutputFile> [-|verse|chapter|div|q][,...]