Merge d2baeb0 into 902028f

.gitignore

@@ -25,6 +25,9 @@ var/
 pip-log.txt
 pip-delete-this-directory.txt
 
+# Virtual environments
+**/venv*
+
 # Unit test / coverage reports
 htmlcov/
 .tox/

requirements/python

@@ -5,7 +5,8 @@ argcomplete~=1.10.0
 beautifulsoup4~=4.8.0
 chardet==3.*
 docx2txt~=0.8
-EbookLib==0.*
+epubfile==0.0.8
+  voussoirkit==0.0.70  # Breaking changes are introduced in 0.0.71
 extract-msg<=0.29.* #Last with python2 support
 pdfminer.six==20191110 #Last with python2 support
 python-pptx~=0.6.18

tests/epub/raw_text.txt

@@ -1,11 +1,7 @@
-
 Epub testing
 With subtitle...
-
 Introduction
 Welcome here! All the text have ben generate with the Samuel L lorem ipsum.
-
-
 We happy?
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 We happy?
@@ -16,7 +12,6 @@ No man, I don't eat pork
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 Is she dead, yes or no?
 The path of the righteous man is beset on all sides by the iniquities of the selfish and the tyranny of evil men. Blessed is he who, in the name of charity and good will, shepherds the weak through the valley of darkness, for he is truly his brother's keeper and the finder of lost children. And I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy My brothers. And you will know My name is the Lord when I lay My vengeance upon thee.
-
 We happy?
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 We happy?
@@ -27,7 +22,6 @@ No man, I don't eat pork
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 Is she dead, yes or no?
 The path of the righteous man is beset on all sides by the iniquities of the selfish and the tyranny of evil men. Blessed is he who, in the name of charity and good will, shepherds the weak through the valley of darkness, for he is truly his brother's keeper and the finder of lost children. And I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy My brothers. And you will know My name is the Lord when I lay My vengeance upon thee.
-
 We happy?
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 We happy?
@@ -38,7 +32,6 @@ No man, I don't eat pork
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 Is she dead, yes or no?
 The path of the righteous man is beset on all sides by the iniquities of the selfish and the tyranny of evil men. Blessed is he who, in the name of charity and good will, shepherds the weak through the valley of darkness, for he is truly his brother's keeper and the finder of lost children. And I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy My brothers. And you will know My name is the Lord when I lay My vengeance upon thee.
-
 We happy?
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 We happy?
@@ -49,18 +42,6 @@ No man, I don't eat pork
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 Is she dead, yes or no?
 The path of the righteous man is beset on all sides by the iniquities of the selfish and the tyranny of evil men. Blessed is he who, in the name of charity and good will, shepherds the weak through the valley of darkness, for he is truly his brother's keeper and the finder of lost children. And I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy My brothers. And you will know My name is the Lord when I lay My vengeance upon thee.
-
-We happy?
-Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
-We happy?
-The lysine contingency - it's intended to prevent the spread of the animals is case they ever got off the island. Dr. Wu inserted a gene that makes a single faulty enzyme in protein metabolism. The animals can't manufacture the amino acid lysine. Unless they're continually supplied with lysine by us, they'll slip into a coma and die.
-Oh... what I'm gon' do?
-The path of the righteous man is beset on all sides by the iniquities of the selfish and the tyranny of evil men. Blessed is he who, in the name of charity and good will, shepherds the weak through the valley of darkness, for he is truly his brother's keeper and the finder of lost children. And I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy My brothers. And you will know My name is the Lord when I lay My vengeance upon thee.
-No man, I don't eat pork
-Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
-Is she dead, yes or no?
-The path of the righteous man is beset on all sides by the iniquities of the selfish and the tyranny of evil men. Blessed is he who, in the name of charity and good will, shepherds the weak through the valley of darkness, for he is truly his brother's keeper and the finder of lost children. And I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy My brothers. And you will know My name is the Lord when I lay My vengeance upon thee.
-
 We happy?
 Well, the way they make shows is, they make one show. That show's called a pilot. Then they show that show to the people who make shows, and on the strength of that one show they decide if they're going to make more shows. Some pilots get picked and become television programs. Some don't, become nothing. She starred in one of the ones that became nothing.
 We happy?

textract/parsers/epub_parser.py

@@ -1,5 +1,5 @@
-from ebooklib import epub, ITEM_DOCUMENT
-from bs4 import BeautifulSoup
+# from ebooklib import epub, ITEM_DOCUMENT
+from epubfile import Epub
 
 from .utils import BaseParser
 
@@ -9,17 +9,17 @@ class Parser(BaseParser):
     """
 
     def extract(self, filename, **kwargs):
-        book = epub.read_epub(filename)
+        book = Epub(filename, read_only=True)
         result = ''
-        for id, _ in book.spine:
-            item = book.get_item_with_id(id)
+        for text_name in book.get_texts():
+            soup = book.read_file(text_name, soup=True)
             # Don't fail with some AttributeError exception when the item is of NoneType
             # (i.e. at the last position).
-            if item is None:
+            if soup is None:
                 continue
-            soup = BeautifulSoup(item.content, 'lxml')
-            for child in soup.find_all(
-                ['title', 'p', 'div', 'h1', 'h2', 'h3', 'h4']
-            ):
-                result = result + child.text + '\n'
+            html_content_tags = ['title', 'p', 'h1', 'h2', 'h3', 'h4']
+            for child in soup.find_all(html_content_tags):
+                inner_text = child.text.strip() if child.text else ""
+                if inner_text:
+                    result += inner_text + '\n'
         return result