Skip to content

Commit

Permalink
Fix bs4 recursion issue
Browse files Browse the repository at this point in the history
Why these changes are being introduced:
During processing of one set of deleted Alma records, bs4's internal
process was hitting Python's internal recursion limit. Raising the
recursion limit seems to solve the issue without a notable impact on
performance.

How this addresses that need:
* In the helpers module where this issue occured, sets Python's internal
  recursion limit to 10000 instead of the default 1000.

Side effects of this change:
If we ever hit a real infinite loop, it will take longer for Python to
crash.
  • Loading branch information
hakbailey committed Mar 7, 2023
1 parent 3ab9bf6 commit 558ec90
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions transmogrifier/helpers.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import json
import logging
import os
import sys
from datetime import datetime
from typing import Iterator, Optional

Expand All @@ -19,6 +20,9 @@
logger = logging.getLogger(__name__)


sys.setrecursionlimit(10000)


def generate_citation(extracted_data: dict) -> str:
"""Generate a citation in the Datacite schema format.
Expand Down

0 comments on commit 558ec90

Please sign in to comment.