Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: This context cannot be stopped as it never started (extract). #255

Open
fredmajor opened this issue Mar 5, 2018 · 2 comments
Labels

Comments

@fredmajor
Copy link

Pretty often, when I run Bonobo's ETL graph with PyCharm debugger, I get an exception stated in the title. (RuntimeError: This context cannot be stopped as it never started (extract).)
What is strange about the issue, is that it seems to be non-deterministic. Sometimes it throws, sometimes it doesn't (around 50/50 likelihood). When running my code straight from the command line, everything is ok. (At least I haven't encountered the exception yet.

bonobo version  
bonobo v.0.6.1 (in /Users/walkingdude/.pyenv/versions/3.6.2/envs/rb-3.6.2/lib/python3.6/site-packages)
python --version
Python 3.6.2

The way I run the code:

    def load_transcripts(self, data_file_names):
        self.logger.info("Will use Bonobo now, to prepare flat data collection.")
        bonobo.run(
            self.get_graph(data_file_names),
            services=get_services()
        )
        flat_collect_df = pd.DataFrame(flat_dict_list)
        record_count = flat_collect_df.shape[0]
        self.logger.info("Bonobo extracted {} messages out of {} input files."
                         .format(record_count, len(data_file_names)))
        # flat_collect_df.to_csv("output-p1.csv", index=False)
        flat_collect_df = self.enrich_contact_and_user(flat_collect_df, self.connector)
        self.load_chat_data(flat_collect_df, self.connector)

    def get_graph(self, input_files):
        graph = bonobo.Graph()
        graph.add_chain(
            extract(input_files),
            get_conversations,
            get_conversation_parts_with_data,
            filter_parts_without_messages,
            prepare_flat_message_records,
            self.load_to_list,
        )
        return graph

def get_services():
    return {}

    def flat_list(self, context):
        yield flat_dict_list

    @use_context_processor(flat_list)
    def load_to_list(self, flat_list, row):
        flat_df_lock.acquire()
        flat_list.append(row)
        flat_df_lock.release()

Stack trace:

Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1668, in <module>
    main()
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1662, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1072, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/walkingdude/PycharmProjects/rbscience/rapportboost/etl/commands/loadHomebaseIntercomTranscripts.py", line 295, in <module>
    local_run()
  File "/Users/walkingdude/PycharmProjects/rbscience/rapportboost/etl/commands/loadHomebaseIntercomTranscripts.py", line 284, in local_run
    homebase_loader.run()
  File "/Users/walkingdude/PycharmProjects/rbscience/rapportboost/etl/commands/loadHomebaseIntercomTranscripts.py", line 50, in run
    self.load_transcripts(data_file_names)
  File "/Users/walkingdude/PycharmProjects/rbscience/rapportboost/etl/commands/loadHomebaseIntercomTranscripts.py", line 59, in load_transcripts
    services=get_services()
  File "/Users/walkingdude/.pyenv/versions/rb-3.6.2/lib/python3.6/site-packages/bonobo/_api.py", line 67, in run
    return strategy.execute(graph, plugins=plugins, services=services)
  File "/Users/walkingdude/.pyenv/versions/rb-3.6.2/lib/python3.6/site-packages/bonobo/execution/strategies/executor.py", line 45, in execute
    context.stop()
  File "/Users/walkingdude/.pyenv/versions/rb-3.6.2/lib/python3.6/site-packages/bonobo/execution/contexts/graph.py", line 102, in stop
    node_context.stop()
  File "/Users/walkingdude/.pyenv/versions/rb-3.6.2/lib/python3.6/site-packages/bonobo/execution/contexts/node.py", line 169, in stop
    super().stop()
  File "/Users/walkingdude/.pyenv/versions/rb-3.6.2/lib/python3.6/site-packages/bonobo/execution/contexts/base.py", line 96, in stop
    raise RuntimeError('This context cannot be stopped as it never started ({}).'.format(get_name(self)))
RuntimeError: This context cannot be stopped as it never started (extract).
@hartym hartym added the bug label Mar 18, 2018
@hartym hartym added the ready label Oct 27, 2018
@hartym hartym removed the ready label Jun 1, 2019
@bsieber-mozilla
Copy link

Hi, I'm still seeing this issue. No rush, but is there any progress on a release fix? Or any workarounds (to add to run configurations or otherwise)?

Appreciate it!

@NagosCrit
Copy link

Still a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

4 participants