Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

call function in etl.convert is not working #484

Open
BouchaaraAdil opened this issue Feb 6, 2020 · 3 comments
Open

call function in etl.convert is not working #484

BouchaaraAdil opened this issue Feb 6, 2020 · 3 comments
Labels
Can't Reproduce Needs more information to reproduce or locate the problem

Comments

@BouchaaraAdil
Copy link

Minimal, reproducible code sample, a copy-pastable example if possible

            temp_table1 = etl.convert(temp_table, 'startTimeTS', lambda v, row: self.calc_starttime(row.created_at, row.startTime), pass_row=True)

Problem description

I'm working on data migration from MySQL to Postgres and wanted to convert NULL fields for rows
I coded a separated function that I need to call in etl.convert,

  • calc_starttime is not taking effect at all the, although that it's called but when i verify the data using et.head(..) still column is null

Version and installation information

  • Value of petl==1.3.0
  • Version of Python interpreter Python 3.7.4
  • Operating system (Linux/Windows/Mac) linux
  • How petl was installed (e.g., "using pip into virtual environment", or "using conda"): in django project(Pipfile)
@alimanfoo
Copy link
Collaborator

Hi @BouchaaraAdil, you could try:

temp_table1 = etl.convert(temp_table, 'startTimeTS', lambda v, row: self.calc_starttime(row.created_at, row.startTime), pass_row=True, failonerror=True)

This will cause petl to propagate an except if one is raised by the conversion function.

@alimanfoo
Copy link
Collaborator

The failonerror argument can be False (default), True (raise any exception immediately) or "inline" (use exceptions as values in the table). If you want to set the behaviour globally, you can do:

petl.config.failonerror = True

@juarezr
Copy link
Member

juarezr commented Sep 24, 2020

Reopening: not related to #484

@juarezr juarezr reopened this Sep 24, 2020
@juarezr juarezr added the Can't Reproduce Needs more information to reproduce or locate the problem label Sep 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Can't Reproduce Needs more information to reproduce or locate the problem
Projects
None yet
Development

No branches or pull requests

3 participants