Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

panic on import #4209

Closed
CaptainStabs opened this issue Aug 26, 2022 · 3 comments · Fixed by #4284
Closed

panic on import #4209

CaptainStabs opened this issue Aug 26, 2022 · 3 comments · Fixed by #4284
Assignees
Labels
bug Something isn't working new format Issues related specifically to the __DOLT__ storage format panic

Comments

@CaptainStabs
Copy link

CaptainStabs commented Aug 26, 2022

Attempting to import this file into the us-housing-prices-v2 database results in the following panic:

>dolt version
    dolt version 0.40.29
    database storage format: NEW ( __DOLT__ )

>dolt table import -u sales test.csv
Warning: There are fewer columns in the import file's schema than the table's schema.
If unintentional, check for any typos in the import file's header.
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x175cb36]

goroutine 434 [running]:
github.com/dolthub/dolt/go/libraries/doltcore/table/untyped/csv.interpretRowSizeError({0x23405e0?, 0xc000bed4f0?}, {0xc06799ecc0, 0x18, 0xc00116c2e2?})
        /src/libraries/doltcore/table/untyped/csv/reader.go:487 +0x676
github.com/dolthub/dolt/go/libraries/doltcore/table/untyped/csv.(*CSVReader).ReadSqlRow(0xc00050d500, {0x5?, 0x23405e0?})
        /src/libraries/doltcore/table/untyped/csv/reader.go:207 +0x2fa
github.com/dolthub/dolt/go/cmd/dolt/commands/tblcmds.moveRows({0x2329350, 0xc067c52e40}, 0xc0009e9500, {0x23297b0, 0xc00050d500}, 0xc000096b00, 0xc0004fdbc0, 0xc067c57950)
        /src/cmd/dolt/commands/tblcmds/import.go:600 +0x14e
github.com/dolthub/dolt/go/cmd/dolt/commands/tblcmds.move.func2()
        /src/cmd/dolt/commands/tblcmds/import.go:557 +0x7a
golang.org/x/sync/errgroup.(*Group).Go.func1()
        /go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
        /go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x8d
        ```

 
@timsehn timsehn added new format Issues related specifically to the __DOLT__ storage format bug Something isn't working labels Aug 26, 2022
@timsehn timsehn added the panic label Aug 31, 2022
@CaptainStabs
Copy link
Author

Same issue on 0.40.32:

Warning: There are fewer columns in the import file's schema than the table's schema.
If unintentional, check for any typos in the import file's header.
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x215d296]

goroutine 722 [running]:
github.com/dolthub/dolt/go/libraries/doltcore/table/untyped/csv.interpretRowSizeError({0x2d54960?, 0xc000647740?}, {0xc071f2c000, 0x18, 0xc000128467?})
        /src/libraries/doltcore/table/untyped/csv/reader.go:487 +0x676
github.com/dolthub/dolt/go/libraries/doltcore/table/untyped/csv.(*CSVReader).ReadSqlRow(0xc0006477a0, {0x5?, 0x2d54960?})
        /src/libraries/doltcore/table/untyped/csv/reader.go:207 +0x2fa
github.com/dolthub/dolt/go/cmd/dolt/commands/tblcmds.moveRows({0x2d399f0, 0xc0720e2880}, 0xc0720be540, {0x2d39e88, 0xc0006477a0}, 0xc00017e000, 0xc000f02900, 0xc072124660)
        /src/cmd/dolt/commands/tblcmds/import.go:600 +0x14e
github.com/dolthub/dolt/go/cmd/dolt/commands/tblcmds.move.func2()
        /src/cmd/dolt/commands/tblcmds/import.go:557 +0x7a
golang.org/x/sync/errgroup.(*Group).Go.func1()
        /go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
        /go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x8d
     ```

@timsehn
Copy link
Sponsor Contributor

timsehn commented Sep 7, 2022

@jcor11599 has a PR out to fix the panic for this one.

@jycor
Copy link
Contributor

jycor commented Sep 8, 2022

Fix is merged and is in release that came out earlier today

Just a heads up, I think the csv itself has some issues

$ dolt table import -u sales test.csv 
Warning: There are fewer columns in the import file's schema than the table's schema.
If unintentional, check for any typos in the import file's header.
Rows Processed: 0, Additions: 0, Modifications: 0, Had No Effect: 0
A bad row was encountered while moving data.
Bad Row: [10488170,SINGLE FAMILY RESIDENTIAL,20360 ARDORE LN,ESTERO,33928,1,2005,2017-12-08 00:00:00,417000,LEE COUNTY,FL,https://www.leepa.org/Display/Displayparcel.aspx?folioID=10488170,<nil>,229691.00000000000,79140.00000000000,0.17600000000,4928.00000000000,2.00000000000,4,3.50000000000,2017000260542,<nil>,<nil>,314407.00000000000]
CSV reader expected 25 values, but saw 24.
row values: '{
        "property_id": "10488170"
        "property_type": "SINGLE FAMILY RESIDENTIAL"
        "property_street_address": "20360 ARDORE LN"
        "property_city": "ESTERO"
        "_property_zip5": "33928"
        "building_num_units": "1"
        "building_year_built": "2005"
        "sale_datetime": "2017-12-08 00:00:00"
        "sale_price": "417000"
        "property_county": "LEE COUNTY"
        "county": "FL"
        "state": "https://www.leepa.org/Display/Displayparcel.aspx?folioID=10488170"
        "source_url": ""
        "appraisal_total": "229691.00000000000"
        "building_assessed_value": "79140.00000000000"
        "land_assessed_value": "0.17600000000"
        "land_area_acres": "4928.00000000000"
        "building_area_sqft": "2.00000000000"
        "building_num_stories": "4"
        "building_num_beds": "3.50000000000"
        "building_num_baths": "2017000260542"
        "sale_id": ""
        "book": ""
        "page": "314407.00000000000"
        "assessed_total": ""
}
'
These can be ignored using '--continue'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working new format Issues related specifically to the __DOLT__ storage format panic
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants