Data error

[benscott]

Hi -

When I try running ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2 it imports the mbdump.tar.bz2 data, but when it moves to derived...

Importing data from mbdump-derived.tar.bz2

I get an error:

Traceback (most recent call last):
File "./mbslave-import.py", line 49, in
load_tar(filename, db, schema, ignored_tables)
File "./mbslave-import.py", line 27, in load_tar
cursor.copy_from(tar.extractfile(member), fulltable)
psycopg2.DataError: value too long for type character varying(255)
CONTEXT: COPY annotation, line 33312, column changelog: "Name used up to 1997/08/25: 松澤由実 Name used after 1997/08/25: 松澤由美 (real name) Read..."

It seems the string length is too long for the field?

Cheers,

Ben

[lalinsky]

It looks like it tries to import the data into the wrong column ("text" into "changelog"). It seems weird, because there were no changes to the table, but can you open psql console and check the output of '\d annotation' and paste here the line 33312 from the annotation file?

[benscott]

Hi -

Thanks very much for getting back to me so quickly. The output of \d annotation is:

                             Table "musicbrainz.annotation"

Column | Type | Modifiers
-----------+--------------------------+---------------------------------------------------------
id | integer | not null default nextval('annotation_id_seq'::regclass)
editor | integer | not null
text | text |
changelog | character varying(255) |
created | timestamp with time zone | default now()

Cheers,

Ben

[benscott]

Hi -

Sorry, I missed the second part of the data you needed. Here's the line from the annotation file...

192366 285909 Name used up to 1997/08/25: [http://musicbrainz.org/artist/83c9e077-92af-4c45-8d40-d3f8b1608219.html|松澤由実](this entry)\r\nName used after 1997/08/25: [http://musicbrainz.org/artist/a454e56f-4e88-418f-aa70-b12ed0a614c4.html|松澤由美](real name)\r\nReading for both names is まつざわゆみ (Matsuzawa Yumi).\r\nAll releases before 1997/08/25 should go to first entry, and remaining to the second one.\r\nThere are also instances of YUMI and [http://musicbrainz.org/artist/c8bb5d21-2215-40ce-ba5b-515412b49c3d.html|まつざわゆみ]. 2011-04-04 15:34:27.598794+00

Cheers,

Ben

[benscott]

Hi -

Is there any update on this? I've tried installing from scratch with the latest release & am still getting the same error. Has anyone else got this to work?

Cheers,

Ben

[lalinsky]

Sorry, I don't know yet. I'll have to try to do a fresh import today, as I don't see any obvious problem.

[benscott]

Ah, cool - thanks for looking into it.

[lalinsky]

Well, I just downloaded the latest database dump and it imported fine for me. I don't really know what could be wrong. If you unpack the mbdump-derived file and run this, does it give you the same error?

\copy musicbrainz.annotation from '/path/to/mbdump/annotation'

[benscott]

Just tried it & yep, I get exactly the same error.

[lalinsky]

What OS are you using? What version of PostgreSQL? This is really weird, because \copy should be able to import the data. It seems to be misinterpreting some character as a field separator and loading the data into the wrong column.

[benscott]

Hi - It's running on Ubuntu 11.04 & PostgreSQL 8.4.8.