Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PostgreSQL COPY ... FROM STDIN throws a syntax error #492

Closed
iamalryz opened this issue Dec 28, 2020 · 1 comment · Fixed by #495
Closed

PostgreSQL COPY ... FROM STDIN throws a syntax error #492

iamalryz opened this issue Dec 28, 2020 · 1 comment · Fixed by #495

Comments

@iamalryz
Copy link

Describe the Bug
I'm trying to apply a migration with COPY ... FROM STDIN command inside and get a syntax error. But actually, there is no problem with SQL syntax, because I can apply the same migration manually or with other tools like FlyWay.

Steps to Reproduce
An environment described below present in this archive: test.tar.gz.

  1. Use this docker-compose manifest:

    ---
    version: '3.3' 
    
    services:
      db:
        container_name: db
        hostname: db
        image: postgres:12
        ports: 
          - 5432:5432
        environment:
          - POSTGRES_DB=test
          - POSTGRES_PASSWORD=postgres
        networks:
          - migrate
    
      migrate:
        container_name: migrate
        hostname: migrate
        image: migrate/migrate:v4.14.1
        entrypoint: ''
        command: ['sh', '-c', 'sleep 5 && migrate -path=/sql -database=postgres://postgres:postgres@db/test?sslmode=disable up']
        volumes:
          - ./sql:/sql
        depends_on:
          - db
        networks:
          - migrate
    
    networks:
      migrate:
        driver: bridge
    
    
  2. Put these migration files into sql folder near the docker-compose manifest:

    • 1_schema.up.sql:
      create table sample (
          id int primary key,
          text varchar(64) not null unique
      ); 
      
    • 2_data.up.sql:
      COPY sample (id, text) FROM stdin;
      10  Some text 1
      20  Some text 2
      30  Some text 3
      \.
      
      
  3. Run the docker-compose up -d command and see logs from the migrate container: docker-compose logs migrate and you will see the next error:

    Attaching to migrate
    migrate    | 1/u schema (25.207327ms)
    migrate    | error: migration failed: syntax error at or near "10" (column 1) in line 2: COPY sample (id, text) FROM stdin;
    migrate    | 10 Some text 1
    migrate    | 20 Some text 2
    migrate    | 30 Some text 3
    migrate    | \.
    migrate    |  (details: pq: syntax error at or near "10")
    

Expected Behavior
Migration file with COPY ... FROM STDIN should be applied successfully.

Migrate Version
v4.14.1

Loaded Source Drivers

  • go-bindata
  • godoc-vfs
  • gcs
  • file
  • bitbucket
  • github
  • github-ee
  • gitlab, s3

Loaded Database Drivers

  • mongodb
  • mysql
  • sqlserver
  • stub
  • clickhouse
  • crdb-postgres
  • firebird
  • postgresql
  • redshift
  • cassandra
  • postgres
  • cockroach
  • cockroachdb
  • mongodb+srv
  • neo4j
  • spanner

Go Version
There is no Go installed inside the official Docker image used.

Stacktrace
There are no stacktraces.

Additional context
There is no additional context.

Is it a bug or I do something wrong in migrate usage?

@dhui
Copy link
Member

dhui commented Dec 29, 2020

I'm not sure how flyway works with stdin and migration files. It looks like the migration data is being passed as a "statement". Unfortunately, the migrate postgres db driver doesn't support multiple statements, so if my understanding is correct, COPY ... FROM STDIN won't work.
Feel free to open a PR to add support for x-multi-statement. You can use the neo4j, cassandra, and clickhouse db drivers as examples.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants