JSONB Array in existing schema stops app loading #8770

berts83231 · 2021-10-13T20:58:44Z

Preflight Checklist

I have completed all Troubleshooting Steps.
I'm on the latest version of Directus.
There's no other issue that already describes my problem.

Describe the Bug

I have an existing app and existing database that I'm trying to use with Directus.

One of the fields is a JSONB array field.

Directus throws the following errors when trying to load the login page.

20:08:11 ✨ Initializing bootstrap...
20:08:11 ✨ Database already initialized, skipping install
20:08:11 ✨ Running migrations...
20:08:11 ✨ Done
20:08:13 ⚠️  PUBLIC_URL should be a full URL
20:08:13 ⚠️  PostGIS isn't installed. Geometry type support will be limited.
20:08:13 ✨ Server started at http://localhost:8055
20:09:24 ✨ request completed GET 302 / 4ms
20:09:24 ✨ request completed GET 304 /admin 3ms
20:09:24 ✨ request completed GET 304 /admin/assets/vendor.de5758fd.css 4ms
20:09:24 ✨ request completed GET 304 /admin/index.b9638729.js 2ms
20:09:24 ✨ request completed GET 304 /admin/assets/index.3086355a.css 2ms
20:09:24 ✨ request completed GET 304 /admin/assets/vendor.90d6ad72.js 2ms
20:09:24 ✨ request completed GET 200 /admin/assets/use-sync.44b55f8e.js 3ms
20:09:24 ✨ request completed GET 304 /admin/assets/vendor.90d6ad72.js 2ms
20:09:24 ✨ request completed GET 304 /admin/assets/use-sync.44b55f8e.js 2ms
20:09:24 🚨 Unexpected token A in JSON at position 0 
SyntaxError: Unexpected token A in JSON at position 0
    at JSON.parse (<anonymous>)
    at parseDefaultValue (/directus/node_modules/knex-schema-inspector/dist/dialects/postgres.js:80:21)
    at Postgres.<anonymous> (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:114:89)
    at step (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:67:23)
    at Object.next (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:48:53)
    at fulfilled (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:39:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
20:09:24 ✨ request errored GET 500 /extensions/interfaces/index.js 85ms
20:09:24 🚨 Unexpected token A in JSON at position 0 
SyntaxError: Unexpected token A in JSON at position 0
    at JSON.parse (<anonymous>)
    at parseDefaultValue (/directus/node_modules/knex-schema-inspector/dist/dialects/postgres.js:80:21)
    at Postgres.<anonymous> (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:114:89)
    at step (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:67:23)
    at Object.next (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:48:53)
    at fulfilled (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:39:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
20:09:24 ✨ request errored POST 500 /auth/refresh 94ms
20:09:24 🚨 Unexpected token A in JSON at position 0 
SyntaxError: Unexpected token A in JSON at position 0
    at JSON.parse (<anonymous>)
    at parseDefaultValue (/directus/node_modules/knex-schema-inspector/dist/dialects/postgres.js:80:21)
    at Postgres.<anonymous> (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:114:89)
    at step (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:67:23)
    at Object.next (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:48:53)
    at fulfilled (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:39:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
20:09:24 ✨ request errored GET 500 /extensions/displays/index.js 93ms
20:09:24 🚨 Unexpected token A in JSON at position 0 
SyntaxError: Unexpected token A in JSON at position 0
    at JSON.parse (<anonymous>)
    at parseDefaultValue (/directus/node_modules/knex-schema-inspector/dist/dialects/postgres.js:80:21)
    at Postgres.<anonymous> (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:114:89)
    at step (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:67:23)
    at Object.next (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:48:53)
    at fulfilled (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:39:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
20:09:24 ✨ request errored GET 500 /extensions/layouts/index.js 98ms
20:09:24 🚨 Unexpected token A in JSON at position 0 
SyntaxError: Unexpected token A in JSON at position 0
    at JSON.parse (<anonymous>)
    at parseDefaultValue (/directus/node_modules/knex-schema-inspector/dist/dialects/postgres.js:80:21)
    at Postgres.<anonymous> (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:114:89)
    at step (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:67:23)
    at Object.next (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:48:53)
    at fulfilled (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:39:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
20:09:24 ✨ request errored GET 500 /extensions/modules/index.js 99ms
20:09:24 🚨 Unexpected token A in JSON at position 0 
SyntaxError: Unexpected token A in JSON at position 0
    at JSON.parse (<anonymous>)
    at parseDefaultValue (/directus/node_modules/knex-schema-inspector/dist/dialects/postgres.js:80:21)
    at Postgres.<anonymous> (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:114:89)
    at step (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:67:23)
    at Object.next (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:48:53)
    at fulfilled (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:39:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
20:09:24 ✨ request errored GET 500 /extensions/panels/index.js 115ms
20:09:24 🚨 Unexpected token A in JSON at position 0 
SyntaxError: Unexpected token A in JSON at position 0
    at JSON.parse (<anonymous>)
    at parseDefaultValue (/directus/node_modules/knex-schema-inspector/dist/dialects/postgres.js:80:21)
    at Postgres.<anonymous> (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:114:89)
    at step (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:67:23)
    at Object.next (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:48:53)
    at fulfilled (/directus/node_modules/@directus/schema/dist/dialects/postgres.js:39:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
20:09:24 ✨ request errored GET 500 /server/info?limit=-1 47ms

I do not need access to the table or field in Directus.

But I do need Directus to run even if the field type is not supported.

The table does not have a primary key, so Directus ignores it anyway.

⚠️ Collection "oban_jobs" doesn't have a primary key column and will be ignored

p.s I'm blown away with Directus so far. I love the fact I can layer it on top of an existing application.

To Reproduce

Run the latest Directus and Postgres 14 in Docker
Set CACHE_ENABLED: "false" and CACHE_SCHEMA: "false"
Import sql table (see below)
Start Directus, open login page and view logs in Docker

If you delete the field "errors". The login page will load.

SQL to create table with problem jsonb array field

CREATE TABLE public.oban_jobs (
    id bigint NOT NULL,
    queue text DEFAULT 'default'::text NOT NULL,
    worker text NOT NULL,
    args jsonb DEFAULT '{}'::jsonb NOT NULL,
    errors jsonb[] DEFAULT ARRAY[]::jsonb[] NOT NULL,
    attempt integer DEFAULT 0 NOT NULL,
    max_attempts integer DEFAULT 20 NOT NULL,
    inserted_at timestamp without time zone DEFAULT timezone('UTC'::text, now()) NOT NULL,
    scheduled_at timestamp without time zone DEFAULT timezone('UTC'::text, now()) NOT NULL,
    attempted_at timestamp without time zone,
    completed_at timestamp without time zone,
    attempted_by text[],
    discarded_at timestamp without time zone,
    priority integer DEFAULT 0 NOT NULL,
    tags character varying(255)[] DEFAULT ARRAY[]::character varying[],
    meta jsonb DEFAULT '{}'::jsonb,
    cancelled_at timestamp without time zone,
    CONSTRAINT attempt_range CHECK (((attempt >= 0) AND (attempt <= max_attempts))),
    CONSTRAINT positive_max_attempts CHECK ((max_attempts > 0)),
    CONSTRAINT priority_range CHECK (((priority >= 0) AND (priority <= 3))),
    CONSTRAINT queue_length CHECK (((char_length(queue) > 0) AND (char_length(queue) < 128))),
    CONSTRAINT worker_length CHECK (((char_length(worker) > 0) AND (char_length(worker) < 128)))
);

What version of Directus are you using?

9.0.0-rc.96 (Docker: directus/directus:latest)

What version of Node.js are you using?

16.10.0 (Docker: directus/directus:latest)

What database are you using?

Postgres 14 (Docker: postgres:latest)

What browser are you using?

Chrome

What operating system are you using?

Linux Manjaro

How are you deploying Directus?

Locally via Docker

The text was updated successfully, but these errors were encountered:

rijkvanzanten · 2021-10-13T21:04:51Z

It seems to fail in the step where knex-schema-inspector tries reading the default value as JSON, as it wrongly recognizes JSONB as JSON.

It's technically a bug in knex-schema-inspector but we can leave this open as a reminder to fix it there 👍🏻

rijkvanzanten · 2021-10-13T21:05:30Z

https://github.com/knex/knex-schema-inspector/blob/0f5f9835ef93a076b6f1ee582f300a332d723727/lib/dialects/postgres.ts#L73

This bit needs a try/catch, or explicitly check against JSONB vs JSON

ntma · 2021-10-14T13:20:59Z

I can try to solve this in the knex-schema-inspector repo!

rijkvanzanten · 2021-10-14T14:01:01Z

Much appreciated @ntma!

rijkvanzanten · 2021-12-03T01:16:36Z

We'll keep tracking this in knex/knex-schema-inspector#72 👍🏻

berts83231 added the Bug (Potential) label Oct 13, 2021

rijkvanzanten added API 🍰 Bug and removed Bug (Potential) labels Oct 13, 2021

rijkvanzanten added Good First Issue labels Oct 13, 2021

ntma mentioned this issue Oct 14, 2021

Uncaught error when retrieving columns info with jsonb[] with default values knex/knex-schema-inspector#72

Closed

rijkvanzanten removed the Hacktoberfest label Nov 2, 2021

rijkvanzanten closed this as completed Dec 3, 2021

rijkvanzanten added External and removed API 🍰 Bug labels Dec 3, 2021

github-actions bot locked as resolved and limited conversation to collaborators Feb 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JSONB Array in existing schema stops app loading #8770

JSONB Array in existing schema stops app loading #8770

berts83231 commented Oct 13, 2021

rijkvanzanten commented Oct 13, 2021

rijkvanzanten commented Oct 13, 2021

ntma commented Oct 14, 2021

rijkvanzanten commented Oct 14, 2021

rijkvanzanten commented Dec 3, 2021

JSONB Array in existing schema stops app loading #8770

JSONB Array in existing schema stops app loading #8770

Comments

berts83231 commented Oct 13, 2021

Preflight Checklist

Describe the Bug

To Reproduce

What version of Directus are you using?

What version of Node.js are you using?

What database are you using?

What browser are you using?

What operating system are you using?

How are you deploying Directus?

rijkvanzanten commented Oct 13, 2021

rijkvanzanten commented Oct 13, 2021

ntma commented Oct 14, 2021

rijkvanzanten commented Oct 14, 2021

rijkvanzanten commented Dec 3, 2021