Bug: Dr4ft Crashing on JSON import #1432

ZeldaZach · 2021-02-03T01:53:48Z

Error seen in the logs, that we should investigate:

2021-02-02 11:06:22,924-0500 [INFO] [965382d0-6570-11eb-aff1-1f39dcc7d535] Game 965382d0-6570-11eb-aff1-1f39dcc7d535 started.

    Game State
    ----------
    id: 965382d0-6570-11eb-aff1-1f39dcc7d535
    hostId: 1q1lvbf10cx
    title: KLD
    seats: 1
    type: sealed
    sets: 2XM,2XM,2XM,2XM,2XM,2XM
    isPrivate: true
    picksPerPack: 1
    modernOnly: false
    totalChaos: false
    chaosPacksNumber: undefined
    packsInfos: 2XM / 2XM / 2XM / 2XM / 2XM / 2XM
    players: 1 (Rumo)
    bots: 0
    
2021-02-02 11:06:56,411-0500 [ERROR] [GLOBAL] Could not parse XML file: TypeError: pt.split is not a function or its return value is not iterable - TypeError: pt.split is not a function or its return value is not iterable
    at /var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:79:35
    at Array.forEach (<anonymous>)
    at Object.parse (/var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:51:14)
    at /var/www/dr4ft.info/dr4ft/backend/api/sets.js:25:30
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at /var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:335:12)
2021-02-02 11:06:56,542-0500 [ERROR] [GLOBAL] Could not parse XML file: TypeError: pt.split is not a function or its return value is not iterable - TypeError: pt.split is not a function or its return value is not iterable
    at /var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:79:35
    at Array.forEach (<anonymous>)
    at Object.parse (/var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:51:14)
    at /var/www/dr4ft.info/dr4ft/backend/api/sets.js:25:30
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at /var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:335:12)
2021-02-02 11:07:02,423-0500 [INFO] [GLOBAL] adding new set with code "ADD" to database
2021-02-02 11:07:05,317-0500 [INFO] [GLOBAL] adding new set with code "GYPO" to database
2021-02-02 11:07:05,319-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:07:05,319-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:07:05,319-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:07:05,319-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:07:05,319-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:07:05,319-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:07:05,323-0500 [INFO] [GLOBAL] Saved custom set as file ADD.json
2021-02-02 11:07:05,323-0500 [INFO] [GLOBAL] Saved custom set as file GYPO.json
2021-02-02 11:07:05,360-0500 [ERROR] [GLOBAL] Could not parse XML file: TypeError: pt.split is not a function or its return value is not iterable - TypeError: pt.split is not a function or its return value is not iterable
    at /var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:79:35
    at Array.forEach (<anonymous>)
    at Object.parse (/var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:51:14)
    at /var/www/dr4ft.info/dr4ft/backend/api/sets.js:25:30
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at /var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:335:12)
2021-02-02 11:07:10,350-0500 [ERROR] [GLOBAL] Could not parse XML file: TypeError: pt.split is not a function or its return value is not iterable - TypeError: pt.split is not a function or its return value is not iterable
    at /var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:79:35
    at Array.forEach (<anonymous>)
    at Object.parse (/var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:51:14)
    at /var/www/dr4ft.info/dr4ft/backend/api/sets.js:25:30
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at /var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:335:12)
2021-02-02 11:07:11,542-0500 [ERROR] [GLOBAL] Could not parse XML file: Error: node <cards> must be present - Error: node <cards> must be present
    at Object.parse (/var/www/dr4ft.info/dr4ft/backend/import/xml/parser.js:12:11)
    at /var/www/dr4ft.info/dr4ft/backend/api/sets.js:25:30
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at next (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/layer.js:95:5)
    at /var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:335:12)
    at next (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:275:10)
    at Function.handle (/var/www/dr4ft.info/dr4ft/node_modules/express/lib/router/index.js:174:3)
2021-02-02 11:08:47,220-0500 [INFO] [GLOBAL] user maxi created a game with id ee35d520-6570-11eb-aff1-1f39dcc7d535
2021-02-02 11:08:47,741-0500 [DEBUG] [ee35d520-6570-11eb-aff1-1f39dcc7d535] maxi joined the game
2021-02-02 11:08:53,926-0500 [DEBUG] [ee35d520-6570-11eb-aff1-1f39dcc7d535] new round started
2021-02-02 11:08:53,927-0500 [INFO] [GLOBAL] GameID: ee35d520-6570-11eb-aff1-1f39dcc7d535, Bot, picked: Smoky, perra de guerra
2021-02-02 11:08:53,927-0500 [INFO] [GLOBAL] GameID: ee35d520-6570-11eb-aff1-1f39dcc7d535, Bot, picked: Smoky, perra de guerra
2021-02-02 11:08:53,927-0500 [INFO] [GLOBAL] GameID: ee35d520-6570-11eb-aff1-1f39dcc7d535, Bot, picked: Smoky, perra de guerra

And what probably crashed the service, as there are no more relevant logs after this:

2021-02-02 11:14:56,988-0500 [INFO] [GLOBAL] adding new set with code "GYP20" to database
2021-02-02 11:14:56,988-0500 [INFO] [GLOBAL] Custom set GYPO already existing. Overriding with new file...
2021-02-02 11:14:59,700-0500 [INFO] [GLOBAL] adding new set with code "GYPO" to database
2021-02-02 11:15:01,184-0500 [INFO] [GLOBAL] adding new set with code "LOA" to database
2021-02-02 11:15:03,967-0500 [INFO] [GLOBAL] adding new set with code "LOO" to database
2021-02-02 11:15:05,454-0500 [INFO] [GLOBAL] adding new set with code "GYP1" to database
2021-02-02 11:15:06,940-0500 [INFO] [GLOBAL] adding new set with code "ADD" to database
2021-02-02 11:15:08,554-0500 [INFO] [GLOBAL] adding new set with code "ERAS" to database
2021-02-02 11:15:10,188-0500 [INFO] [GLOBAL] adding new set with code "CDLS" to database
2021-02-02 11:15:10,198-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:15:10,199-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on sets.json
2021-02-02 11:15:10,200-0500 [DEBUG] [GLOBAL] filewatch - change on cards.json
2021-02-02 11:15:10,201-0500 [DEBUG] [GLOBAL] filewatch - change on cubable_cards_by_name.json
2021-02-02 11:15:10,203-0500 [INFO] [GLOBAL] GameID: ee35d520-6570-11eb-aff1-1f39dcc7d535, player maxi, picked: Smoky, perra de guerra
2021-02-02 11:15:10,204-0500 [INFO] [GLOBAL] GameID: ee35d520-6570-11eb-aff1-1f39dcc7d535, Bot, picked: Smoky, perra de guerra
2021-02-02 11:15:10,206-0500 [INFO] [GLOBAL] Saved custom set as file GYP20.json
2021-02-02 11:15:10,206-0500 [INFO] [GLOBAL] Saved custom set as file LOA.json
2021-02-02 11:15:10,206-0500 [INFO] [GLOBAL] Saved custom set as file LOO.json
2021-02-02 11:15:10,206-0500 [INFO] [GLOBAL] Saved custom set as file GYP1.json
2021-02-02 11:15:10,207-0500 [INFO] [GLOBAL] Saved custom set as file ADD.json
2021-02-02 11:15:10,207-0500 [INFO] [GLOBAL] Saved custom set as file CDLS.json
2021-02-02 11:15:10,207-0500 [INFO] [GLOBAL] Saved custom set as file ERAS.json
2021-02-02 11:15:10,207-0500 [INFO] [GLOBAL] Saved custom set as file GYPO.json
2021-02-02 11:15:21,619-0500 [INFO] [GLOBAL] Started up on port 1337 with version 8793772
2021-02-02 11:15:21,894-0500 [INFO] [GLOBAL] Checking boosterRules repository
2021-02-02 11:15:21,983-0500 [INFO] [GLOBAL] Checking if AllSets.json is up to date
2021-02-02 11:15:22,098-0500 [INFO] [GLOBAL] AllSets.json is up to date

The text was updated successfully, but these errors were encountered:

ZeldaZach · 2021-02-03T01:54:19Z

GYP20.json

{
    "code": "GYP20",
    "name": "Inicial 2020",
    "type": "custom",
    "releaseDate": 2020,
    "cards": [],
    "baseSetSize": 0
}

ZeldaZach · 2021-02-03T01:54:45Z

GYPO.json

{
    "code": "GYPO",
    "name": "GYPO Exclusivo",
    "type": "custom",
    "releaseDate": 2020,
    "cards": [
        {
            "name": "Smoky, perra de guerra",
            "number": 0,
            "layout": "normal",
            "colors": [
                "A"
            ],
            "types": [
                "Ciudadana"
            ],
            "supertypes": [],
            "subtypes": [],
            "manaCost": 1,
            "url": "Smoky, perra de guerra.jpg",
            "identifiers": [],
            "rarity": "uncommon",
            "power": 2,
            "toughness": 2,
            "loyalty": "",
            "text": "Influencia (Entra en juego ejerciendo influencia sobre un súbdito.)\nSi el súbdito influenciado es un militar, guerrero o animal, ese súbdito tiene Resistencia.\n3, T, matarla: Hasta el final del turno, tus militares tienen +2/+2."
        }
    ],
    "baseSetSize": 1
}

HerveH44 · 2021-02-07T14:01:12Z

We should really have another way to handle custom sets. We shouldn't save them in the DB but keep them just for the session the game.

mixmix · 2021-02-07T15:41:18Z

I'm a big fan of JSONschema for validating custom inputs. Happy to help someone set that up if validation on this entry point is something useful

I've used is-my-json-valid, but there are newer modules out there which are even faster I think

ZeldaZach · 2021-02-09T00:32:40Z

We had another crash, caused by a user upload. See #1443

ebbit1q · 2021-02-09T01:08:48Z

Seems unlikely to be a json issue, it's more likely to be related to that "url", it seems formatted wrong.

ebbit1q · 2021-02-09T03:42:07Z

after a LOT of trial and error I concluded the cause of the problem is uploading the same card name in two different sets.

Simple repro steps:

Upload a set with the following json: {"code":"TEST1","cards":[{"name":"test","colors":[],"types":[]}]}
Upload a set with the following json: {"code":"TEST2","cards":[{"name":"test","colors":[],"types":[]}]}
reload the page

mixmix · 2021-02-09T04:36:49Z

This is really good to know.
It puts some hard constraints on what can be included with custom sets I imagine... (if this logic remains same).

I'll try and hunt down why this causes a crash after I fix the bugs in the new export options

ebbit1q · 2021-02-09T23:18:57Z

Today I tried to reproduce it using the xml format using this minimal xml:

<cockatrice_carddatabase>
  <cards>
    <card>
      <name>test</name>
      <set rarity="common">TEST1</set>
      <prop></prop>
    </card>
  </cards>
</cockatrice_carddatabase>

But the error didn't happen if I uploaded the same xml with TEST2 as the set afterwards, instead if I then used the json with TEST2 it did break. This means that this bug is related to the json import. Also the step 1 in my previous repro steps can be replaced with this xml, it only breaks on step 2.

(note this isn't a valid xml for cockatrice, just the minimum for dr4ft)

ebbit1q · 2021-02-09T23:37:35Z

I also noticed something else, uploading the same card in the same set with new properties does not update the card or overwrite it, instead just nothing happens and the old card stays.

ZeldaZach added the bug label Feb 3, 2021

ZeldaZach added the high priority label Feb 3, 2021

ZeldaZach mentioned this issue Feb 9, 2021

Bug: Updater crashes the service #1443

Closed

ebbit1q mentioned this issue Feb 9, 2021

build(deps): Bump fast-xml-parser from 3.17.6 to 3.18.0 #1441

Closed

ZeldaZach mentioned this issue Feb 10, 2021

Bug: dr4ft.info page only displaying blank page again #1452

Closed

tooomm mentioned this issue Jun 15, 2022

Cleanup old upload hint #1627

Open

tooomm changed the title ~~Bug: Dr4ft Crashing~~ Bug: Dr4ft Crashing on JSON import Jun 21, 2022

tooomm mentioned this issue Jun 21, 2022

TEMP: disable JSON upload of custom sets #1453

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: Dr4ft Crashing on JSON import #1432

Bug: Dr4ft Crashing on JSON import #1432

ZeldaZach commented Feb 3, 2021

ZeldaZach commented Feb 3, 2021

ZeldaZach commented Feb 3, 2021

HerveH44 commented Feb 7, 2021

mixmix commented Feb 7, 2021

ZeldaZach commented Feb 9, 2021

ebbit1q commented Feb 9, 2021

ebbit1q commented Feb 9, 2021 •

edited

Loading

mixmix commented Feb 9, 2021

ebbit1q commented Feb 9, 2021 •

edited

Loading

ebbit1q commented Feb 9, 2021

Bug: Dr4ft Crashing on JSON import #1432

Bug: Dr4ft Crashing on JSON import #1432

Comments

ZeldaZach commented Feb 3, 2021

ZeldaZach commented Feb 3, 2021

ZeldaZach commented Feb 3, 2021

HerveH44 commented Feb 7, 2021

mixmix commented Feb 7, 2021

ZeldaZach commented Feb 9, 2021

ebbit1q commented Feb 9, 2021

ebbit1q commented Feb 9, 2021 • edited Loading

mixmix commented Feb 9, 2021

ebbit1q commented Feb 9, 2021 • edited Loading

ebbit1q commented Feb 9, 2021

ebbit1q commented Feb 9, 2021 •

edited

Loading

ebbit1q commented Feb 9, 2021 •

edited

Loading