koch crashes when module lines > int16.high

[nitely]

I've an auto-generated file of more than 32767 lines and I'm getting this error when running ./koch temp c myfile.nim. I can reduce the lines, but the point is the compiler should not crash when there is an error on a big file.

./koch temp c myfile.nim output:

nim.nim(121)             nim
nim.nim(77)              handleCmdLine
main.nim(172)            mainCommand
main.nim(77)             commandCompileToC
modules.nim(240)         compileProject
modules.nim(180)         compileModule
passes.nim(218)          processModule
passes.nim(136)          processTopLevelStmt
sem.nim(556)             myProcess
sem.nim(524)             semStmtAndGenerateGenerics
semstmts.nim(1889)       semStmt
semexprs.nim(782)        semExprNoType
semexprs.nim(2360)       semExpr
importer.nim(178)        evalImport
importer.nim(155)        impMod
importer.nim(141)        myImportModule
modules.nim(203)         importModule
modules.nim(180)         compileModule
passes.nim(218)          processModule
passes.nim(136)          processTopLevelStmt
sem.nim(556)             myProcess
sem.nim(524)             semStmtAndGenerateGenerics
semstmts.nim(1889)       semStmt
semexprs.nim(782)        semExprNoType
semexprs.nim(2360)       semExpr
importer.nim(178)        evalImport
importer.nim(155)        impMod
importer.nim(141)        myImportModule
modules.nim(203)         importModule
modules.nim(180)         compileModule
passes.nim(218)          processModule
passes.nim(136)          processTopLevelStmt
sem.nim(556)             myProcess
sem.nim(524)             semStmtAndGenerateGenerics
semstmts.nim(1889)       semStmt
semexprs.nim(782)        semExprNoType
semexprs.nim(2360)       semExpr
importer.nim(178)        evalImport
importer.nim(155)        impMod
importer.nim(141)        myImportModule
modules.nim(203)         importModule
modules.nim(180)         compileModule
passes.nim(204)          processModule
syntaxes.nim(46)         parseTopLevelStmt
parser.nim(2143)         parseTopLevelStmt
parser.nim(2041)         complexOrSimpleStmt
parser.nim(1661)         parseSection
parser.nim(1689)         parseConstant
parser.nim(1094)         parseExpr
parser.nim(781)          simpleExpr
parser.nim(771)          simpleExprAux
parser.nim(1168)         primary
parser.nim(666)          identOrLiteral
parser.nim(422)          exprColonEqExprList
parser.nim(410)          exprColonEqExprListAux
parser.nim(372)          exprColonEqExpr
parser.nim(1094)         parseExpr
parser.nim(781)          simpleExpr
parser.nim(771)          simpleExprAux
parser.nim(1168)         primary
parser.nim(597)          identOrLiteral
parser.nim(184)          newIntNodeP
parser.nim(181)          newNodeP
parser.nim(171)          parLineInfo
lexer.nim(155)           getLineInfo
msgs.nim(619)            newLineInfo
system.nim(2817)         sysFatal
Error: unhandled exception: value out of range: 32768 [RangeError]
FAILURE

Compiler won't crash in release mode though, because line count will simply overflow :)

[nitely]

msgs.nim(619) is doing an explicit conversion (int16(x)) and so it crashes even on release mode :( koch only works on debug mode it seems.

@nitely are you sure? If compiler will crash in release mode we'll be not able to use oldwinapi module (which should work and no one reported that it doesn't yet)

[nitely]

@Yardanico Ah, hum. nim c myfile.nim crashes, but not coz of the int conversion. I get:

Error: internal error: (filename: "vmgen.nim", line: 162)
No stack traceback available
To create a stacktrace, rerun compilation with ./koch temp c <file>

Then koch crashes coz of the int conversion.

Edit: I reduced the lines in my module and now koch no longer crashes on it. But it crashes later with the same error as nim c myfile.nim above which is unrelated (see #7059) .

[Araq]

I can reduce the lines, but the point is the compiler should not crash when there is an error on a big file.

Well I can make it produce a nice "file too long" error message instead. A file with 32K lines is unreasonable.

[nitely]

A file with 32K lines is unreasonable.

A file with a big dataset is unreasonable?

@nitely you can have it in some text file and parse on compile-time thought (I know this this will increase compile-time more than using a nim file but it's still an option)

[nitely]

On a second thought, a file bigger than 32K lines x 32K columns is probably unreasonable. The descriptive error message would work.

[nitely]

@Yardanico @Araq it's not like it can't compile it. It can't generate the stack trace if there is an error somewhere past the 32K lines, IIRC. So, the error message should say something like "file too long to generate a stack-trace".

Edit: or better yet display -1 as line/col number (or just don't display it), coz I think the rest of the stack-trace is useful.

[krux02]

I think I also got this problem once, too. I think somewhere in the compiler there is a number stored in a 16 bit integer for no good reason.

@krux02 Araq said that this is an optimization (line numbers are stored as int16 numbers)

[Araq]

Line numbers are now uint16, yay.