Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ATN graph cannot be drawn even after code generation. #28

Closed
smwikipedia opened this issue Mar 22, 2018 · 21 comments
Closed

ATN graph cannot be drawn even after code generation. #28

smwikipedia opened this issue Mar 22, 2018 · 21 comments
Labels
bug

Comments

@smwikipedia
Copy link

@smwikipedia smwikipedia commented Mar 22, 2018

After I right click a rule and select "Show ATN graph for rule", I only got this:

no_atn

@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 22, 2018

What happens when you change something and save the grammar. Does it show the little busy animation in the status bar? Does the ATN graph show something useful afterwards?

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 22, 2018

Yes, it shows a little busy animation in the status bar after I modify and save the grammar. And I can see new parser files generated. But still cannot see ATN graph. The error is as the screenshot. But I can see rail road diagram.

@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 22, 2018

Are there any error messages in the Developer Tools console (see help menu)?

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 23, 2018

I tried below:
Open Help -> Developer Tools -> Console (Choose verbose mode in filter)
Open a .g4 file and modify and save.

What I see:
Busy animation shows.
New parser files generated.
Nothing shows in Developer Tools Console.

Now I am trying to reinstall the VS Code and the ANTLR plugin.

@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 23, 2018

Does the call graph show up? Does debugging work? The message you see about no ATN data found means the *.interp files couldn't be loaded. The question is: why? Are they generated and reachable?

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 23, 2018

I tried the C# target and Java target with C.g4. In both scenarios, a CLexer.interp file got generated. But I don't know why they cannot be loaded.

I haven't tried the debugging. But I will try it.

Below are my user settings:


{
    "window.zoomLevel": 0,
    "workbench.colorTheme": "Visual Studio Dark",
    "editor.renderWhitespace": "all",
    "python.linting.enabled": false,
    "antlr4.generation": {
        "mode": "external",
        "language": "Java",
        "listeners": true,
        "visitors": false,
        "outputDir": "Java"
    },
    "antlr4.rrd.saveDir": "antlr4.rrd",
    "antlr4.atn.saveDir": "antlr4.atn",
    "antlr4.call-graph.saveDir": "antlr4.callgraph"
}
@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 23, 2018

I just tried the debug feature. There seems to be some issue.

My 1st debug settings:

{
    "version": "2.0.0",
    "configurations": [
        {
            "name": "antlr4-c",
            "type": "antlr-debug",
            "request": "launch",
            "input": "${workspaceFolder}/${command:AskForTestInput}",
            //"input": "../TestSuite/full.c",
            "grammar": "C.g4",
            "startRule": "compilationUnit",
            "printParseTree": true,
            "visualParseTree": true
        }
    ]
}

I got this error:

debug1failure

My 2nd debug settings:


{
    "version": "2.0.0",
    "configurations": [
        {
            "name": "antlr4-c",
            "type": "antlr-debug",
            "request": "launch",
            //"input": "${workspaceFolder}/${command:AskForTestInput}",
            "input": "../TestSuite/full.c",    // <===============HERE changes.
            "grammar": "C.g4",
            "startRule": "compilationUnit",
            "printParseTree": true,
            "visualParseTree": true
        }
    ]
}

debug2failure

I guess the *.interp is the interpreter data in above screenshot. So I guess same issue as ATN graph here.

But the *.interp file did get generated. See below.

debug3failure

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 23, 2018

I just tried reinstall Visual Studio Code and the ANTLR extension. Still the same error.

@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 23, 2018

That's not a vscode issue. There are several problems at work here.

For debugging:

  • The action to ask for that file wasn't recognized, which is totally strange. As a consequence that action was taken as file name.
  • In the second case it seems as if the grammar wasn't read from the config, even though it's there. There's a mode where the debugger uses the currently active editor, if that is a grammar, if no grammar is given in the config. If the current editor is not debuggable by the antlr4 extension it shows that other error.

Also for the debugger you need the ATN data. You found the *.interp files in the target folder, right? But for ATN + debugging they must be in the internal .antlr folder (in the same folder where your grammar is). Try with internal generation mode. Does this work? Could be something with the interp data move is wrong. These files should be automatically moved from the external output folder to the internal one. You shouldn't even see them.

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 24, 2018

I tried below 2 things:

  1. I changed to use "mode": "internal". And I saw below files generated in the .antlr folder.
    internal mode

Then I tried debug by pressing F5. It gave me this:
But the start rule is just there.
start rule error

And the ATN graph failed as before.

  1. Then I tried "mode": "external". And I manually copy the *.interp files to the .antlr folder. And then I tried Debug and ATN graph. Still failed as 1. And below are the logs from the Debug Console:
10:14:03 AM, 3/24/2018
From client: initialize({"clientID":"vscode","adapterID":"antlr-debug","pathFormat":"path","linesStartAt1":true,"columnsStartAt1":true,"supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"locale":"en-us"})
To client: {"seq":0,"type":"event","event":"initialized"}
To client: {"seq":0,"type":"response","request_seq":1,"command":"initialize","success":true,"body":{"supportsConfigurationDoneRequest":true,"supportsStepInTargetsRequest":true}}
From client: threads(undefined)
To client: {"seq":0,"type":"response","request_seq":2,"command":"threads","success":true,"body":{"threads":[{"id":1,"name":"Interpreter"}]}}
From client: launch({"name":"antlr4-c","type":"antlr-debug","request":"launch","input":"c:\\Test\\TestSuite\\full.c","grammar":"c:\\Test\\grammar\\C.g4","startRule":"compilationUnit","trace":true,"printParseTree":true,"visualParseTree":true,"debugServer":17865,"__sessionId":"42bc26d0-999c-426d-985f-810661544990"})
To client: {"seq":0,"type":"response","request_seq":3,"command":"launch","success":false,"message":"Error while launching debug session: start rule \"compilationUnit\" not found","body":{"error":{"id":1,"format":"Error while launching debug session: start rule \"compilationUnit\" not found"}}}
From client: disconnect({"restart":false})
To client: {"seq":0,"type":"response","request_seq":4,"command":"disconnect","success":true}

And this is the g4 file I used.

C.g4.zip

@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 24, 2018

There was an error in the interpreter file name construction, which is fixed now. Until the next release you can split your combined grammar into individual lexer and parser grammars. This should work.

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 24, 2018

Thanks for the quick fix. Looking forward to the new release.

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 24, 2018

BTW, I just tried splitting my grammar into individual parser and lexer grammars. Unfortunately, still the same error.

It seems the split parser grammar CParserRules.g4 still generate its own CParserRulesLexer.interp file. So now, I got 3 *.interp files. Hope this won't be an issue. See below:

split grammar

My split grammars:
CParser&LexerRules.zip

And the 3 interp files:
3 interp files.zip

@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 24, 2018

Now, that's a situation I hadn't on the radar. You get 3 files because you still have lexer rules in your parser grammar (all the text literals) which cause ANTLR4 to generate that CParserRuleLexer.

However, that shouldn't affect the load of the interpreter data. Though, I just found another bug and fixed it. The previous change went a little too far. Can you clone the repo, transpile the TS code and manually copy the SourceContext.js file the extension folder (the one in your home dir, as given in the error screenshot above)? This way you should finally get what you want without waiting for the next release of the extension.

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 24, 2018

I am not familiar with TS project. Here is what I tried:

  1. cloned the repo.
  2. run tsc --out SourceContext.js SourceContext.ts

Then I got many errors. Such as:

C:\D\Sources.2\github\vscode-antlr4\src\backend\SourceContext.ts(13,1): error TS1008: Unexpected token; 'module, class, interface, enum, import or statement' expected.
C:\D\Sources.2\github\vscode-antlr4\src\backend\SourceContext.ts(13,8): error TS1008: Unexpected token; 'module, class, interface, enum, import or statement' expected.
C:\D\Sources.2\github\vscode-antlr4\src\backend\SourceContext.ts(13,13): error TS1005: ';' expected.
C:\D\Sources.2\github\vscode-antlr4\src\backend\SourceContext.ts(13,27): error TS1005: ';' expected.
C:\D\Sources.2\github\vscode-antlr4\src\backend\SourceContext.ts(13,32): error TS1005: ';' expected.
C:\D\Sources.2\github\vscode-antlr4\src\backend\SourceContext.ts(14,1): error TS1008: Unexpected toke
...


Then I tried to open it in VS code as workspace. Then run Ctrl+Shift+B. VS Code complains that :

The following workspace folders are ingored since they use task version 0.1.0: vscode-antlr4.

I installed tsc and node, maybe I missed something.

  • tsc v1.0.3.0
  • node v9.9.0

I am learning the TS related knowledge now.

@mike-lischke mike-lischke reopened this Mar 24, 2018
@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 24, 2018

Mikes-iMac:vscode-antlr4 mike$ npm update
+ vscode@1.1.14
+ @types/node@8.10.0
updated 11 packages in 7.924s
Mikes-iMac:vscode-antlr4 mike$ npm install

> vscode-antlr4@2.0.0 postinstall /Volumes/Extern/Work/projects/vscode-antlr4
> node ./node_modules/vscode/bin/install

Detected VS Code engine version: ^1.17.0
Found minimal version that qualifies engine range: 1.17.0
Fetching vscode.d.ts from: https://raw.githubusercontent.com/Microsoft/vscode/be377c0faf7574a59f84940f593a6849f12e4de7/src/vs/vscode.d.ts
vscode.d.ts successfully installed!

up to date in 2.685s
Mikes-iMac:vscode-antlr4 mike$ npm run compile

> vscode-antlr4@2.0.0 compile /Volumes/Extern/Work/projects/vscode-antlr4
> tsc --outDir ./out/src

Mikes-iMac:vscode-antlr4 mike$ 

Then in out/src/frontend you can find the transpiled SourceContext.js.

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 24, 2018

Below are my try. Bold part are my command inputs.

C:\D\Sources.2\github\vscode-antlr4>npm update

  • fs-extra@5.0.0
  • d3@4.13.0
  • antlr4ts@0.4.1-alpha.0
  • vscode-debugprotocol@1.27.0
  • vscode-debugadapter@1.27.0
  • antlr4-c3@1.1.7
    added 394 packages in 22.514s

C:\D\Sources.2\github\vscode-antlr4>npm install

> vscode-antlr4@2.0.0 postinstall C:\D\Sources.2\github\vscode-antlr4
> node ./node_modules/vscode/bin/install

Detected VS Code engine version: ^1.17.0
Found minimal version that qualifies engine range: 1.17.0
Fetching vscode.d.ts from: https://raw.githubusercontent.com/Microsoft/vscode/be377c0faf7574a59f84940f593a6849f12e4de7/src/vs/vscode.d.ts
vscode.d.ts successfully installed!

C:\D\Sources.2\github\vscode-antlr4>npm run compile

> vscode-antlr4@2.0.0 compile C:\D\Sources.2\github\vscode-antlr4
> tsc --outDir ./out/src

src/backend/ContextSymbolTable.ts(255,38): error TS2554: Expected 0 arguments, but got 1.
src/backend/SourceContext.ts(593,78): error TS2339: Property 'add' does not exist on type 'Symbol[]'.
src/backend/SourceContext.ts(596,9): error TS2322: Type 'Symbol[]' is not assignable to type 'Set'.
Property 'add' is missing in type 'Symbol[]'.
src/backend/facade.ts(479,38): error TS2554: Expected 0 arguments, but got 1.
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! vscode-antlr4@2.0.0 compile: tsc --outDir ./out/src
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the vscode-antlr4@2.0.0 compile script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\mshao\AppData\Roaming\npm-cache_logs\2018-03-24T16_02_25_053Z-debug.log

@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 24, 2018

Try all that again. I had some changes not yet published in my antlr4-c3 node module.

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 24, 2018

Now the npm commands finished without error. I manually copied all the js files from the output folder to the extension folder. I can debug and see ATN graphs now. Many thanks!

debug and atn

@mike-lischke

This comment has been minimized.

Copy link
Owner

@mike-lischke mike-lischke commented Mar 24, 2018

If you like rate the extension on the vscode Marketplace.

@smwikipedia

This comment has been minimized.

Copy link
Author

@smwikipedia smwikipedia commented Mar 24, 2018

Sure I will rate it 5.😀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.