XCodeMapper Updates #27
Comments
I don't mind it at all :) In fact this is exactly what I had in mind with CodeMap as a product. I wanted it to be a lean, simple and reliable platform, which would be used by ST3 devs to create various code mappers and regex mapping algorithms. The mappers for everything, even the most exotic syntaxes. Or even competing mappers for the already covered languages. So we are good... There is one thing that I found challenging though. I found your post (above) is overwhelming to read because of the size. But... this is really subjective. Cheers, |
|
I agree with you - I need to take the time to rewrite it to outline the features, pros vs cons, etc.. along with screenshots - similar to what I posted on the sublime forum with more organization and less fluff. I'll do that soon - there are still more things to do with the mapper system such as cleaning out some of the excess code / combining some of the functions, and structuring the callbacks. I've been experimenting with using sublime-settings too and I think it'll actually be a welcome change. Even if we don't end up merging - I can make each ext.py file identical and have the back end route to the most-child class and import parents upwards to avoid circular inclusion - not a fan of each file being identical, but I'll make them as small as possible and using a slightly different system so circular imports aren't broken by it including the XCM base.. I am also looking into modifying my repo so it can be simply added to package control and still upload the files to the correct location. This may or may not be possible without creating a separate package with a dependency - and packages that use others mean the others are automatically extracted by package control. With regards to the settings system with sublime text - if the file isn't in a package root directory then it looks in the user/ folder root. There are ways around this... The settings file can become quite large depending what people want to add ( DataType / Arg replacement system with short vs long output for the data-type for the args that are replaced along with func vs class support and linking the keys from args to data-type keys via their values -- AccessorFuncs with the option to link the value to a config key or the actual value assigned ) so I'm also considering splitting them up - a default file with all of the default AccessorFunc / CfgAccessorFunc definitions, etc... and each language can then expand using inheritance so only the key values which are altered from default need to be supplied, but others extend such as the AccessorFunc tables... -- If I use one file per, then the User/ folder can end up with a LOT of configuration files depending the final number of languages supported, but I can manually choose the locations of the config files - ideally in User/CodeMap/ so they're organized with the package which will use them ( I'm considering asking the SublimeText team to add default support for User// when looking for config files because it's odd they ALL need to be grouped together in User/ ).. Just some of the things I'm doing... and I still haven't forgotten about cleaning up the original post above - following posts would be used for changelog and user requests along with the post on sublime text forum... But because it is here people will see it as a way to enhance an already excellent addon. A nice feature to see with CodeMap would be a way to navigate through the entries using keys - but instead of just up or down - use blank lines as divides so you can jump between them and then continue up / down. Regarding merging:
Merging would enable to move a lot of the features to the back-end and prevent the need for ext.py files being identical for nested classes ( I've been working on the settings system so you can use the full-name of a class or just the short-name so User vs Default vs ProjectX, etc.. plus a bunch of other systems ) meaning less code to execute and it'd be more streamlined - but the option to use ext.py would still exist for those which want to use a global mapper per language - the one issue with one mapper per language is because many make projects out of code and those projects add things or change how the language is used which confuses a lot of mapping systems - one of the reasons why I added the project classes with inheritance is because of GMod Lua but also because I have a project within and if I'm looking at Lua without my project I don't need all of the other syntax to be looked for - increasing speed at which the data is processed... I do like CodeMap but it has a major flaw. The lack of a category system means when one thing is encountered twice with something in between causes the current system to output the same category twice. It also prevents the data from being ordered. Other than this - it is a fantastic base. The ability to use shortcuts to scroll through the output is useful - extending this to jump between categories would make it even better along with the other features makes it an excellent choice for mapping system. If we don't merge fully - moving the category system, when finished, to the back end would be a major bonus for CodeMap and those that use it. I still have some work to do before it is fully ready. Sorry for the wall of text - I guess I just have too much to say and so little time to do... |
|
I have to confess, I have missed somewhere among these enormous amount of reading your primary message: "you see your work being merged with the CodeMap plugin". Sorry but, as I have already expressed, this contradicts with the plugin model. CodeMap is to stay simple and completely decoupled from the mappers. Which are to be deployed on top of the CodeMap either separately. And only in some cases by being included in the plugin package as resources. However there is not going to be any merge of the plugin codebase with anything else. Or any code change for that matter. Fixes, missing features - yes, but no refactoring for the sake of refactoring, no design changes to accommodate specific mapper unless it brings other benefits. All mappers need to follow their own development life cycle. Versioning, releases, design challenges , everything should be separate and should have no effect on CodeMap. That's why I don't want you to have the wrong expectations about the merger. As I see it you need to develop your comprehensive mapper(s) in a separate GitHab project. Own your product, describe it, promote it (it looks like it deserves it) and the only connection to CodeMap it would have is that your product "XCodeMapper" would have another product "CodeMap" as a prerequisite. BTW this type of relationship between plugins and subplugins is quite common. In VSCode plugin manifest you can even specify another plugin as a dependency and the plugin will be automatically installed for you by the IDE. |
|
I wasn't aware of your viewpoint - but I did suggest the merging of the category system without needing to merge anything else... Also, I didn't say merge the mappers - I meant a different way to load them which would fit the model in the back-end so instead of needing to load ext.py when one extension uses many different languages, etc... a rule can be set up to decide how they load ( what I am going to do is load it and have dynamic loading - but it'd be more straight-forward in the back-end because of how you'd end up importing... If you aren't necessarily interested in merging, that's no problem.. I did mention I don't mind if you are interested in no features, or only a few ( such as the category system ). Also, the comprehensive mappers would be separate from "merging" if merging - ie the core features of XCM such as caching, categories, easy adding of syntax to map ( which can be extended to be called from a tmLanguage or sublime-syntax file ) etc... The mappers are the collections of those calls, and the callbacks to modify how the data is processed... I did ask whether or not to include the mappers by default but it isn't important... If you're not interested in the core features ( You did mention it was your vision of how CodeMap should work ) then no problem. I'm just not sure of your standing because you mentioned the backend ( I'm assuming ) is how you envisioned CodeMap then stated you don't want to merge at all because of the mappers ( the collection of AddSyntax calls, and other rules to modify the data found )... As for refactoring / design - I didn't mean recode from scratch or anything - basically deciding on the coding standard of how the code should look within CodeMap so it doesn't appear as though it comes from a ton of people with their own style... this is to display professionalism in the code. For design, I meant what specific features you're interested in - ie basic categories vs unlimited nested support - how are the categories added, etc... A basic outline - not a complete rewrite of whatever... The code change to bring other benefits - for the mappers - as I mentioned above - a different way to load using rules brings many benefits ( easier to process files which use multiple languages - PHP uses 6 [ PHP, HTML, JavaScript, SQL, CSS, Regular Expressions ] or more? same with ASP, etc.. ) that's what I meant by redesigning the loading system so it doesn't need to rely on a single ext.py - it can load multiple and then either process the code in chunks / lines and strip out what isn't needed using a rule system and supply the data to the mapper - similar to how Sublime text supplies function names, etc... to the command palette. I'm only speaking of benefits as far as I know? I did notice your VSCode addon ( I have been wanting to give VSCode a shot so if it is similar, and supports python I may port this over too but the output would likely be different )... I am going to explore modifying the repo so it can easily be added to Package Control - the problem I see is with modifying the sublime-syntax and tmLanguage files to add extra chars to highlight but I need to dig deeper... Just to be clear - when I spoke of merging I never meant merging the mappers ( collection of Addsyntax, etc ) to the back-end because that takes away from the system by locking the mappers in place and making it harder to edit... The mappers in the front end is best - the question posed was if we did decide to merge the back-end features ( category system, easy adding of things to look for with many search modes, etc.. and more ) whether or not to include the mappers I've come up with as default mappers ( in the custom_mappers folder as you have done with py.py and others? I'm sorry if it seemed any other way... New Update New update - Caching system... If developer mode is on ( which reloads the core files if they are modified without needing to restart st3 ) then those file timestamps are added to the mix to determine whether to load the cached file or process the file and create the cached file... Caching can be disabled per mapper ( or globally when I add sublime-text )... Todo: Integrate OnCalcGutterIcon - so when a gutter icon is set ( can be automatically bound to categories, or manually overwritten in this callback ) to show you in your mapped file where everything is ( to work with the caching system I'll allow adding saved-data to another cached file - same file .rules or something which would be a basic text file indicating line, rule to execute, and data ( icon, or otherwise ) )... Add delete system to cache system so cached files which are N days old are removed - configurable to the second if wanted... Also, as for name of this system since I am closing in on most of the main features I want in ( backup system may not be added ) - CodePeer or something else... I have a few others I like. |
Note: I'm currently working on formatting the post to provide the most information in an organized way...
XCodeMapper
The package modifier to make CodeMap the greatest Code Navigator for SublimeText available by weeding out what you don't need to see and displaying what you do backed by an unlimited nesting category system to organize that output in a way that makes sense, makes you more efficient, and even helps debug your code all while taking up as little space from your coding area as possible.
Our goal is to make you more efficient by showing you what's relevant and hiding what isn't and by organizing the relevant data in a way that instantly shows you its place within a file or project and by showing you common or uncommon errors which can otherwise be difficult to track down because of the way our brain processes text. It is to also allow you to decide what's relevant based on the language you're using, or the project your working on in that language by using simple function calls to add things to earmark in your code.
The hmuan mnid is albe to raed tihgns wihch may ohwtierse look iopmsibsle wehn alcuatly tnhinkig aobut it. Because of this, it's easy for us screen code and miss things because it makes sense in other languages or because certain characters are used in the current language but won't work in other situations, etc.. Words simply blend together and to us, we look at the first and last characters in place, and the content to determine a word, our minds don't actually care about the order the content is to make a word appear. It's also possible for us to omit special characters, doubles, etc.. while reading because they don't mean anything in terms of communicative language. Having a second pair of eyes which doesn't read the way we do makes us better and more efficient overall while coding.
In the future I want to add a more advanced syntax error detection system to make tracking down typos and little errors common with those of us who deal with dozens of programming, scripting and display / stylization languages - This is especially important for code which goes through an evaluator within a language as error-checking is incredibly difficult because you typically end up with incredibly vague descriptions which could match more than half of what's processed.
#Frequently Asked Questions and Answers#
How do we give back?
Where do I go to report a bug?
Where do I submit a feature request?
Where can I download XCodeMapper?
Where do I install This Addon?
How do I install it?
How do I know if I successfully installed it?
-JavaScript.sublime-syntax ( DEFAULT )
Why can't I simply add the repo to Package Control?
What custom mappers does XCodeMapper come with and what do they track in the code?
CATEGORY_USED_, SEARCH_METHOD_REGEX_STARTSWITH_ENDSWIDTH_CONTAINS_case_sensitive_CONTAINSI_as_insensitive, VARARGS ) where VARARGS are 'x', 'y', 'etc' for as many entries as you want which are added to the same category and search-type used to find them... The regex below should be obvious - others are startwith or contains / i for the most part.. The reason Startswith is used for quite a lot is because its inexpensive logic ( it basically grabs length of string we're checking, SubStrings the code we're looking at then checks SubStringedCode == OurLookingString ) - we don't need to use it in any of the mappers but it is also more human readable than most regex...
)(.+)()'How many languages do you plan on supporting?
What Major features do you currently have planned?
- In short: a rule system will allow us in the classes or configuration to define certain basic things, or even a toggle, to process data. One example is how Python tracks nested classes - we'd be able to do the same thing with a rule-system and a few definitions such as indenting or replacing scope when class keyword is found, unindenting / popping off the stack when code on a different depth is found. For other languages we can use then, end, {, }, etc.. to track these things. This way we don't need to repeat long chunks of logic for different languages which are essentially the same... You could say we're making everything as generic as possible... Other rules will be added too..
- Right now we strip comments from lines so they don't appear if code is found on that line we're interested in. We also strip comment blocks from lines, etc... from the start and end and we return the appropriate code...
- Certain files support many languages per. One example is PHP which uses HTML when not in php tags. For HTML JavaScript can be found in Script tags, and CSS in Style tags. When a php tag is found, we process PHP but SQL can also be found in certain function calls not to mention Regular Expressions... These are things we need to look for. HTML as said above supports 3 primary languages. PHP supports 6. ASP Supports 6. Without supporting these extra languages, valuable information is slipping through our fingertips and re-adding rulesets or entire languages multiple times defeats the purpose of my coding standard - expect this feature soon!
What Minor features do you currently have planned?
What Code Alterations do you currently have planned ie things we won't notice?
What Optimizations do you currently have planned?
What Optimization Features exist?
What Bug Fixes do you currently have planned / ie What are the known bugs or issues?
What Other Changes do you currently have planned?
What is your story?
What do you spend your time doing?
Are you accepting new students?
What does Acecool mean?
https://bitbucket.org/Acecool/acecooldev_sublimetext3
I hope you don't mind I'm posting this in a new thread... Would you mind marking as optional enhancement?
Sales Pitch: XCodeMapper ( developmental name ) is a way to make an already incredibly useful extension, CodeMap by oleg-shilo, even better.. It modifies the way you earmark code in your project to have those instances at your fingertips. It is customizable with many callbacks you can design how you want it to look, behave, and how much or little information you want to show at any given time. You can modify code going to the output on the fly, look for hard to find syntax errors and flag them for review, and much much more...
The latest update added nested category support ( for Python to have a way to display functions nested in classes differently and so the alphabetical sorting method would work on the function names without shuffling everything ( which is what happens if everything is added to 1 category - that old hack is now gone )... The nested categories are sorted alphabetically too now because they are the class-headers...
I've just added a brand new feature to change how functions nested in classes are displayed... There are 2 main options and a modifier...
1 ) Vertical mode where the CLASS_FUNCTION category is populated with alphabetically sorted nested categories and those populated with the functions from those classes.. This is great if you don't have a lot of width to rent out on your monitor while coding...
2 ) Horizontal / Lua style mode where CLASS_FUNCTION category is populated with the function entries - except the function entries show up similar to how Lua handles their objects / meta-tables because everything is in tables.... It can show up as DirectParentClassName.FunctionName( _args ) per line - all alphabetically sorted.... OR you can enable the modifier which ShowsTheShallowestClass.All.The.Way.To.The.DirectParentClassName.FunctionName( _args ) - This is useful to know exactly how to call a function if it is nested deeply... This is rarely the case though..
Images:
sublime.py both modes ( not many nested so the modifier isn't needed to be shown disabled )..
This shows the Vertical mode where CLASS_FUNCTION Category is primary category for them, DirectParentName is a nested category within, and the FunctionName( _args ) is an entry to that category... This shows how it looks on a traditional Python file...
https://www.dropbox.com/s/odm96jm87fhxuck/sublime_text_unregistered__screenshot_20180111124446_2835.png?dl=0
This shows DirectParentClassName.FunctionName( _args ) as entries to CLASS_FUNCTION primary category... This shows how it looks on a traditional python file...
https://www.dropbox.com/s/wum1gil6gg0jplh/sublime_text_unregistered__screenshot_20180111124503_732.png?dl=0
Python.py EXAMPLE where categories are all over the place and functions are above and below other class declarations - this is designed to confuse the system to show it works properly... with both systems and the modifier..
Display in component form... CLASS_FUNCTION Category with DirectParentClassName nested Category and FunctionName within that.... I will add unlimited nesting later on so that will be yet another option - so you will be able to display vertically, or staggered.... look for that soon!!!
https://www.dropbox.com/s/gu8dd4qavt5z1a5/sublime_text_unregistered__screenshot_20180111124517_8960.png?dl=0
Display using FullPathFromShallowest.ToThe.Deepest.Until.DirectParentClassName.FunctionName( _args )
https://www.dropbox.com/s/3qzsc315vx6v7tx/sublime_text_unregistered__screenshot_20180111124528_5402.png?dl=0
Display using DirectParentClassName.FunctionName( _args )
https://www.dropbox.com/s/5su6hr2uqmx0fj3/sublime_text_unregistered__screenshot_20180111124537_2601.png?dl=0
I added the vertical layout to the Lua mapper too ( very few lines of code - shown here and the default method ):
Default method ( string.FuncName( _text ), Acecool.C.timekeeper:Lerp( _args ) ):
https://www.dropbox.com/s/y2epp63smzedk72/sublime_text_unregistered__screenshot_20180111130414_2558.png?dl=0
Display in component parts and organize in nested categories... ( For files with a lot of libraries, classes, this organizes them in a very easy to read manner...
https://www.dropbox.com/s/y3w4iqytelqd1u5/sublime_text_unregistered__screenshot_20180111130239_9770.png?dl=0
Note - I include the updated syntax files for various languages because, by default I replace def / function with the character designed for mathematical functions which is: ƒ - If you do not want to use this char, edit:
CFG_FUNC_NAME_SEARCH = 'def'
to
CFG_FUNC_NAME_SEARCH = ''
When the SEARCH var is empty, it will not replace....
Additional images:
Showing the JavaScript mapper with my Web Framework Project enabled and how it converts function calls into usable information along with auto editing _tag ( I use _tag for the other functions other than the first to allow for easy copy / paste to add new features to other sites )
https://www.dropbox.com/s/8pcqdubas5ycrma/sublime_text_unregistered__screenshot_20180109044803_7634.png?dl=0
Showing the AutoHotkey mapper ( It also converts the modifier key chars from ! as Alt, ^ as Ctrl, + as Shift and # as Win on top of * for Allow with other mod-keys pressed, ie hotkey with Alt F12 will run with Ctrl Alt F12.... and ~ for NoBlock meaning you can intercept hotkeys and still allow other programs to receive the original hotkey... ):
https://www.dropbox.com/s/ko5ejadvbyvgvcw/sublime_text_unregistered__screenshot_20180109044734_8188.png?dl=0
Shows the Lua mapper modifying output data and expanding AccessorFuncs using the AccessorFuncExpansion Callback ( allows you match a search pattern, prevent the original pattern from being added an an entry, and you simply call self.AddEntry or self.AddSimpleEntry with the new function names which the AccessorFunc adds - easy to set up...
https://www.dropbox.com/s/4v1sw803jk65let/sublime_text_unregistered__screenshot_20180109044704_1329.png?dl=0
Same but scrolled down... the __ functions are internal functions generated by the AccessorFuncs - I have a config var to hide them as my AccessorFuncs generate upwards of 10 or more depending on the data-type, and what I need from them...
https://www.dropbox.com/s/tipzrjufbb6b7sy/sublime_text_unregistered__screenshot_20180109044658_5826.png?dl=0
Shows sublime.py with the standard XCodeMapper for python.. Here I didn't modify def so you can see what it looks like with the categorized methods ( note the new version simply has ☼ set as a prefix - I'm going to remove it from the default probably - but I've been finding some highlighters have trouble so I've written code to end open quotes, add comment to the end of the line, or use a block comment hack... And it works for most, but some such as PHP refuse to highlight so I would recommend using Python or JavaScript for PHP... Also the additional prefix chars ( all customizable in CFG_ vars ) can be removed, edited, or whatever... By default the tasks list are now commented.... I was thinking of doing the same to comments but I haven't run into any problems with the end-line chars I'm using...
https://www.dropbox.com/s/f9yj00p2jf9tg02/sublime_text_unregistered__screenshot_20180109044555_7884.png?dl=0
I update XCodeMapper frequently to meet my needs and to integrate new features which are designed to increase efficiency while not only coding, but for setting up new mappers...
All mappers come with multiple mapper classes in the file - _USer is enabled by default and has the vanilla language set as extension so you can edit what you want and when an update comes out simply copy and paste the _User class back in ( I will be exporting them soon to different files to avoid this.. )
Current To Do List:
-MORE
If you encounter any issues - please let me know! You can post on BitBucket, or here ( if I'm allowed to keep this going - I think XCodeMapper - when I come up with a better name, it'll change - and CodeMap would go great together because of a combined user base... I am piggybacking right now, but hopefully my mod will gain traction and bring more people to CodeMap... I haven't found anything else like it so far...
The text was updated successfully, but these errors were encountered: