Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] major rewrite for v2.0-preview #18

Merged
merged 146 commits into from
Oct 29, 2020
Merged

Conversation

binary1230
Copy link
Collaborator

@binary1230 binary1230 commented Oct 7, 2020

[DRAFT DO NOT MERGE YET]

This pull request is a pretty major rewrite of most parts of Diztinguish.

Major: Code architecture overhaul

  • Move away from most static-based data structures, use objects directly now
  • Model-view-controller (ish) architecture implemented, some databinding in place
  • Starting to replace manual piping with C# events / INotifyPropertyChanged / databinding / Observable collections
  • Separate GUI and core SNES logic into different projects
  • Should make it easier to write automations, tests, and undo/redo support in the future
  • Should make it easier to port to other platforms in the future like linux/etc or move away from Winforms if we want to.
  • Will make it easier for future features (like graphics/sound viewers) to update/notify other parts of app about changes, and to extend Diz to support things like SNES data structures, tighter integration with Asar, etc.
  • Utilize newer C# language features to simplify lots of code
  • Simplify/unify project import vs load codepaths

Major: New XML-based save format

  • Goal: multiple users able to work on the same .diz file in source control and merge changes
  • Utilizes https://github.com/ExtendedXmlSerializer/home which uses reflection to dig into the object graph and save/load any public properties. This means adding stuff to the save format is a breeze, and migrations are easy.
  • Everything is saved as normal XML except, to save space, there is a custom serializer that writes the RomByte metadata in compact text form (one RomByte = ~eight real bytes in the file).
  • Custom compression is then used to further reduce the RomByte data format, doing some simple text substitution and replacing large chunks with a marker. End result, while not being human-friendly, is still one byte per line and human mergable.
  • Loading from original binary file format is still supported, but the original binary file format is now deprecated for saving
  • XML format is still gzipped by default. With GZIP compression on, project size is ~100K. With gzip off, filesize for human-readable XML is 1.3MB, which is pretty easy to live with.

Minor:

  • Add Xunit unit testing and start working on some early tests
  • Integrated and refactored the excellent Bizhawk and BSNES trace log / usage map features from @gocha and @VitorVilela7
  • Allow importing multiple tracelogs at once
  • Optimized tracelog import to import much faster
  • various GUI improvements like progressbars, speed improvements
  • support new BSNES tracelog format (flags are different)
  • Better control over importing labels CSV
  • Use GitInfo nuget package to embed git branch info into builds

I opened this pull request mostly to track remaining open issues. When it's in, I'd like to consider the codebase as 2.0-preview1 and continue further testing ahead of a release.

Files have been moved around heavily, so I expect the diffs here to be pretty ugly. As a result I was fairly careless with my checkins. Honestly, I might delete this branch and re-create with a squashed commit that puts all this in one commit. After this stuff is in, we'll get back to a more normal github flow workflow.

Before merging this PR:

  • TEST, just need to monkey test this thing a bunch
  • Implement some really simple save/load unit testing. I think serializing the sample ROM data will do nicely.
  • Xml is storing the wrong Project.ProjectFileName if you "save as". (Actually... we probably don't need to store this in the XML at all)

Before releasing 2.0:

  • Perf: optimize load/save. Takes about 10 seconds which is livable, I bet that can get cut down greatly.
  • Check licensing on the new NuGet modules. I think they're all MIT or GPL3, which is compatible with us.
  • Revisit .diz vs .dizraw save format naming. I think .diz.xml or .diz.bin would be good.
  • rename project? I like the "Diz" part, typing "Diztinguish" is kind of a mouthful : )
  • Application settings.defaults (for storing last file opened/etc) needs to be moved to ProjectController and probably tested
  • test out some invalid situations, like loading a broken ROM, save/load fails. make sure user has enough breadcrumbs to know what's going on.
  • fix I broke the sample text label output #17 - labels aren't generating correctly in asm
  • review open pull requests
  • finish testing external bugfix I wrote in Asar https://github.com/RPGHacker/asar/pull/171/files, this is a situation where Asar is erroring despite Diz outputting the right thing (related to relative branch statements + labels)
  • check that XML format migration support is working. I'd like this to be ready out of the box so we don't have to do any weird stuff later like "figure out if this is the old save format before migrations worked"
  • probably more stuff

@Dotsarecool can I get maintainer access to repo? And thanks for writing such a great tool. I am having a lot of fun working on it, though my spare time is kinda limited at the moment.

VitorVilela7 and others added 30 commits April 22, 2019 01:37
I'm afraid these settings only work for my PC (125%), though...
The low byte of DP was affecting the opcode itself.
- store comments specific to labels
- print label comments in labels.asm
- print all labels in all-labels.txt for analysis
- update save file format to v2
- refactor save file code, combine all versions into one
it can now deal with being pasted stuff like:
$C1/08A7

probably needs a bit more testing to make sure validation handles ROM memory bounds correctly, etc.
- faster
- allow selection of multiple files
- add progress bar UI funtionality
…ess level

- also, move RomMapMode and RomSpeed out of Data
- was incorrectly adding an extra line

add unit tests!
- make ProjectFileManager.cs not reliant on having a rom prompt function (easier to test or handle differently now)
- add function to get true SMC ROM bytes
- add working full load/save cycle unit tests
- fix existing tests
… of a wash for optimization but a little neater, so leaving it.

- add unit tests for performance testing load/save
# Conflicts:
#	DiztinGUIsh/window/dialog/BSNESTraceLogBinaryMonitorForm.UI.cs
@binary1230
Copy link
Collaborator Author

Latest visualizer for ROM (and, it updates in realtime, so you can do live tracelog capturing and watch it fill in unknown areas. still a little slow but it does work)
image

@binary1230
Copy link
Collaborator Author

@gocha I just wanted to show you since you wrote the original code, I extended it and added BSNES support to be able to read the same tracelog data in realtime over a socket. Combined with the new visual window, you can literally watch the ROM get marked up in realtime in Diztinguish:

This is a sped up video of about a 1 minute run (full version is here: https://www.youtube.com/watch?v=NCZUESf82Rg&feature=youtu.be)

ezgif com-gif-maker

I put the details in the BSNES pull request I just opened: devinacker/bsnes-plus#268

@binary1230
Copy link
Collaborator Author

alright, I checked off the main items on my list and I'm ready to merge this back into master. I still want to clean up some further things before we do an official 2.0 release but, it's pretty stable. I'll get a preview build out shortly.

@binary1230 binary1230 linked an issue Oct 29, 2020 that may be closed by this pull request
@binary1230 binary1230 merged commit b99b9c7 into IsoFrieze:master Oct 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

I broke the sample text label output Bunch of random improvements in my branch
3 participants