Merge from private version. Adds support for Ma2 charts.

CHANGELOG.md

@@ -0,0 +1,90 @@
+# 0.12.0
+* Add support for Simai ex notes written as "ex" instead of just "x". [Reported]
+* Force CRLF for Sdt and Ma2 export.
+* Fixed bug when a note and bpm change happens at the same time causing gap to be incorrectly computed. [Report: Kyan-pasu]
+* Handle division by zero in Simai chart duration parsing. [Reported]
+* Refactored Simai chart export solving the following bugs:
+  * Simai divisors ("{}") sometimes don't appear at the beginning after the initial bpm declaration. [Report: Kyan-pasu]
+  * Incorrect computation in `get_measure_computation` function. [Report: Kyan-pasu]
+  * Better handling of float arithmetic.
+* Fix bug that causes a slide note to ignore pseudo each modifier. [Reported]
+* Fix error in prepending the first 0x10 bytes in chart and db encryption.
+* Handle case when decrypted chart or database doesn't have gzip magic number.
+* Fix bug in decryption when padding is 0.
+* Added more tests and documentation.
+
+# 0.11.0
+* Rewritten lark grammar to utilize LALR parser for around 50% speed improvement.
+* Tweaked how debug information is presented.
+
+# 0.10.1
+* Added debug information when parsing Simai charts and files.
+* Minor changes to Simai class.
+
+# 0.10.0
+* Added support for simai pseudo each \`. Current implementation is to offset the succeeding note by 1/384 (384 is ma2's default resolution.) [Reported]
+* Added support for simai notes with 0 positions (e.g. 3/0, 0>0[4:1]). Current implementation is to ignore such notes. [Reported]
+* Added support for simai touch notes with regions A and D. Current implementation is to ignore such notes. [Reported]
+* Added support for hold notes where the modifier is first (e.g. 3xh, Cfh). [Reported]
+* Ma2 parsing no longer ignores MET events.
+* Replaced the word zone with region in touch notes. I used to refer to DX's C, B, and E touch regions as zones. I have since changed them.
+
+# 0.9.0
+* Rewritten offset functions to accept inputs in terms of seconds and fractions of measures: "0.5s", "2s", or "1/64"
+* Fixed offset not taking into account BPM changes [Report: Kyan-pasu]
+* Fixed erroneous check in after_next_measure duration check in `get_rest` in simai module [Reported]
+* Add suppport for integer values for simai `&first` fields [Reported]
+* Fix simai touch notes parsing. [Reported]
+* Removed deprecated MaiFinaleCrypt class.
+
+# 0.8.0
+* Rewritten Ma2 parsing to be more flexible for future versions of the format.
+* Added support for Ma2 version 1.02.00 [Report: StonerSto].
+* Fixed `add_touch_hold` and `del_touch_hold` not using `THO` key for updating note statistics.
+* Added ma2 parsing tests.
+* Added \_\_main\_\_.py to allow `python -m maiconverter arguments here`
+
+# 0.7.1
+* Fixed ma2 parsing of ex star notes [Report: StonerSto].
+
+# 0.7.0
+* Added error handling in parallel Simai chart parsing for Windows [Report: Kurimu Pantsu].
+* Added support for simai hold notes that don't have duration [Report: Kurimu Pantsu].
+* More tests are added for Simai file parsing.
+* Added `parse_file_str` for Simai module to parse Simai files in string format rather than opening a file.
+* Added `finale_encrypt` and `finale_decrypt` to reduce repetition in code and provide users a way to encrypt/decrypt strings by themselves.
+
+# 0.6.5
+* Added two more fields specific to Maipad plus: `demo_seek` and `demo_len` [Report: Kurimu Pantsu]. Current implementation is to ignore these fields.
+
+# 0.6.2 - 0.6.4
+* Various bug fixes in Simai file parsing and Simai chart exporting [Report: Kurimu Pantsu].
+* Command-line tool now uses the new finale crypt functions instead of the soon-to-be removed `MaiFinaleCrypt`.
+* Updated neglected tests module with more tests coming.
+
+# 0.6.1
+* Added the old scripts back in the new `scripts` folder.
+
+# 0.6.0
+Added four functions in MaiCrypt package: `finale_db_encrypt`, `finale_db_decrypt`, `finale_chart_encrypt`, and `finale_chart_decrypt`. The chart functions are simply the old `MaiFinaleCrypt` class turned into functions. While the db functions are more suited for encrypting and decrypting Finale's database files. Included in the new db functions are easy handling of UTF-16. In accordance with this, the `MaiFinaleCrypt` class is now pending deprecation and will be removed in 0.9.0.
+
+`del_slide` methods in all three chart classes now require a third parameter, `end_position`, to prevent deleting multiple slides that start at the same button and measure.
+
+# 0.5.0
+Most of the changes were made to make making charts for the three formats easier and consistent across all the formats.
+
+Added del methods for deleting notes for all three chart classes. Breaking changes are made to Ma2 and SDT classes to unify the method signatures for all three classes. 
+
+In SDT, the StarNote class and add_star method were removed and moved to the TapNote class and add_tap method. Both of them now accept an `is_star` parameter. For the TapNote class an `amount` attribute is added for star notes. A star note's `amount` is now automatically incremented when an `add_slide` method is called that has the same button position and measure.
+
+The `add_slide` method for SDTs no longer need a `slide_id` parameter and has since been removed. The method now automatically assigns a unique `slide_id` when creating a `SlideStart` and `SlideEnd` class.
+
+For Ma2 and SDT classes, the position of `pattern` and `duration` for the `add_slide` method was moved to make it more consistent with other add methods. To avoid code from being broken please use keyword arguments for your programs.
+
+For Simai parallel processing, chunksizes are now based on the number of fragments divided by the amount of available CPU.
+
+# 0.4.0
+Added parrellism in Simai fragments parsing for (small) speed improvements. Ongoing debloat of the SimaiChart class and fix PyLint and MyPy warnings and errors.
+
+# 0.3.3
+First semi-public release of MaiConverter-Private. Has encrypt/decrypt, S\*T, Ma2, and 3Simai support.

MANIFEST.in

@@ -0,0 +1,5 @@
+include CHANGELOG.md
+include README.md
+include LICENSE
+include how_to_make_charts.md
+recursive-include *.lark

README.md

@@ -1,50 +1,117 @@
 # MaiConverter
-Converts standard 3simai files to sdt files so you can play custom charts in your favorite touchlaundry machine. Python done quick and dirty.
-
-Rewrite now includes a proper lark parser for simai and should handle more 3simai quirks. Conversion logic has also been completely rewritten to be more accurate and maintainable. As such a new s\*t to simai converter has been added.
+A Python program for parsing and converting Maimai charts. Made up of two parts: 
+* An importable package for parsing, creating, exporting, and converting SDT, Ma2, and 3Simai charts. And
+* a commandline script for parsing and converting the 3 formats.
 
 If you're not familiar with these file formats, then you can read about sdt files [here](https://listed.to/@donmai/18173/the-four-chart-formats-of-maimai-classic) and simai files [here](https://w.atwiki.jp/simai/pages/25.html). You can read about my blog post about this [here](https://listed.to/@donmai/18284/newly-released-simai-to-sdt-converter)
 
-If you're interested in anything maimai modding related, then go join [MaiMai TEA](https://discord.gg/82UR3e2akE) in Discord.
-
 # Dependencies 
 * [Pycryptodome](https://pypi.org/project/pycryptodome)
 * [Lark](https://pypi.org/project/lark-parser)
 
-# Usage
-## simai_to_sdt.py
-Converts a simai file or a directory containing simai files to sdt.
+# Commandline
+The command-line script, installed as part of the package, can parse, convert, encrypt, or decrypt Maimai chart formats. The general form is:
+
+```maiconverter COMMAND /path/to/file/or/directory```
+
+`COMMAND` can be the following, with descriptions later:
+* encrypt
+* decrypt
+* ma2tosdt
+* ma2tosimai
+* sdttoma2
+* sdttosimai
+* simaifiletoma2
+* simaifiletosdt
+* simaitoma2
+* simaitosdt
+
+The second positional argument is the path to a chart file or directory. If given a directory, it will convert all relevant files found in the directory.
+
+The program will save all converted file in the "output" folder in the input file's parent directory or the input directory. It will make an "output" folder if there is no existing folder.
+
+## encrypt, decrypt
+
+These commands will either encrypt an S\*T chart file to their equivalent S\*B file or vice versa.
+**Requires**: -k or --key parameter followed by a hexadecimal AES key. The program can encrypt or decrypt a **table** by adding a -db or --database toggle parameter.
+
+### Example
+Convert an SDT to SDB:
+
+```maiconverter encrypt --key 0xFEDCBA9876543210 100_songname_02.sdt```
+
+Convert an SCB to SCT:
+
+```maiconverter decrypt --key 0xFEDCBA9876543210 252_donmaime_05.scb```
+
+Decrypts an encrypted table:
+
+```maiconverter decrypt --key 0xFEDCBA9876543210 mmtablename.bin```
+
+## sdttoma2, sdttosimai
+
+These commands will either convert an S\*T file to ma2 or Simai, respectively. **Requires** -b or --bpm parameter followed by the song's BPM as either an int or float. 
+**Note**: For sdttosimai, it does not produce a complete Simai file. sdttosimai only generates a Simai chart.
+
+### Example
+Convert a 200 bpm SRT file to Simai:
+
+```maiconverter sdttosimai --bpm 200 300_segapls.srt```
+
+Convert a 130 bpm SDT file to Ma2:
+
+```maiconverter sdttoma2 --bpm 130 301_dontsue.sdt```
+
+## ma2tosdt, ma2tosimai
+
+These commands will either convert a Ma2 file to SDT or Simai, respectively.
+**Note**: For ma2tosimai, it does not produce a complete Simai file. ma2tosimai only generates a Simai chart.
+
+### Example
+Convert a Ma2 file to SDT:
+
+```maiconverter ma2tosdt 000404_02.ma2```
+
+Convert a Ma2 file to Simai:
+
+```maiconverter ma2tosimai 001401_04.ma2```
 
-**NOTE**: This is a proof of concept and does **not** accept a complete simai file. The input file should only contain the chart for one difficulty with no "&inote_=" or any simai fields.
+## simaitoma2, simaitosdt
 
-All output files are stored in 'output' folder in the same directory as the input. Unless output directory is specified by using the --output or -o parameter. Touch notes can be converted by adding the --convert-touch or -ct parameter. You can offset all the notes by adding the --delay or -d parameter.
+These commands convert a text file containing only a Simai chart file to either a Ma2 or SDT, respectively.
 
-```python simai_to_sdt.py /path/to/file/or/directory```
+## simaifiletoma2, simaifiletosdt
 
-## sxt_to_simai.py
-Converts an s\*t file or a directory containing s\*t files to simai.
+These commands differ from the previous by parsing an entire maidata.txt. All charts are individually converted to a Ma2 or SDT, respectively.
 
-**NOTE**: This is a proof of concept and does **not** produce a complete simai file. The output file only contains the chart for one difficulty with no "&inote_=" or any simai fields.
+# Misc commandline arguments
+## -o, --output
+Specify an output directory, or it defaults to the input directory.
 
-All output files are stored in 'output' folder in the same directory as the input. Unless output directory is specified by using the --output or -o parameter. You need to specify the bpm of the song by using the `--bpm` parameter. You can offset all the notes by adding the --delay or -d parameter.
+## -d, --delay
+If you want to apply an offset to every converted chart's notes, you can do so using this argument. It accepts both negative and positive offsets in terms of measures.
 
-```python sxt_to_simai.py --bpm 120 /path/to/file/or/directory```
+## -ct, --convert_touch
+If converting from Ma2 or Simai to SDT, you can add this toggle to (naively) convert touch notes to regular tap and hold notes. Useful when you want to manually convert touch notes to tap and hold notes. You just need to modify the note's button, no need to figure out the timing.
 
-## encrypt_decrypt.py
-Encrypts and decrypts finale files. S\*T files are converted to S\*B files and vice versa. Database files can be encrypted and decrypted by using the --database or -db parameter.
+## -md, --max-divisor
+Sets the max Simai divisor ("{}") that is allowed when exporting a Simai chart. Set it to a low number like 128, should you want a more readable output. Defaults to 1000. 
 
-All output files are stored in 'output' folder in the same directory as the input. Unless output directory is specified by using the --output or -o parameter
+# Python package
+If you installed the wheel file, you could import the program like a standard Python package. If you want to make a chart maker or GUI frontend for this converter, please use it. See `how_to_make_charts.md` for an introductory guide on using MaiConverter for chart making. There is also (incomplete) documentation for classes and functions in the package. See licensing below.
 
-### Encrypting
-```python encrypt_decrypt.py encrypt 'AES KEY HERE IN HEX' /path/to/file/or/directory```
+# TODOS
+* Documentation
+* Do all the `TODO`s scattered in the package
+* Reduce jank
 
-### Decrypting
-```python encrypt_decrypt.py decrypt 'AES KEY HERE IN HEX' /path/to/file/or/directory```
+# Contact
+If you have questions or bug reports, send me a DM or ping me at MaiTea Discord server.
 
-# TODOs
-Contributions are welcome and appreciated just make sure to format using Black.
-* simai_to_sdt.py should accept an actual simai file with simai fields (e.g. '&title=', '&inote_=')
-* Do `TODO`s scattered in the library especially in the lark file.
+* Discord: donmai#1493
+* Twitter: @donmai_me
+* GitHub: donmai-me
+* Listed.to: @donmai
 
 # License
-This is an open-sourced application licensed under the [MIT License](https://github.com/donmai-me/MaiConverter/blob/master/LICENSE)
+This is an open-sourced application licensed under the MIT License

how_to_make_charts.md

@@ -0,0 +1,92 @@
+# Introduction
+Are you a depraved charter tired of making Simai charts by hand on your text editor? Or are you a new Maimai charter that feels intimidated by Simai's convoluted format? Do you want to focus on writing charts by adding notes with only the required information? Suppose you don't mind using the command line. In that case, you can use MaiConverter to make and export your Simai (and other chart formats) easily.
+
+> Note: This isn't a full documentation/walkthough of every feature in the SimaiChart class in MaiConverter. You are encouraged to read the source files to get a better understanding of MaiConverter (and so you can help me document the code.)
+
+# Import your existing chart
+To import an existing chart, strip all Simai metadata until you're left with only the chart text. One chart difficulty at a time and no `&inote=`s. Save this as a separate file and import it to Simai.
+
+```python
+>>> from maiconverter.simai import SimaiChart
+>>> with open("simai_chart.txt", "r") as f:
+...     text = f.read()
+...
+>>> simai = SimaiChart.from_str(text)
+```
+
+Suppose there are no errors during parsing. Then you're left with a SimaiChart instance that has all your work imported.
+
+# Chart from scratch
+If you want to start from scratch, then import SimaiChart and make an instance.
+
+```python
+>>> from maiconverter.simai import SimaiChart
+>>> simai = SimaiChart()
+```
+
+# Setting BPMs
+If you're starting from scratch, you should first set your starting BPM. First, specify the measure the BPM takes effect, then the BPM. Let's define the starting BPM as 220:
+
+```python
+>>> simai.set_bpm(1.0, 220)
+```
+
+Should you change your mind, you can use `set_bpm` again, and it will automatically overwrite the previously set BPM.
+
+```python
+>>> simai.set_bpm(1.0, 300)
+```
+
+You can remove a BPM change by using `del_bpm`, providing only the measure of the BPM change.
+
+```python
+# Change BPM to 200 at measure 13
+>>> simai.set_bpm(13, 200.0)
+# Oops we meant measure 14
+>>> simai.del_bpm(13)
+>>> simai.set_bpm(14, 200)
+```
+
+# Adding (and deleting) notes
+**NOTE**: Although the Simai buttons and touch screen locations start at 1 and end at 8, MaiConverter begins at 0 and ends at 7. Why? Because I'm a programmer, and the arcade chart formats also start at 0. MaiConverter automatically adds 1 to every button when exporting.
+
+```python
+# Add an ex tap note for first measure in button 0 and a hold note at measure 1.5 at button 5 for 0.25 measures
+>>> simai.add_tap(1.0, 0, is_ex=True)
+>>> simai.add_hold(1.5, 5, 0.25)
+...
+# Add a > slide at measure 50.5 from button 5 to 1 with a duration of 0.75 measures.
+>>> simai.add_tap(50.5, 5, is_star=True)
+>>> simai.add_slide(50.5, 5, 1, 0.75, ">")
+# Add a touch tap note in measure 70, at E5
+>>> simai.add_touch_tap(70, 5, "E")
+# Add a firework touch hold note in measure 70.5, at C with a duration of 1 measure.
+>>> simai.add_touch_hold(70.5, 0, "C", 1, is_firework=True)
+```
+
+If you don't know the order of arguments of an `add` function, then invoke the function using keywords. Invoking an `add` function with keyworded arguments is recommended for frontend programs.
+
+```python
+# Add an ex tap note for first measure in button 0 and a hold note at measure 1.5 at button 5 for 0.25 measures
+>>> simai.add_tap(measure=1.0, position=0, is_ex=True)
+>>> simai.add_hold(measure=1.5, position=5, duration=0.25)
+...
+# Add a > slide at measure 50.5 from button 5 to 1 with a duration of 0.75 measures.
+>>> simai.add_tap(measure=50.5, position=5, is_star=True)
+>>> simai.add_slide(measure=50.5, start_postion=5, end_position=1, duration=0.75, pattern=">")
+# Add a touch tap note in measure 70, at E5
+>>> simai.add_touch_tap(measure=70, position=5, zone="E")
+# Add a firework touch hold note in measure 70.5, at C with a duration of 1 measure.
+>>> simai.add_touch_hold(measure=70.5, position=0, zone="C", duration=1, is_firework=True)
+```
+
+For every `add_tap` and `add_slide` functions, there are corresponding `del_tap` and `del_slde` functions. The only parameters that the del functions need are the measure and button where a note happened. For `del_touch_tap` and `del_touch_hold` functions, the measure, location, and touch zone are required.
+
+```python
+# Delete tap note at measure 6.125 at button 0
+>>> simai.delete_tap(6.125, 0)
+# Delete hold note starting at measure 12 at button 5
+>>> simai.delete_hold(12, 5)
+# Delete slide note starting at measure 9, starting at button 2, ending at button 7
+>>> simai.delete_slide(9, 2, 7)
+```

maiconverter/__init__.py

@@ -1,27 +1,7 @@
-from .event import EventType
-from .note import NoteType
-from .crypt import MaiFinaleCrypt
-from .maisdt import MaiSDT, sdt_note_to_str
-from .simai import (
-    SimaiChart,
-    SimaiHoldNote,
-    SimaiTapNote,
-    SimaiSlideNote,
-    SimaiTouchTapNote,
-    SimaiTouchHoldNote,
-    SimaiBPM,
-    simai_pattern_to_int,
-    simai_slide_to_pattern_str,
-    simai_pattern_from_int,
-    simai_get_rest,
-    simai_convert_to_fragment,
-    simai_parse_chart,
-    slide_is_cw,
-    slide_distance,
-)
-from .simai_lark_parser import SimaiTransformer
+from importlib.metadata import version, PackageNotFoundError
+from .maiconverter import main
 
-from .maisdttosimai import sdt_to_simai
-from .simaitomaisdt import simai_to_sdt
-
-__version__ = "2.1.1_Public"
+try:
+    __version__ = version("maiconverter")
+except PackageNotFoundError:
+    __version__ = "not installed"

maiconverter/__main__.py

@@ -0,0 +1,3 @@
+from .maiconverter import main
+
+main()

maiconverter/converter/__init__.py

@@ -0,0 +1,6 @@
+from .maima2tomaisdt import ma2_to_sdt
+from .maima2tosimai import ma2_to_simai
+from .maisdttomaima2 import sdt_to_ma2
+from .maisdttosimai import sdt_to_simai
+from .simaitomaima2 import simai_to_ma2
+from .simaitomaisdt import simai_to_sdt