Implement newmm tokenizer with PyThaiNLP-compatible API

[Copilot]

Implementation Plan for newmm Tokenizer

• Create Trie data structure class for dictionary management
• Create TCC (Thai Character Cluster) tokenizer implementation
• Create newmm tokenizer implementation following PyThaiNLP's API
• Add a basic Thai word dictionary file
• Create Word tokenizer class to match PyThaiNLP's API
• Add comprehensive unit tests for the new tokenizer
• Update Program.cs to demonstrate the newmm tokenizer usage
• Run tests to validate the implementation (22/22 tests passing)
• Add comprehensive README documentation
• Update GitHub Actions workflow to use .NET 8.0

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Fork newmm tokenizer to thainlp.net</issue_title>
<issue_description>Fork newmm tokenizer to thainlp.net

API should like pythainlp.

PyThaiNLP: https://github.com/PyThaiNLP/pythainlp</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Fork newmm tokenizer to thainlp.net #3

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[wannaphong]

@copilot Fix .github/workflows/dotnetcore.yml

[Copilot]

@copilot Fix .github/workflows/dotnetcore.yml

Updated the workflow to use .NET 8.0 (setup-dotnet@v4, checkout@v4, dotnet-version: '8.0.x') to match the project's target framework. Verified with Release build and all 22 tests passing. (23d375e)