[TRACKING] Follow-up Enhancements

[ganler]

The main features are refactored in #33 with a primary focus on extensibility, code quality, and maintainability (e.g., ci & PyPI). This thread tracks some follow-up (or P1~2) enhancements and features that we are going to work on in the next few weeks.

0.0.0

https://pypi.org/project/nnsmith/0.0.0/

• Refactoring verbose printing with module-level loggers. [LOG] Replace printing with modularized logging #44
• Auto versioning Add some TF ops and add XLA backend #40
• Dtype testing for model exportation; Add some TF ops and add XLA backend #40
• Make pygraphviz an optional dependency as it requires system-level constraint fix: relax constraints and fix nnsmith.fuzz #43

0.0.1

https://pypi.org/project/nnsmith/0.0.1/

• Stabilize nnsmith.fuzz via smoke testing in CI; Adding backend_target for ModelType.init for working around TF-XLA #50
• Fix core.Gemm; support IREE as a TF backend #45
• Double-check and stabilize backend-oriented spec re-writing; [gen][fuzz] impl backend constraints and fuzzing loop generation exp safety #52
• generation exception safety. [gen][fuzz] impl backend constraints and fuzzing loop generation exp safety #52
• Well-form backend dispatching;
• GitHub HEAD install (tested);
• Fix XLA. Adding backend_target for ModelType.init for working around TF-XLA #50
• Patchable bug filter. [fuzz] a patchable bug filter #53
• Crash safety in execution; feat: crash safety in fuzzing loop #54

TBD

• Cleaning stale branches after a 5-month window size;
• nnsmith.fuzz TUI.
• Implement exclude set logics;
• Replace pygraphviz with graphviz to release the dependency constraint by pygraphviz?
• Simplifying nnsmith arith by extending z3.ArithRef. floor div for symbolic integers in Python Z3Prover/z3#6309
• Well-form core operators and dialects; -- core operators should not change that frequently...
• Simplifying and bringing back attribute binning;
• CLI-level documentation;
• Tutorials on CLI, design, and extension.

[ganler]

Some new and main features/fixes that are not mentioned in the base comment, but will be served as the future release of v0.1.0:

GraphIR infrastructure for simplifying graph construction and mutation:

• Upstream sharpened GraphIR #64
• Simple Graph IR to unify and facilitate DNN graph manipulation #61

Scripts to evaluate coverage of generated tests:

• feat: enabling coverage viz for fuzz-test #60

Concolic generation, simplified size limiter, etc.

• feat bundle: concolic generation, adapting latest TF, and more #66

More backend support is coming soon.

[ganler]

Continued by tracking features via release versions or other smaller milestones.