[Minor][BugFix] Remove PyTest Dependency

[zxybazh]

This PR fixes the dependency on pytest as optional for mlc-llm as shown in mlc-ai/llm-perf-bench#11.

CC @junrushao

[tvm-bot]

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

• No users to tag found in teams: minor, bugfix _{See #10317 for details}

_{Generated by tvm-bot}