Support mistral and sliding window attention

[grimoire]

• Mistral 7b v0.1
• sliding window attention kernel
• window block manager
• rename params

Important

This PR contains refactoring of the engine core mechanism.

[zhyncs]

@grimoire is very productive, and the support for new models on the PyTorch engine is very timely. Currently, Mistral, Qwen 1.5, and DeepSeek MoE all rely on sliding window attention. @lvhan028 @RunningLeon Will we consider prioritizing this PR for review and merging it as soon as possible? Given that these are relatively large features, the community would likely appreciate being able to use them sooner rather than later.

[RunningLeon]

@grimoire is very productive, and the support for new models on the PyTorch engine is very timely. Currently, Mistral, Qwen 1.5, and DeepSeek MoE all rely on sliding window attention. @lvhan028 @RunningLeon Will we consider prioritizing this PR for review and merging it as soon as possible? Given that these are relatively large features, the community would likely appreciate being able to use them sooner rather than later.

@zhyncs Hi, feel free to review and test this PR. Any comment would be sincerely appreciated.

[lvhan028]

This PR makes a great change. QA needs more time to test it.
I am afraid it cannot catch the latest v0.2.4 version, which will be released this week.
We'll release it in v0.2.5 next week.

[lvhan028]

@RunningLeon Is there evaluation result of this PR?

[RunningLeon]

@RunningLeon Is there evaluation result of this PR?

here are the results. There might be something wrong.

[lvhan028]

@zhulinJulia24 may perform regression test.

[RunningLeon]

@lvhan028 The perform test is OK as here

[RunningLeon]

LGTM