🏂 Bert SNPE conversion & evaluation case

[trajepl]

Describe your changes

1. Add Bert SNPE conversion & evaluation case
2. Fix bugs in raw dataloder and snpe evaluator
3. Expose max_length for huggingface pre_process_data

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

[jambayk]

Thanks for the changes

[guotuofeng]

do we need rename the parameter batch_size of FileListCommonDataLoader to file_chunk_size?

[trajepl]

Maybe not. There are still many other cases which create the file list data loader directly. Let us say user prepare the local inputs text(like what we did for mobilenet/inception/vgg case), they still need the concept of batch_size to run batch inputs.

[guotuofeng]

In mobilenet/inception/vgg case, the batch_size is used as normal batch_size like dataset? I see the source code https://github.com/microsoft/Olive/blob/main/olive/platform_sdk/qualcomm/utils/data_loader.py#L73 use the batch size as chunk. No?

[trajepl]

For the file list data loader, the batch size always performs like file chunk.

I am thinking for Mobilenet case, when user create the the local file and file list data loader, then they begin to iterate the data loader. Then batch_size may looks better? As it is a common concept in any data loader.

[trajepl]

If you insist to choose the file_chunk_size, maybe I can do it in a separated PR? I do not know how many example configs need to be updated.

[guotuofeng]

does the parameter changes need to update the config json?

[guotuofeng]

I am just thinking the batch_size is a common term associated with dataloader. If we use same term but for different purpose, we'd better use a different name.

[guotuofeng]

Please split the batch_size and file_chunk_size in next PR.