Long context summarize demo

[HALIS-sh]

No description provided.

[research4pan]

Some revisions can be made before merge.

contrib/long-context/hf_sft_full_finetune.sh

• [Style] line 1: shebang line #!/bin/bash can be added to specify the default shell.
• [Bug] line 2: avoid exporting CUDA_VISIBLE_DEVICES to make this script runnable on single-gpu servers.
• [Bug] line 5: please use a general huggingface model name instead of a local name. Currently the script cannot be run on other's servers.

contrib/long-context/hf_sft_lora_flashattn.sh

• [Style] line 1: shebang line #!/bin/bash can be added to specify the default shell.
• [Style]: The naming can be improved: why there is flashattn in the name?
• [Bug] line 2, 5: similar to contrib/long-context/hf_sft_full_finetune.sh
• [Architecture]: I didn't see lora options in the script. I am wondering if this script is really doing lora training?

contrib/long-context/sft_summarizer.py

• [Style] line 1: shebang line #!/usr/bin/env python can be added.
• [Style] line 1: string encoding # coding=utf-8 is highly recommended to be specified.
• [Bug] line 7, 24: put them in main would be better. Otherwise other .py modules loading this one would potentially execute those codes as well, causing unintended and hard-to-locate bugs.
• [Bug] line 29: should not include your own personal wandb key in a public repo, otherwise others' experiments will be messed up with your. Even worse, there can be a security issue for your own account.
• [Style] line 35-40, 56, 72, 90-91, 109-110, 112: use logging instead of print, and restrict the logging level to debug if this information is not necessary
• [Style] line 98-102: the indentation here can be significantly improved, please refer to google coding style (https://google.github.io/styleguide/pyguide.html).

[research4pan]

LGTM, thanks!

[HALIS-sh]