Skip to content

v1.2.1: Fix External GGUF Model Loading & Exception Swallow

Choose a tag to compare

@zmxnpquryet-gif zmxnpquryet-gif released this 07 Sep 16:24
· 65 commits to main since this release

Local LLM Android v1.2.1 Release Notes

๐Ÿ› ๏ธ GGUF & ์™ธ๋ถ€ ๋ชจ๋ธ ๋กœ๋”ฉ ์น˜๋ช…์  ๋ฒ„๊ทธ ์™„์ „ ํ•ด๊ฒฐ

  • llama.cpp ๋„ค์ดํ‹ฐ๋ธŒ ์ปจํ…์ŠคํŠธ ์ง์ ‘ ์ธ์Šคํ„ด์Šคํ™” (LlamaContext):
    • ๊ธฐ์กด \LlamaAndroid.startEngine\์ด ๋‚ด๋ถ€ ์˜ˆ์™ธ๋ฅผ ๊ฐ€๋กœ์ฑ„
      ull\๋กœ ์‚ผ์ผœ๋ฒ„๋ ค \llama.cpp contextId ๋ฐ˜ํ™˜ ์‹คํŒจ: null\์„ ๋ฐœ์ƒ์‹œํ‚ค๋˜ ์น˜๋ช…์  ๊ฒฐํ•จ์„ ์›์ฒœ ์ œ๊ฑฐ
    • ๋„ค์ดํ‹ฐ๋ธŒ ์˜ˆ์™ธ ๋ฐœ์ƒ ์‹œ ์‹ค์ œ ์—๋Ÿฌ ์‚ฌ์œ  ๋ฐ ์›์ธ ์ฝ”๋“œ๋ฅผ ์‹ค์‹œ๊ฐ„ ์ง„๋‹จ ๋Œ€ํ™”์ƒ์ž์— ์ƒ์„ธ ํ‘œ์ถœ
  • ์•ˆ๋“œ๋กœ์ด๋“œ ์ €์žฅ์†Œ ํŒŒ์ผ ๋””์Šคํฌ๋ฆฝํ„ฐ ์ง๊ฒฐ (ParcelFileDescriptor.open):
    • \Uri.parse\ ๋ฐ \openInputStream\ ์‹คํŒจ ์ทจ์•ฝ์ ์„ ์šฐํšŒํ•˜์—ฌ ๊ธฐ๊ธฐ ์ €์žฅ์†Œ/์™ธ๋ถ€์—์„œ ๊ฐ€์ ธ์˜จ ํŒŒ์ผ๋„ OS ๋„ค์ดํ‹ฐ๋ธŒ ๋ ˆ๋ฒจ์—์„œ ์•ˆ์ „ํ•˜๊ฒŒ FD ์˜คํ”ˆ
  • 6๋‹จ๊ณ„ ์ง€๋Šฅํ˜• ํด๋ฐฑ ๋ณต๊ตฌ ๋กœ๋” (Multi-Stage Fallback Loader):
    • 1๋‹จ๊ณ„: ์ปค๋„ \mmap\ + ๋‚ด์žฅ/์™ธ์žฅ ๋น„์ „ํƒ€์›Œ ์—ฐ๋™
    • 2๋‹จ๊ณ„: ์ง์ ‘ ๋ฉ”๋ชจ๋ฆฌ ๋กœ๋“œ (\use_mmap = false) + ๋น„์ „ํƒ€์›Œ ์—ฐ๋™
    • 3๋‹จ๊ณ„: ํ‘œ์ค€ ํ…์ŠคํŠธ ๋””์ฝ”๋” ๋ชจ๋“œ (์‚ฌ์šฉ์ž ์„ค์ • ์ปจํ…์ŠคํŠธ ์œˆ๋„์šฐ)
    • 4๋‹จ๊ณ„: ์ง์ ‘ ๋ฉ”๋ชจ๋ฆฌ ๋ชจ๋“œ (mmap ๋ฏธ์ง€์› ์ปค๋„ ๋Œ€์‘)
    • 5๋‹จ๊ณ„: ์ปจํ…์ŠคํŠธ ์ถ•์†Œ ์•ˆ์ •ํ™” ๋ชจ๋“œ (2048 ctx)
    • 6๋‹จ๊ณ„: ์ €์‚ฌ์–‘ ๊ธฐ๊ธฐ ์ ˆ์ „ ๋ชจ๋“œ (1024 ctx)
  • ์ปจํ…์ŠคํŠธ ๋ˆ„์ˆ˜ ๋ฐฉ์ง€ ๋ฐ ๋ฆฌ๋ฐ‹ ํ™•์žฅ:
    • \LlamaAndroid\ ๊ธฐ๋ณธ ํ•œ๋„(1)๋ฅผ 16์œผ๋กœ ํ™•์žฅํ•˜์—ฌ ๋ชจ๋ธ ์ „ํ™˜ ์‹œ ์ฆ‰๊ฐ ๋กœ๋“œ ๊ฐ€๋Šฅ
    • ๋ชจ๋ธ ์–ธ๋กœ๋“œ ์‹œ ๊ธฐ์กด ๋„ค์ดํ‹ฐ๋ธŒ ์ปจํ…์ŠคํŠธ ๋งต์„ ์ „์ˆ˜ ํ•ด์ œํ•˜์—ฌ ํฐ RAM ๋ˆ„์ˆ˜ ์ฐจ๋‹จ