v0.0.130

• New Features
  • Added support for processor-based text generation, allowing use of processors in addition to tokenizers for both synchronous and streaming output.
  • Introduced new functions to load processors and to generate text using processors.
• Bug Fixes
  • Improved conditional logic for handling model, tokenizer, and processor inputs to ensure correct generation pathways and error handling.
• Refactor
  • Updated generation and output functions to flexibly handle different return formats and decoding methods based on available processor or tokenizer.

Walkthrough

This change introduces processor-based generation support to the codebase. It adds methods for loading and using AutoProcessor objects from the transformers library, updates the generation logic to handle processor-based workflows alongside tokenizer-based ones, and exposes processor-related functions through the module’s public API.

Changes

File(s)	Change Summary
app/predacons/init.py	Added load_processor to the list of exported functions.
app/predacons/src/generate.py	Implemented processor loading and processor-based text generation (sync and streaming); updated public API.
app/predacons/src/predacons.py	Refactored generation logic to support processor-based workflows; added load_processor function; adjusted output handling.

What's Changed

• Feature/update transformer by @shouryashashank in #49
• 50 add support for processors for multi input types by @shouryashashank in #52
• 50 add support for processors for multi input types by @shouryashashank in #53

Full Changelog: v0.0.129...v0.0.130

v0.0.129

What's Changed

• updated pypdf by @shouryashashank in #48

Full Changelog: v0.0.128...v0.0.129

Walkthrough

The pull request updates the PDF handling across the project by replacing the old PyPDF2 library with pypdf. This change is reflected in the import statement in the data loading module, the dependency specifications in both requirements.txt and setup.py, and the version constraints therein. Additionally, the Predacons version number has been incremented in both the printed rollout message and the package metadata.

Changes

Files	Change Summary
app/.../load_data.py, requirements.txt, setup.py	PDF library transitioned from PyPDF2 to pypdf with updated dependency constraints (requirements: >= 3.9.0; setup: >= 5.3.0).
app/.../predacons.py, setup.py	Version update: Incremented from 0.0.128 to 0.0.129 indicated in the rollout print message and package metadata.

v0.0.128

• New Features

  • Introduced streaming capabilities for text and chat generation, allowing real-time output.
  • Added new functions: text_stream and chat_stream for enhanced streaming functionality.

• Bug Fixes

  • Updated error handling for streaming setup and generation processes with descriptive messages.
    -Examples

# %%
import predacons

# %%
predacons.rollout()

# %%
import os
os.environ['CUDA_VISIBLE_DEVICES'] ='0'
import torch
if torch.cuda.is_available():
    # torch.set_default_device('cuda')
    print("Using GPU")
else:
    print("No GPU available")

# %%
model_path = "Predacon/Pico-Lamma-3.2-1B-Reasoning-Instruct"

# %%
model = predacons.load_model(model_path)
tokenizer = predacons.load_tokenizer(model_path)

# %%
seq = "The quick brown fox jumps over the"

# %%


thread,stream = predacons.text_generate(model=model, tokenizer = tokenizer, sequence = seq, max_length=100, temperature=0.1,stream=True)

# %%
thread.start()
try:
    out = ""
    for new_text in stream:
        out = out + new_text
        print(new_text, end=" ")
finally:
    thread.join()

# %%
a = predacons.text_stream(model=model, tokenizer = tokenizer, sequence = seq, max_length=100, temperature=0.1)

# %%
a

# %%
chat = [
    {"role": "user", "content": "A train travelling at a speed of 60 km/hr is stopped in 15 seconds by applying the brakes. Determine its retardation."},
]

# %%
thread,stream = predacons.chat_generate(model=model, tokenizer = tokenizer, sequence = chat, max_length=500, temperature=0.1,stream=True)

# %%
thread.start()
try:
    out = ""
    for new_text in stream:
        out = out + new_text
        print(new_text, end="")
finally:
    thread.join()

# %%
b = predacons.chat_stream(model=model, tokenizer = tokenizer, sequence = chat, max_length=500, temperature=0.1)

# %%
b

• Documentation

  • Enhanced documentation for functions to include detailed parameter descriptions and functionalities.

• Chores

  • Incremented version number to 0.0.128 and updated package dependencies, removing compatibility concerns.

Walkthrough

This pull request introduces significant enhancements to the Generate class and the predacons module by adding streaming capabilities for output generation. New methods are implemented to allow real-time streaming of both text and chat outputs. The rollout function in predacons.py is updated with a new version number and additional print statements for better documentation. The setup.py file reflects a version increment and removal of specific dependencies, indicating a shift in package requirements.

Changes

File	Change Summary
app/predacons/src/generate.py	- Added methods for streaming outputs: generate_output_stream, generate_chat_output_stream, generate_output_from_model_stream, generate_chat_output_from_model_stream. - Restructured __generate_chat_output to support streaming and updated error handling.
app/predacons/src/predacons.py	- Updated rollout function version from v0.0.126 to v0.0.128. - Added new functions: text_stream, chat_stream. - Updated generate, text_generate, and chat_generate functions to include a stream parameter.
setup.py	- Updated version from 0.0.126 to 0.0.128. - Removed dependencies for torch and bitsandbytes from install_requires.
app/predacons/init.py	- Added text_stream and chat_stream to the list of exported functions.

What's Changed

• Update README.md by @shouryashashank in #44
• Update predacons.py by @shouryashashank in #45
• Feature/quick fix by @shouryashashank in #46

Full Changelog: v0.0.126...v0.0.128

v0.0.127

• New Features

  • Introduced streaming capabilities for text and chat generation, allowing real-time output.
  • Added new functions: text_stream and chat_stream for enhanced streaming functionality.

• Bug Fixes

  • Updated error handling for streaming setup and generation processes with descriptive messages.
    -Examples

# %%
import predacons

# %%
predacons.rollout()

# %%
import os
os.environ['CUDA_VISIBLE_DEVICES'] ='0'
import torch
if torch.cuda.is_available():
    # torch.set_default_device('cuda')
    print("Using GPU")
else:
    print("No GPU available")

# %%
model_path = "Predacon/Pico-Lamma-3.2-1B-Reasoning-Instruct"

# %%
model = predacons.load_model(model_path)
tokenizer = predacons.load_tokenizer(model_path)

# %%
seq = "The quick brown fox jumps over the"

# %%


thread,stream = predacons.text_generate(model=model, tokenizer = tokenizer, sequence = seq, max_length=100, temperature=0.1,stream=True)

# %%
thread.start()
try:
    out = ""
    for new_text in stream:
        out = out + new_text
        print(new_text, end=" ")
finally:
    thread.join()

# %%
a = predacons.text_stream(model=model, tokenizer = tokenizer, sequence = seq, max_length=100, temperature=0.1)

# %%
a

# %%
chat = [
    {"role": "user", "content": "A train travelling at a speed of 60 km/hr is stopped in 15 seconds by applying the brakes. Determine its retardation."},
]

# %%
thread,stream = predacons.chat_generate(model=model, tokenizer = tokenizer, sequence = chat, max_length=500, temperature=0.1,stream=True)

# %%
thread.start()
try:
    out = ""
    for new_text in stream:
        out = out + new_text
        print(new_text, end="")
finally:
    thread.join()

# %%
b = predacons.chat_stream(model=model, tokenizer = tokenizer, sequence = chat, max_length=500, temperature=0.1)

# %%
b

• Documentation

  • Enhanced documentation for functions to include detailed parameter descriptions and functionalities.

• Chores

  • Incremented version number to 0.0.127 and updated package dependencies, removing compatibility concerns.

Walkthrough

This pull request introduces significant enhancements to the Generate class and the predacons module by adding streaming capabilities for output generation. New methods are implemented to allow real-time streaming of both text and chat outputs. The rollout function in predacons.py is updated with a new version number and additional print statements for better documentation. The setup.py file reflects a version increment and removal of specific dependencies, indicating a shift in package requirements.

Changes

File	Change Summary
app/predacons/src/generate.py	- Added methods for streaming outputs: generate_output_stream, generate_chat_output_stream, generate_output_from_model_stream, generate_chat_output_from_model_stream. - Restructured __generate_chat_output to support streaming and updated error handling.
app/predacons/src/predacons.py	- Updated rollout function version from v0.0.126 to v0.0.127. - Added new functions: text_stream, chat_stream. - Updated generate, text_generate, and chat_generate functions to include a stream parameter.
setup.py	- Updated version from 0.0.126 to 0.0.127. - Removed dependencies for torch and bitsandbytes from install_requires.
app/predacons/init.py	- Added text_stream and chat_stream to the list of exported functions.

v0.0.126

New Features

• Incremented version number of the predacons package to 0.0.126.

• Introduced the PredaconsEmbedding class for generating sentence embeddings using a pre-trained transformer model.

• Bug Fixes

• Removed problematic dependencies (torch and bitsandbytes) from the installation requirements to prevent potential installation issues.

Chores

• Deleted the GitHub Actions workflow for automating package uploads to Test PyPI.
• Enhanced documentation within the predacons module to clarify function parameters and purposes.

Example

# Generate embeddings for sentences
from predacons.src.embeddings import PredaconsEmbedding

# this embedding_model object can be used directly in every method langchain   
embedding_model = PredaconsEmbedding(model_name="sentence-transformers/paraphrase-MiniLM-L6-v2")
sentence_embeddings = embedding_model.get_embedding(["Your sentence here", "Another sentence here"])

What's Changed

• removed prerelease workflow by @shouryashashank in #39
• Feature/add embedding by @shouryashashank in #41
• Feature/add embedding by @shouryashashank in #42

Full Changelog: v0.0.125...v0.0.126

v0.0.125

New Features

Introduced a new keyword argument dont_print_output in the chat generation function for improved output management.
Version Updates

Incremented the application version from 0.0.124 to 0.0.125.

What's Changed

• added dont print by @shouryashashank in #38

Full Changelog: v0.0.124...v0.0.125

v0.0.124

• Added auto quantize for generation models reducing the max memory requirement by 4 folds

example

# load model with auto_quantize
model = predacons.load_model(model_name,auto_quantize="4bit")
tokenizer = predacons.load_tokenizer(model_name)

# generate response
sequence = "Explain the concept of acceleration in physics."
output,tokenizer =predacons.generate(model = model,
        sequence = sequence,
        max_length = 500,
        tokenizer = tokenizer,
        trust_remote_code = True)

# Decode and print the generated text
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)

What's Changed

• added auto quantize to all the model methods by @shouryashashank in #37

Full Changelog: v0.0.123...v0.0.124

v0.0.123

• added support for gguf model files

model = predacons.load_model(model_path=model_id, gguf_file=gguf_file)
tokenizer = predacons.load_tokenizer(model_id, gguf_file=gguf_file)
chat = [
    {"role": "system", "content": "you are a travel planner who plans trips for people. and list down the places to visit at that place"},
    {"role": "user", "content": "I want to plan a trip to new delhi. Can you help me with that?"},
]
predacons.chat_generate(model = model,
        sequence = chat,
        max_length = 200,
        tokenizer = tokenizer,
        trust_remote_code = True,
        do_sample=True,
        )

What's Changed

• added default chat template by @shouryashashank in #36

Full Changelog: v0.0.122...v0.0.123

v0.0.122

Changelog

Added

• Introduced a new chat generation feature.
  • Added predacons.chat_generate method to generate chat responses using trained models.
  • Example usage:

    chat = [
        {"role": "user", "content": "Hey, what is a car?"}
    ]
    chat_output = predacons.chat_generate(
        model=model,
        sequence=chat,
        max_length=50,
        tokenizer=tokenizers,
        trust_remote_code=True
    )

Updated

• README.md to include documentation and example usage for the new chat generation feature.

What's Changed

• Feature/chat generation by @shouryashashank in #35

Full Changelog: v0.0.121...v0.0.122

v0.0.121

Changelog

Added

• Introduced a new chat generation feature.
  • Added predacons.chat_generate method to generate chat responses using trained models.
  • Example usage:

    chat = [
        {"role": "user", "content": "Hey, what is a car?"}
    ]
    chat_output = predacons.chat_generate(
        model=model,
        sequence=chat,
        max_length=50,
        tokenizer=tokenizers,
        trust_remote_code=True,
        apply_chat_template=True
    )

Updated

• README.md to include documentation and example usage for the new chat generation feature.

What's Changed

• added docs by @shouryashashank in #31
• added chat generation by @shouryashashank in #33
• Feature/chat generation by @shouryashashank in #34

Full Changelog: v0.0.120...v0.0.121