v0.5.0

LLMTextAnalysis v0.5.0

Diff since v0.4.0

Added

• Added a classification function train_classifier to train a model to classify documents into a set of predefined labels (as opposed to the more open-ended topic modeling in build_clusters!). You can either provide a small set of labeled documents to train the model (that are in the index), or just specify the num_samples and the LLM model will generate its own training data based on the labels and labels_description provided.
• Added a new template TextWriterFromLabel to generate synthetic documents for any given label (=topic).
• Added methods for build_clusters! to add custom topic levels, eg, from a TrainedClassifier (build_clusters!(index,cls; topic_level="MyTopics")) or directly via providing a vector of document assignments (build_clusters!(index, assignments; topic_level="MyTopics")). The convention is to use topic_level::Integer for auto-generated topics, and topic_level::String for custom topics.

Updated

• Updated to use PromptingTools 0.15.

Fixed

• Fixed a bug where keywords were not properly filtered before being provided to the auto-labeling function.

Commits

Merged pull requests:

• Add Fixed Classifier (#14) (@svilupp)
• Docstring fix for train_classifier (#15) (@svilupp)

v0.4.0

LLMTextAnalysis v0.4.0

Diff since v0.3.2

Added

• Added a new example on topic label customization (examples/3_customize_topic_labels.jl) and the corresponding sections in the FAQ.
• Added a few string cleanup tricks in build_topic function to strip unnecessary repetition of the prompt template in the generated labels.
• Added new templates TopicLabelerWithInstructions and TopicSummarizerWithInstructions that include the placeholder instructions to allow users to easily customize the labels and summaries, respectively.

Fixed

• Fixed small typos in templates TopicLabelerBasic and TopicSummarizerBasic.

Updated

• Updated logic in the plot to ensure topic labels are generated only when necessary. Use build_clusters! to force the generation of topic labels, or plot to generate them only if necessary.
• Increased compatibility for PromptingTools to 0.12.

Commits

Merged pull requests:

• Customize Topic Labels (#13) (@svilupp)

v0.3.2

LLMTextAnalysis v0.3.2

Diff since v0.3.1

Merged pull requests:

• Minor readme adjustment (#8) (@cpfiffer)
• Bump codecov/codecov-action from 3 to 4 (#9) (@dependabot[bot])
• Fix UndefVarError(:scores) #10 (#11) (@svilupp)

Closed issues:

• ERROR: UndefVarError: scores1 not defined (#10)

v0.3.1

LLMTextAnalysis v0.3.1

Diff since v0.3.0

Fixed

• wrap_string utility would error with SubString chunks. Now it works with any AbstractString type.

Commits

Merged pull requests:

• Fix wrap_string (#7) (@svilupp)

v0.3.0

LLMTextAnalysis v0.3.0

Diff since v0.2.1

Added

• Changed compat for PromptingTools to 0.10.0 (with new default models! Ie, default embeddings will not match the previous version)

Commits

Merged pull requests:

• Increase Compat for PromptingTools (#6) (@svilupp)

v0.2.1

LLMTextAnalysis v0.2.1

Diff since v0.2.0

Fixed

• Updated documentation to show Example 2 for concept/spectrum training.

Commits

Merged pull requests:

• Fix docs (#4) (@svilupp)
• Fix docs (#5) (@svilupp)

v0.2.0

LLMTextAnalysis v0.2.0

Diff since v0.1.0

Added

• Added train_concept. Introduces the ability to train a model focusing on a single, specific concept within a collection of documents. This function helps in identifying and scoring the presence or intensity of the selected concept across the document set. Ideal for thematic studies, sentiment analysis, or tracking specific ideas in the text.
• Added train_spectrum. Adds functionality to analyze documents across a spectrum defined by two contrasting concepts. This feature allows for a comparative analysis, providing insights into how documents align or contrast with two polar themes or sentiments.
• Spectrum and concept can be plotted using plot function.
• Improved plotting support:
  • Added package extension for PlotlyJS for plot function.
  • Enabled plot function to accept an arbitrary hoverdata table with information to be added to the tooltip for each document (expects Tables.jl-compatible data).

Commits

Merged pull requests:

• Add concept labeling (#2) (@svilupp)
• Update plotting for concept/spectrum (#3) (@svilupp)

v0.1.0

LLMTextAnalysis v0.1.0