Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify zero-shot topic modeling to fix issues with downstream tasks #2

Open
ianrandman opened this issue May 22, 2024 · 0 comments
Open

Simplify zero-shot topic modeling to fix issues with downstream tasks #2

ianrandman opened this issue May 22, 2024 · 0 comments
Assignees
@ianrandman

Comments

@ianrandman
Copy link
Collaborator

MaartenGr#1967
@ianrandman ianrandman self-assigned this May 22, 2024
ianrandman added a commit that referenced this issue May 28, 2024
@ianrandman
Zeroshot fixes (#2)
b95de5d 
- zero-shot topic modeling is now only the equivalent of a clustering step
  - removed implementation where this functionality is done through merging two models
  - all documents are used at once when calculating representations
  - probability comes from cosine similarity when zeroshot topics are used
- validate `nr_topics` with respect to how many zero-shot topics matched
- track `self._outliers` and `self.topic_labels_` using `@property`, as they are derivatives of other attributes
- validate existence of outliers before outlier reduction
ianrandman added a commit that referenced this issue May 28, 2024
@ianrandman
Fix outliers for update topics (#2)
96b1d6f
ianrandman added a commit that referenced this issue May 28, 2024
@ianrandman
Fix reduce topics with zeroshot and fix indexing of topic labels for…
69fa29c 
… zeroshot (#2)
ianrandman added a commit that referenced this issue Jun 14, 2024
@ianrandman
Fix uninitialized topic_id_to_zeroshot_topic_idx (#2)
cf3af3c
ianrandman added a commit that referenced this issue Jun 14, 2024
@ianrandman
Combine merged zero-shot topics with '_' (#2)
1cea27c
ianrandman added a commit that referenced this issue Jun 14, 2024
@ianrandman
Fix uninitialized topic_id_to_zeroshot_topic_idx (#2)
fc12e03
ianrandman added a commit that referenced this issue Jun 14, 2024
@ianrandman
Combine merged zero-shot topics with '_' (#2)
02134d6
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
Remove check for embedding model when check if zero-shot (#2)
97bbe64
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
Fix naming issue with topics_from and topic_to for merging and wh…
d2db056 
…en reducing topics, keep only one zero-shot label, if any (#1, #2)
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
When merging zero-shot, keep single zero-shot label if meets threshol…
3f5ef3f 
…d with new topic embedding (#2)
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
Add validation check for num topics when zero-shot (#2)
1701282
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
Fix typo (#2)
8481b97
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
Remove check for embedding model when check if zero-shot (#2)
cfd75a4
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
Fix naming issue with topics_from and topic_to for merging (#2)
ecd0224
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
When merging zero-shot, keep single zero-shot label if meets threshol…
19af331 
…d with new topic embedding (#2)
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
Fix validation check for num topics when zero-shot (#2)
9155a87
ianrandman added a commit that referenced this issue Jun 18, 2024
@ianrandman
Fix typo (#2)
2a7b194
ianrandman added a commit that referenced this issue Jun 21, 2024
@ianrandman
Add test for zero-shot (#2)
0f16c3d
ianrandman added a commit that referenced this issue Jun 21, 2024
@ianrandman
Add test for zero-shot (#2)
f8e4331
ianrandman added a commit that referenced this issue Jun 21, 2024
@ianrandman
Upgrade version and changelog (#1, #2)
839f7e9
ianrandman added a commit that referenced this issue Jun 23, 2024
@ianrandman
Format using ruff (#2)
7766277
ianrandman added a commit that referenced this issue Jun 23, 2024
@ianrandman
Make self._topic_id_to_zeroshot_topic_idx private, add comments/doc…
fbc574b 
…strings, lower threshold zeroshot test, fix outliers for probabilities during zeroshot (#2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ianrandman ianrandman

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

When branches are created from issues, their pull requests are automatically linked.

issue2-simplify-zero-shot semandex/BERTopic
1 participant
@ianrandman