Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggest to loosen the dependency on snownlp #15

Open
Agnes-U opened this issue Nov 24, 2022 · 0 comments
Open

Suggest to loosen the dependency on snownlp #15

Agnes-U opened this issue Nov 24, 2022 · 0 comments

Comments

@Agnes-U
Copy link

Agnes-U commented Nov 24, 2022

Hi, your project LagouJob requires "snownlp==0.12.3" in its dependency. After analyzing the source code, we found that some other versions of snownlp can also be suitable without affecting your project, i.e., snownlp 0.9.8, 0.9.8.2, 0.9.9, 0.9.10, 0.10.1, 0.11.1, 0.12.0, 0.12.1, 0.12.2. Therefore, we suggest to loosen the dependency on snownlp from "snownlp==0.12.3" to "snownlp>=0.9.8,<=0.12.3" to avoid any possible conflict for importing more packages or for downstream projects that may use LagouJob.

May I pull a request to loosen the dependency on snownlp?

By the way, could you please tell us whether such dependency analysis may be potentially helpful for maintaining dependencies easier during your development?



For your reference, here are details in our analysis.

Your project LagouJob(commit id: 2c95ac0) directly uses 1 APIs from package snownlp.

snownlp.__init__.SnowNLP.__init__

From which, 3 functions are then indirectly called, including 2 snownlp's internal APIs and 1 outsider APIs, as follows (neglecting some repeated function occurrences).

[/lucasxlu/LagouJob]
+--snownlp.__init__.SnowNLP.__init__

We scan snownlp's versions among [0.9.8, 0.9.8.2, 0.9.9, 0.9.10, 0.10.1, 0.11.1, 0.12.0, 0.12.1, 0.12.2] and 0.12.3, the changing functions (diffs being listed below) have none intersection with any function or API we mentioned above (either directly or indirectly called by this project).

diff: 0.12.3(original) 0.9.8
['snownlp.seg.y09_2047.CharacterBasedGenerativeModel.load', 'snownlp.tag.__init__.train', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.sentiment.__init__.Sentiment', 'snownlp.classification.bayes.Bayes.classify', 'snownlp.sentiment.__init__.train', 'snownlp.utils.trie.Trie', 'snownlp.__init__.SnowNLP.pinyin', 'snownlp.seg.seg.Seg.load', 'snownlp.__init__.SnowNLP.keywords', 'snownlp.utils.tnt.TnT', 'snownlp.tag.__init__.save', 'snownlp.seg.seg.Seg.__init__', 'snownlp.utils.tnt.TnT.train', 'snownlp.classification.bayes.Bayes.save', 'snownlp.summary.textrank.TextRank.solve', 'snownlp.sentiment.__init__.load', 'snownlp.sentiment.__init__.save', 'snownlp.summary.words_merge.SimpleMerge.merge', 'snownlp.utils.tnt.TnT.load', 'snownlp.utils.trie.Trie.__init__', 'snownlp.utils.tnt.TnT.save', 'snownlp.sentiment.__init__.Sentiment.save', 'snownlp.__init__.SnowNLP', 'snownlp.normal.pinyin.PinYin.__init__', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.normal.zh.transfer', 'snownlp.seg.__init__.train', 'snownlp.seg.__init__.save', 'snownlp.summary.words_merge.SimpleMerge', 'snownlp.summary.textrank.TextRank', 'snownlp.utils.trie.Trie.translate', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.__init__.load', 'snownlp.seg.seg.Seg', 'snownlp.tag.__init__.load', 'snownlp.summary.textrank.KeywordTextRank.solve', 'snownlp.summary.words_merge.SimpleMerge.__init__', 'snownlp.utils.trie.Trie.find', 'snownlp.classification.bayes.Bayes.load', 'snownlp.seg.seg.Seg.save', 'snownlp.seg.seg.Seg.train', 'snownlp.classification.bayes.Bayes', 'snownlp.normal.pinyin.PinYin', 'snownlp.normal.__init__.get_pinyin', 'snownlp.sentiment.__init__.Sentiment.load', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.save', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.utils.trie.Trie.insert', 'snownlp.seg.__init__.seg', 'snownlp.normal.pinyin.PinYin.get']

diff: 0.12.3(original) 0.9.8.2
['snownlp.seg.y09_2047.CharacterBasedGenerativeModel.load', 'snownlp.tag.__init__.train', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.sentiment.__init__.Sentiment', 'snownlp.classification.bayes.Bayes.classify', 'snownlp.sentiment.__init__.train', 'snownlp.utils.trie.Trie', 'snownlp.__init__.SnowNLP.pinyin', 'snownlp.seg.seg.Seg.load', 'snownlp.utils.tnt.TnT', 'snownlp.tag.__init__.save', 'snownlp.seg.seg.Seg.__init__', 'snownlp.utils.tnt.TnT.train', 'snownlp.classification.bayes.Bayes.save', 'snownlp.summary.textrank.TextRank.solve', 'snownlp.sentiment.__init__.load', 'snownlp.sentiment.__init__.save', 'snownlp.utils.tnt.TnT.load', 'snownlp.utils.trie.Trie.__init__', 'snownlp.utils.tnt.TnT.save', 'snownlp.sentiment.__init__.Sentiment.save', 'snownlp.__init__.SnowNLP', 'snownlp.normal.pinyin.PinYin.__init__', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.normal.zh.transfer', 'snownlp.seg.__init__.train', 'snownlp.seg.__init__.save', 'snownlp.summary.textrank.TextRank', 'snownlp.utils.trie.Trie.translate', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.__init__.load', 'snownlp.seg.seg.Seg', 'snownlp.tag.__init__.load', 'snownlp.summary.textrank.KeywordTextRank.solve', 'snownlp.classification.bayes.Bayes.load', 'snownlp.utils.trie.Trie.find', 'snownlp.seg.seg.Seg.save', 'snownlp.seg.seg.Seg.train', 'snownlp.classification.bayes.Bayes', 'snownlp.normal.pinyin.PinYin', 'snownlp.normal.__init__.get_pinyin', 'snownlp.sentiment.__init__.Sentiment.load', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.save', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.utils.trie.Trie.insert', 'snownlp.seg.__init__.seg', 'snownlp.normal.pinyin.PinYin.get']

diff: 0.12.3(original) 0.9.9
['snownlp.seg.y09_2047.CharacterBasedGenerativeModel.load', 'snownlp.tag.__init__.train', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.sentiment.__init__.Sentiment', 'snownlp.classification.bayes.Bayes.classify', 'snownlp.sentiment.__init__.train', 'snownlp.utils.trie.Trie', 'snownlp.__init__.SnowNLP.pinyin', 'snownlp.seg.seg.Seg.load', 'snownlp.utils.tnt.TnT', 'snownlp.tag.__init__.save', 'snownlp.seg.seg.Seg.__init__', 'snownlp.utils.tnt.TnT.train', 'snownlp.classification.bayes.Bayes.save', 'snownlp.sentiment.__init__.load', 'snownlp.sentiment.__init__.save', 'snownlp.utils.tnt.TnT.load', 'snownlp.utils.trie.Trie.__init__', 'snownlp.utils.tnt.TnT.save', 'snownlp.sentiment.__init__.Sentiment.save', 'snownlp.__init__.SnowNLP', 'snownlp.normal.pinyin.PinYin.__init__', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.normal.zh.transfer', 'snownlp.seg.__init__.train', 'snownlp.seg.__init__.save', 'snownlp.utils.trie.Trie.translate', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.__init__.load', 'snownlp.seg.seg.Seg', 'snownlp.tag.__init__.load', 'snownlp.summary.textrank.KeywordTextRank.solve', 'snownlp.classification.bayes.Bayes.load', 'snownlp.utils.trie.Trie.find', 'snownlp.seg.seg.Seg.save', 'snownlp.seg.seg.Seg.train', 'snownlp.classification.bayes.Bayes', 'snownlp.normal.pinyin.PinYin', 'snownlp.normal.__init__.get_pinyin', 'snownlp.sentiment.__init__.Sentiment.load', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.save', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.utils.trie.Trie.insert', 'snownlp.seg.__init__.seg', 'snownlp.normal.pinyin.PinYin.get']

diff: 0.12.3(original) 0.9.10
['snownlp.seg.y09_2047.CharacterBasedGenerativeModel.load', 'snownlp.tag.__init__.train', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.sentiment.__init__.Sentiment', 'snownlp.classification.bayes.Bayes.classify', 'snownlp.sentiment.__init__.train', 'snownlp.utils.trie.Trie', 'snownlp.__init__.SnowNLP.pinyin', 'snownlp.seg.seg.Seg.load', 'snownlp.utils.tnt.TnT', 'snownlp.tag.__init__.save', 'snownlp.seg.seg.Seg.__init__', 'snownlp.utils.tnt.TnT.train', 'snownlp.classification.bayes.Bayes.save', 'snownlp.sentiment.__init__.load', 'snownlp.sentiment.__init__.save', 'snownlp.utils.tnt.TnT.load', 'snownlp.utils.trie.Trie.__init__', 'snownlp.utils.tnt.TnT.save', 'snownlp.sentiment.__init__.Sentiment.save', 'snownlp.__init__.SnowNLP', 'snownlp.normal.pinyin.PinYin.__init__', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.normal.zh.transfer', 'snownlp.seg.__init__.train', 'snownlp.seg.__init__.save', 'snownlp.utils.trie.Trie.translate', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.__init__.load', 'snownlp.seg.seg.Seg', 'snownlp.tag.__init__.load', 'snownlp.summary.textrank.KeywordTextRank.solve', 'snownlp.classification.bayes.Bayes.load', 'snownlp.utils.trie.Trie.find', 'snownlp.seg.seg.Seg.save', 'snownlp.seg.seg.Seg.train', 'snownlp.classification.bayes.Bayes', 'snownlp.normal.pinyin.PinYin', 'snownlp.normal.__init__.get_pinyin', 'snownlp.sentiment.__init__.Sentiment.load', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.save', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.utils.trie.Trie.insert', 'snownlp.seg.__init__.seg', 'snownlp.normal.pinyin.PinYin.get']

diff: 0.12.3(original) 0.10.1
['snownlp.tag.__init__.train', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.sentiment.__init__.Sentiment', 'snownlp.classification.bayes.Bayes.classify', 'snownlp.sentiment.__init__.train', 'snownlp.utils.trie.Trie', 'snownlp.__init__.SnowNLP.pinyin', 'snownlp.tag.__init__.save', 'snownlp.classification.bayes.Bayes.save', 'snownlp.sentiment.__init__.load', 'snownlp.sentiment.__init__.save', 'snownlp.utils.trie.Trie.__init__', 'snownlp.sentiment.__init__.Sentiment.save', 'snownlp.__init__.SnowNLP', 'snownlp.normal.pinyin.PinYin.__init__', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.normal.zh.transfer', 'snownlp.seg.__init__.train', 'snownlp.seg.__init__.save', 'snownlp.utils.trie.Trie.translate', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.__init__.load', 'snownlp.seg.seg.Seg', 'snownlp.tag.__init__.load', 'snownlp.summary.textrank.KeywordTextRank.solve', 'snownlp.classification.bayes.Bayes.load', 'snownlp.utils.trie.Trie.find', 'snownlp.classification.bayes.Bayes', 'snownlp.seg.seg.Seg.train', 'snownlp.normal.pinyin.PinYin', 'snownlp.normal.__init__.get_pinyin', 'snownlp.sentiment.__init__.Sentiment.load', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.utils.trie.Trie.insert', 'snownlp.seg.__init__.seg', 'snownlp.normal.pinyin.PinYin.get']

diff: 0.12.3(original) 0.11.1
['snownlp.classification.bayes.Bayes', 'snownlp.tag.__init__.train', 'snownlp.seg.__init__.train', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.classification.bayes.Bayes.classify', 'snownlp.sentiment.__init__.train', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.normal.__init__.get_pinyin', 'snownlp.seg.__init__.seg', 'snownlp.summary.textrank.KeywordTextRank.solve']

diff: 0.12.3(original) 0.12.0
['snownlp.classification.bayes.Bayes', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.classification.bayes.Bayes.classify', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.normal.__init__.get_pinyin', 'snownlp.seg.__init__.seg', 'snownlp.summary.textrank.KeywordTextRank.solve']

diff: 0.12.3(original) 0.12.1
['snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.normal.__init__.get_pinyin', 'snownlp.seg.__init__.seg', 'snownlp.summary.textrank.KeywordTextRank.solve']

diff: 0.12.3(original) 0.12.2
['snownlp.seg.y09_2047.CharacterBasedGenerativeModel.tag', 'snownlp.summary.textrank.KeywordTextRank', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel.train', 'snownlp.seg.y09_2047.CharacterBasedGenerativeModel', 'snownlp.summary.textrank.KeywordTextRank.solve']

As for other packages, the APIs of @outside_package_name are called by snownlp in the call graph and the dependencies on these packages also stay the same in our suggested versions, thus avoiding any outside conflict.

Therefore, we believe that it is quite safe to loose your dependency on snownlp from "snownlp==0.12.3" to "snownlp>=0.9.8,<=0.12.3". This will improve the applicability of LagouJob and reduce the possibility of any further dependency conflict with other projects/packages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant