Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor for Ruby 3.0 compatibility #68

Merged
merged 2 commits into from
May 2, 2021
Merged

Conversation

alextsui05
Copy link
Contributor

Closes #67

Internally, the segmenter uses PragmaticSegmenter::Text, a subclass of Ruby String, but as of Ruby 3, calling String methods no longer returns the subclass. This PR refactors code so that it doesn't depend on the subclass.

Changes

  • Move helper method Text#apply to Rule#apply, which takes a string and applies a number of rules to it
  • Replace Text with standard Ruby String
  • Relax constraint on bundler version

@sevilaybayatli
Copy link

Hello,

I have been using pragmatic segmenter by following the steps below:
sudo apt-get install ruby-full
gem install pragmatic_segmenter

And after install the pragmatic_segmenter I got this:
Successfully installed pragmatic_segmenter-0.3.22
Parsing documentation for pragmatic_segmenter-0.3.22
Done installing documentation for pragmatic_segmenter after 0 seconds
1 gem installed

And using it to sengemet the sentences in a whole file by applying the code block below:
require 'pragmatic_segmenter'
if (ARGV.length < 3)
puts "\nUsage : ruby2.5 sentenceTokenizer.rb 639-1ISOlangCode textFilePath sentencesFilePath"
exit
end
File.open(ARGV[1]).each do |line1|
line1.delete! ('()[]{}<>|$/'"')
ps = PragmaticSegmenter::Segmenter.new(text: line1, language: ARGV[0], doc_type: 'txt')
sentences = ps.segment
File.open(ARGV[2], "a") do |line2|
sentences.each { |sentence| line2.puts sentence }
end
end

But the problem now I got the error below:
Traceback (most recent call last):
2: from sentenceTokenizer.rb:1:in

' 1: from /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in require'
/usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- pragmatic_segmenter (LoadError)

Could you point me please? the same code and same steps have been used before and worked? I am curious, why I got this error now. Help

@alextsui05
Copy link
Contributor Author

Hi @sevilaybayatli, the above comment is off-topic for this PR, but I saw you opened an issue so I tried to respond over there, so let's take the discussion over there instead! 🙌 #69 (comment)

@ncs1
Copy link

ncs1 commented May 2, 2021

Hi,
Thank you for this PR.
Using it on Ruby 3, please review @diasks2

@diasks2 diasks2 merged commit 42d5f04 into diasks2:master May 2, 2021
@diasks2
Copy link
Owner

diasks2 commented May 2, 2021

Thank you @alextsui05! Sorry for the delay in getting this merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot segment text on Ruby 3
4 participants