Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use existing analyzer module for language detection for highlighting #13522

Merged
merged 7 commits into from
Nov 13, 2020
Merged

Use existing analyzer module for language detection for highlighting #13522

merged 7 commits into from
Nov 13, 2020

Conversation

mrsdizzie
Copy link
Member

@mrsdizzie mrsdizzie commented Nov 11, 2020 •
edited
Loading

Thanks @lafriks for pointing out we can reuse existing code for more reliable language detection here.

silverwind
silverwind approved these changes Nov 11, 2020
View reviewed changes
Copy link
Member

@silverwind silverwind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess it will help a little even thought most of the detectable file types are binary.

@GiteaBot GiteaBot added the lgtm/need 1 This PR needs approval from one additional maintainer to be merged. label Nov 11, 2020
@zeripath zeripath added the topic/ui Change the appearance of the Gitea UI label Nov 12, 2020
@zeripath zeripath added this to the 1.14.0 milestone Nov 12, 2020
@zeripath
Copy link
Contributor

this doesn't seem particularly helpful as most of the mimetypes appear to be binary in the attached library.

@mrsdizzie
Copy link
Member Author

It's for many script languages that are common to not have extensions php/perl/ruby/node/lua etc...

Most people don't use extensions when writing programs in scripting languages. None of these are highlighted now

lafriks
lafriks previously requested changes Nov 12, 2020
View reviewed changes
modules/highlight/highlight.go Outdated Show resolved Hide resolved
@silverwind
Copy link
Member

silverwind commented Nov 12, 2020 •
edited
Loading

Here's what that library gives us. I see it's rather incomplete thought. For example I can't find any shells in it, bash, zsh etc which are prime canidates for extensionless files. Still I guess it's a tiny improvement over the previous state.

I guess shell support could be hacked in by checking the first line of the file against a regexp like ^#!.*\b(ba|c|da|k|fi|tc|z)?sh($|\s).

@mrsdizzie
Use existing analyzer module for language detction for highlighting
9a1cef6 
Thanks @lafriks for pointing out we can reuse existing code for more reliable language detection here.
@mrsdizzie mrsdizzie changed the title Detect mime-type for file highlighting Use existing analyzer module for language detction for highlighting Nov 12, 2020
@mrsdizzie mrsdizzie changed the title Use existing analyzer module for language detction for highlighting Use existing analyzer module for language detection for highlighting Nov 12, 2020
@mrsdizzie
Copy link
Member Author

Changed per @lafriks pointing out we already have an existing module to detect language

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Nov 13, 2020
silverwind
silverwind reviewed Nov 13, 2020
View reviewed changes
go.sum Outdated Show resolved Hide resolved
@codecov-io
Copy link

Codecov Report

Merging #13522 (078dacb) into master (0ae35c6) will increase coverage by 0.01%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #13522      +/-   ##
==========================================
+ Coverage   42.15%   42.17%   +0.01%     
==========================================
  Files         695      695              
  Lines       76395    76394       -1     
==========================================
+ Hits        32207    32219      +12     
+ Misses      38903    38889      -14     
- Partials     5285     5286       +1
Impacted Files Coverage Δ
modules/highlight/highlight.go 25.00% <0.00%> (+0.30%) ⬆️
models/gpg_key.go 53.33% <0.00%> (-0.58%) ⬇️
services/pull/pull.go 40.78% <0.00%> (-0.50%) ⬇️
routers/repo/view.go 38.11% <0.00%> (+0.64%) ⬆️
modules/log/file.go 75.20% <0.00%> (+1.60%) ⬆️
services/pull/patch.go 55.68% <0.00%> (+1.70%) ⬆️
models/unit.go 49.31% <0.00%> (+2.73%) ⬆️
services/pull/check.go 51.82% <0.00%> (+2.91%) ⬆️
services/pull/temp_repo.go 29.78% <0.00%> (+3.19%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ae35c6...078dacb. Read the comment docs.

@techknowlogick
Copy link
Member

🚀

@techknowlogick techknowlogick merged commit 6280af0 into go-gitea:master Nov 13, 2020
@techknowlogick
Copy link
Member

@mrsdizzie please send backport :)

mrsdizzie added a commit to mrsdizzie/gitea that referenced this pull request Nov 13, 2020
@mrsdizzie @zeripath
@lafriks @techknowlogick
Use existing analyzer module for language detection for highlighting (g…
5003eca 
…o-gitea#13522)

* Use existing analyzer module for language detction for highlighting

Thanks @lafriks for pointing out we can reuse existing code for more reliable language detection here.

* Update modules/highlight/highlight.go

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
@mrsdizzie mrsdizzie mentioned this pull request Nov 13, 2020
Merged
techknowlogick added a commit that referenced this pull request Nov 13, 2020
@mrsdizzie @zeripath
@lafriks @techknowlogick
Use existing analyzer module for language detection for highlighting (#…
b4d18da 
…13522) (#13551)

* Use existing analyzer module for language detction for highlighting

Thanks @lafriks for pointing out we can reuse existing code for more reliable language detection here.

* Update modules/highlight/highlight.go

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
@lafriks lafriks added the backport/done All backports for this PR have been created label Nov 13, 2020
@go-gitea go-gitea locked and limited conversation to collaborators Dec 14, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@silverwind silverwind silverwind approved these changes

@lafriks lafriks lafriks left review comments

@lunny lunny lunny approved these changes

Assignees
No one assigned
Labels
backport/done All backports for this PR have been created lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. topic/ui Change the appearance of the Gitea UI
Projects
None yet
Milestone
1.14.0
Development

Successfully merging this pull request may close these issues.

None yet
8 participants
@mrsdizzie @zeripath @silverwind @codecov-io @techknowlogick @lunny @lafriks @GiteaBot