Code view: support glob filters #532

apoorva17 · 2019-02-01T15:19:54Z

Fixes #504

With the addition of #170, users can filter the code view by filetype
by selecting tick boxes.

Let's support a more powerful search box where users can filter files
by glob e.g. **/test/*.java.

With the addition of reposense#170, users can filter the code view by filetype by selecting tick boxes. Let's support a more powerful search box where users can filter files by glob e.g. **/test/*.java.

apoorva17 · 2019-02-01T15:44:43Z

Since I could not find any frontend Javascript library for glob pattern matching, i have used a node-module called minimatch. Therefore, I have used browserify and vueify to use minimatch in the frontend.

Since the build javascript file produced is automated, it does not conform to ESLint style checks, therefore I have excluded it from the travis.yml file.

apoorva17 · 2019-02-02T05:32:51Z

@reposense/devs ready for review

apoorva17 · 2019-02-06T09:34:58Z

@ongspxm kindly review

yamidark

This feature doesn't work together with the current file type filter checkboxes:
Uncheck all file types
Set glob filter to any of the filetypes (e.g. **.java)
All .java files will appear, when the checkbox for .java is still unchecked.

eugenepeh

@apoorva17 I intend to have the file types selection to tap on the search bar functionality when its complete.
E.g. clicking on the file formats check box will update the search bar, letting the search bar to perform the filtration, so that we can remove the redundant code that does the same thing. Can you look into it?

apoorva17 · 2019-02-08T13:33:05Z

This feature doesn't work together with the current file type filter checkboxes:
Uncheck all file types
Set glob filter to any of the filetypes (e.g. **.java)
All .java files will appear, when the checkbox for .java is still unchecked.

@yamidark this is fixed now

apoorva17 · 2019-02-08T13:34:49Z

@apoorva17 I intend to have the file types selection to tap on the search bar functionality when its complete.
E.g. clicking on the file formats check box will update the search bar, letting the search bar to perform the filtration, so that we can remove the redundant code that does the same thing. Can you look into it?

This can be looked at in a separate PR. @damithc your opinion?

damithc · 2019-02-08T13:55:00Z

This can be looked at in a separate PR. @damithc your opinion?

I'm OK to do it as a separate PR.

@eugenepeh Actually, I'm not entirely sure about that feature. Auto-updating the search bar as the user ticks checkboxes is nice, but complications will arise when the user types in the search bar and tick checkboxes in tandem. If we go in that direction, we may have to disable checkboxes the moment the user starts typing in the search bar.
Alternatively, instead of auto-updating the search bar, we can simply consider the combination of the two implicitly. i.e., if the documentation group is ticked and the user types *foo*.adoc in the search bar, we filter out files that meet both criteria.

eugenepeh · 2019-02-08T16:20:26Z

@eugenepeh Actually, I'm not entirely sure about that feature. Auto-updating the search bar as the user ticks checkboxes is nice, but complications will arise when the user types in the search bar and tick checkboxes in tandem. If we go in that direction, we may have to disable checkboxes the moment the user starts typing in the search bar.

The reason I propose this is because it is not going to be easy to get the checkbox and search bar work well with each other. The complication you mentioned, still exist in current implementation. E.g. toggling the checkbox ignores the search bar values and shows results corresponding only to the checkboxes.

I propose that we replace all the checkboxes with buttons that append or replace the search bar value so that both work in coherent. E.g. [clear], [gradle], [java] etc and the lines summary can be moved to the search results which updates along with the search condition. E.g. All: 2741 (443) gradle: 14 (2) jade: 75 (9) java: 1438 (258) js: 477 (72) md: 129 (34) scss: 571 (60) yml: 37 (8).

Alternatively, instead of auto-updating the search bar, we can simply consider the combination of the two implicitly. i.e., if the documentation group is ticked and the user types *foo*.adoc in the search bar, we filter out files that meet both criteria.

The advantage of my proposition over this alternative is that:

Easier to manage the url bookmarking. Only need to parse a single field.
Eliminate race condition and concurrent actions. If we have both checkbox and search bar which operates on a single list of files concurrently, it can result in crash.
Currently, there is no atomic way to show a specific file type. E.g. java. User would have to hide all files first, follow by selecting the java files which result in unnecessary waiting time to hide all files.
With this idea, we can just click on java button and the search bar will be updated accordingly to show only java files in an atomic action. To hide all java files, we can probably implement middle mouse click to show the inverse.
Lower complexity of implementation, and reduce redundant codes.
LoC update with changes.
In tandem with my suggestion in Custom grouping of files #547

@damithc , what do you think?

damithc · 2019-02-09T05:53:20Z

@damithc , what do you think?

Good points.
I kind of dislike the idea of auto-updating a field that is also meant to be editable by the user. Users are unpredictable, for example, they can edit the auto-generated text in arbitrary ways.
In here, I'm proposing to keep tick boxes and filter box independent. i.e., only one will be used at any time. Would that reduce the complications?

eugenepeh · 2019-02-09T08:58:51Z

I kind of dislike the idea of auto-updating a field that is also meant to be editable by the user. Users are unpredictable, for example, they can edit the auto-generated text in arbitrary ways.

I see. Though, another advantage of auto-generated text is that it helps user understand how the glob works .

In here, I'm proposing to keep tick boxes and filter box independent. i.e., only one will be used at any time. Would that reduce the complications?

It does helps to reduce the complexity, though not any helpful to the bookmarking because it is still 2 separate field. However, if users are more comfortable with the ui in #547 (comment), I am all in for it as my main concerns are mainly the complexity and potential bugs getting the 2 working together.

damithc · 2019-02-09T09:33:43Z

I see. Though, another advantage of auto-generated text is that it helps user understand how the glob works .

We can show the auto-generated glob under the tick boxes as a read-only label so that the user can copy it over to the filter box, if he wants.

It does helps to reduce the complexity, though not any helpful to the bookmarking because it is still 2 separate field.

That's true. In this case, we'll have to encode filter text or tickbox configuration, whichever the chosen option. On a general note, it maybe good to define a data structure that can represent all report parameters, and functions to encode it into a URL (and decode it from the URL). perhaps something based on JSON?

However, if users are more comfortable with the ui in #547 (comment), I am all in for it as my main concerns are mainly the complexity and potential bugs getting the 2 working together.

Yes, getting the two to work together will be hard. Let's keep them separate.

ongspxm

minor nit

package.json

apoorva17 · 2019-02-11T14:32:36Z

@damithc i've updated the implementation as per #547 (comment)

yamidark

Do update the report image in the UserGuide as well, only DeveloperGuide updated

apoorva17 · 2019-03-02T13:31:30Z

Do update the report image in the UserGuide as well, only DeveloperGuide updated

Ok, I've updated it.

ongspxm

minor nit, the rest looks ready to go

ongspxm · 2019-03-02T14:31:08Z

frontend/src/static/js/v_authorship.js

+    },
+
+    getFilteredFiles() {
+    },


this is here because?

I tried to do v-on:submit.prevent for glob filter to prevent the page from refreshing without specifying any method but it wasn't working so i had to add this method in.

sflr; pls add a comment here if needed. but try this first, it should work, using vanilla form functions:

form(onsubmit="return false")

damithc · 2019-03-03T04:37:55Z

On a separate note, is there a glob syntax to specify multiple conditions? e.g., having Test or Manager in the file name.
@apoorva17 a follow-up PR you can do is to support more complex conditions in the glob filter, such as OR AND

apoorva17 · 2019-03-03T06:06:00Z

On a separate note, is there a glob syntax to specify multiple conditions? e.g., having Test or Manager in the file name.

Not currently, I'm using minimatch library which does not seem to support such a glob syntax.

yong24s · 2019-03-03T08:52:41Z

On a separate note, is there a glob syntax to specify multiple conditions? e.g., having Test or Manager in the file name.

Not currently, I'm using minimatch library which does not seem to support such a glob syntax.

Can try multimatch. It extends minimatch library to support multiple conditions.

apoorva17 · 2019-03-03T14:10:00Z

Can try multimatch. It extends minimatch library to support multiple conditions.

Sure, I can look into this in a follow-up PR.

eugenepeh · 2019-03-04T14:27:58Z

@ongspxm just a reminder, you have an unresolved conversation #532 (review)

ongspxm

minor nits

ongspxm · 2019-03-05T14:17:41Z

frontend/src/static/js/v_authorship.js

+    },
+
+    getFilteredFiles() {
+    },


sflr; pls add a comment here if needed. but try this first, it should work, using vanilla form functions:

form(onsubmit="return false")

ongspxm · 2019-03-05T14:19:27Z

@apoorva17 sry, i pressed wrong, dun know how to unapprove, but just that change will do

apoorva17 · 2019-03-06T06:09:32Z

@ongspxm I've made the requested change.

apoorva17 · 2019-03-06T07:05:41Z

@yong24s getting same frontend error like here

yamidark · 2019-03-06T07:56:47Z

@apoorva17 Restarted it a few times and it passes now, seems like cypress will fail the tests once in awhile (happens to be locally as well). Will raise an issue for this.

eugenepeh · 2019-03-06T15:35:38Z

Nits to proposed message:

The addition of #170 provides users with the convenience of filtering
the files in code view by formats using tick boxes. However, this does
not solve the problem of finding specific file(s) by name, folder or
path.

As a solution, let's add a search box to provide users with an
additional option of filtering files by glob patterns. E.g.
**/test/*.java.

After #532 , the name of the checkbox is changed from `mui-checkbox` to `mui-checkbox--filetype`, causing an alignment problem with the checkboxes. Since this rename is meaningful, let's add another css part to ensure that the checkboxes align. Meanwhile, some of the indentation in #532 does not follow our standard, let's update them as well.

apoorva17 force-pushed the 504-support-glob-filters branch 2 times, most recently from 4f780f7 to 99ee679 Compare February 1, 2019 15:36

Support glob filters in code view

07ea762

With the addition of reposense#170, users can filter the code view by filetype by selecting tick boxes. Let's support a more powerful search box where users can filter files by glob e.g. **/test/*.java.

apoorva17 force-pushed the 504-support-glob-filters branch from 99ee679 to 07ea762 Compare February 1, 2019 15:44

apoorva17 added the s.ToReview label Feb 2, 2019

eugenepeh requested a review from ongspxm February 2, 2019 08:20

yamidark suggested changes Feb 7, 2019

View reviewed changes

eugenepeh reviewed Feb 7, 2019

View reviewed changes

Sync with checkboxes

dd62c97

Merge branch 'master' into 504-support-glob-filters

c58e15e

apoorva17 added 3 commits February 10, 2019 17:54

Make search bar and tick boxes independent

e420552

Merge branch 'master' into 504-support-glob-filters

bfaca8a

Update v_authorship.js

d7fc7d7

ongspxm approved these changes Feb 11, 2019

View reviewed changes

package.json Outdated Show resolved Hide resolved

Codacy changes

ddb3c51

apoorva17 force-pushed the 504-support-glob-filters branch from 2943096 to ddb3c51 Compare February 11, 2019 08:50

apoorva17 added 2 commits February 11, 2019 22:30

Add build to spuild

e7919fb

Merge branch 'master' into 504-support-glob-filters

e820d2b

Fix checkboxes

1873646

eugenepeh requested review from ongspxm, yamidark and yong24s March 2, 2019 07:47

yong24s approved these changes Mar 2, 2019

View reviewed changes

yamidark reviewed Mar 2, 2019

View reviewed changes

Update diagram in User Guide

64bf6ad

ongspxm reviewed Mar 2, 2019

View reviewed changes

yamidark approved these changes Mar 2, 2019

View reviewed changes

eugenepeh requested a review from damithc March 3, 2019 04:21

damithc approved these changes Mar 3, 2019

View reviewed changes

eugenepeh mentioned this pull request Mar 3, 2019

CodeView#filter: add interactive selection switch #581

Closed

yamidark added s.ToMerge and removed s.ToReview labels Mar 3, 2019

eugenepeh requested a review from ongspxm March 5, 2019 13:44

ongspxm approved these changes Mar 5, 2019

View reviewed changes

Remove getFilteredFiles()

96e4532

eugenepeh merged commit 5da9d45 into reposense:master Mar 6, 2019

fzdy1914 mentioned this pull request Mar 6, 2019

[#588] CodeView: align the checkboxes #589

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Code view: support glob filters #532

Code view: support glob filters #532

apoorva17 commented Feb 1, 2019

apoorva17 commented Feb 1, 2019

apoorva17 commented Feb 2, 2019

apoorva17 commented Feb 6, 2019

yamidark left a comment

eugenepeh left a comment

apoorva17 commented Feb 8, 2019

apoorva17 commented Feb 8, 2019

damithc commented Feb 8, 2019

eugenepeh commented Feb 8, 2019 •

edited

damithc commented Feb 9, 2019

eugenepeh commented Feb 9, 2019

damithc commented Feb 9, 2019

ongspxm left a comment

apoorva17 commented Feb 11, 2019

yamidark left a comment

apoorva17 commented Mar 2, 2019 •

edited

ongspxm left a comment

ongspxm Mar 2, 2019

apoorva17 Mar 3, 2019 •

edited

ongspxm Mar 5, 2019

damithc commented Mar 3, 2019

apoorva17 commented Mar 3, 2019 •

edited

yong24s commented Mar 3, 2019

apoorva17 commented Mar 3, 2019

eugenepeh commented Mar 4, 2019

ongspxm left a comment

ongspxm Mar 5, 2019

ongspxm commented Mar 5, 2019

apoorva17 commented Mar 6, 2019

apoorva17 commented Mar 6, 2019

yamidark commented Mar 6, 2019

eugenepeh commented Mar 6, 2019

Code view: support glob filters #532

Code view: support glob filters #532

Conversation

apoorva17 commented Feb 1, 2019

apoorva17 commented Feb 1, 2019

apoorva17 commented Feb 2, 2019

apoorva17 commented Feb 6, 2019

yamidark left a comment

Choose a reason for hiding this comment

eugenepeh left a comment

Choose a reason for hiding this comment

apoorva17 commented Feb 8, 2019

apoorva17 commented Feb 8, 2019

damithc commented Feb 8, 2019

eugenepeh commented Feb 8, 2019 • edited

damithc commented Feb 9, 2019

eugenepeh commented Feb 9, 2019

damithc commented Feb 9, 2019

ongspxm left a comment

Choose a reason for hiding this comment

apoorva17 commented Feb 11, 2019

yamidark left a comment

Choose a reason for hiding this comment

apoorva17 commented Mar 2, 2019 • edited

ongspxm left a comment

Choose a reason for hiding this comment

ongspxm Mar 2, 2019

Choose a reason for hiding this comment

apoorva17 Mar 3, 2019 • edited

Choose a reason for hiding this comment

ongspxm Mar 5, 2019

Choose a reason for hiding this comment

damithc commented Mar 3, 2019

apoorva17 commented Mar 3, 2019 • edited

yong24s commented Mar 3, 2019

apoorva17 commented Mar 3, 2019

eugenepeh commented Mar 4, 2019

ongspxm left a comment

Choose a reason for hiding this comment

ongspxm Mar 5, 2019

Choose a reason for hiding this comment

ongspxm commented Mar 5, 2019

apoorva17 commented Mar 6, 2019

apoorva17 commented Mar 6, 2019

yamidark commented Mar 6, 2019

eugenepeh commented Mar 6, 2019

eugenepeh commented Feb 8, 2019 •

edited

apoorva17 commented Mar 2, 2019 •

edited

apoorva17 Mar 3, 2019 •

edited

apoorva17 commented Mar 3, 2019 •

edited