Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ez time filter component and story #320

Merged
merged 36 commits into from
Sep 11, 2023
Merged

Conversation

moontrip
Copy link
Contributor

@moontrip moontrip commented Jun 25, 2023

Resolves #245.

It is now wired up to the user analysis saved state and the slider is working smoothly except for a known Firefox bug (see #480). The ez time filter now applies to markers and floaters. But not downloads. I added an on/off toggle, feedback welcome on this.

Remaining subtasks - in no particular order, and many could be done in parallel

  • play with it and figure out any major UX issues
  • grey out most of component when timeSliderActive is false (e.g. toggled off)
  • take it out of the draggable and attach underneath the header and
    - create a super slim minimised mode
    - expand/collapse button (or click on the minimised mode to expand)
    - user interactions should only occur in the maximised mode, but the following basic information should be communicated in minimised mode:
    1. the variable name (plain text, ellipses as required)
    2. the distribution (no x-axis tick labels) and blue selected region if active (grey if inactive?)
  • test with proper date variables and remove the number/integer/ordinal variable support (so it will only work with actual date variables, not year numbers, ordinals or the like)
  • confirm that "show counts for filtered/visible" in the donut/barplot marker categorical variable config tables should include the ez time filter
  • add step left/right buttons (or leave for a future ticket)
  • add date inputs to the date range display at top of widget (or make new ticket)
  • sort out z-axis issues w.r.t. mouse-over marker enlargements (see screenshot)
    image
DK's original notes mainly about the slider component (click to expand)

It is primarily based on Visx Brush component, thus I added @visx/brush at web-components's package.json/dependencies. Also made a story file to test.

Some notes are: a) the plot area is made of a kind of density plot for now. It is because it looks good and the mockup one will probably need much more work; b) an example data was taken from Lineplot data (GEMS1 Case Control; x: Enrollment date; y: Weight) for demo purpose; c) the selected range (or filtered area) is expressed as diagonal lines so that background plot (density plot) can be seen; d) initial position of the filter/selection is set to be whole range; e) the brush handle, which is for mouse drag to slide, is currently shown as small rectangles. It is a typical shape that can be found in other tools so I leave it as it: it seems like one can customize it though.

Here is a demo video:

EzTimeFilter01.mp4

@moontrip
Copy link
Contributor Author

moontrip commented Jul 6, 2023

Update 1: made a time filter at SAM 8440f05 . A demo video is attached in the following. Though, there are several things to do for working, including unknowns.

a) Currently, a temporary test data (a lineplot data) is used to show a kind of density plot to represent the existence of data in the date range. Not quite sure what and how data should be used for this purpose.
b) Although InputVariables component is attached, it is not functional yet. Similar to a), not quite sure what should be used for that variable tree.
c) since the range of the time filter is changed by dragging each end, I think that a Submit button needs to be made to add the selected range to the filter to avoid consecutive data request (by changing filter value).
d) the icon and the position of the close/open button are temporary, which can be adjusted later.

SAM.time.filter01.mp4

@bobular and @d-callan, can you please let me know how to handle above a) and b) ?

@d-callan
Copy link
Member

d-callan commented Jul 11, 2023

a) there is this backend ticket, which more or less aims to establish a new endpoint dedicated to providing the data needed for this. the ticket describes providing a response indicating only the presence or absence of data, rather than how much data
b) any variable annotated as temporal should be available in that variable picker.. i suppose we could treat the new endpoint (from the above link) like a more typical viz and add a constraints pattern to it
c) i dont love a submit button, can we debounce or something? (not sure i fully understand the implications of what im suggesting). also, ideally the text indicating the selected range above the slider could also serve as an input.

how much more work do we think it is to make the bar from the mockups?

@moontrip
Copy link
Contributor Author

a) there is this backend ticket, which more or less aims to establish a new endpoint dedicated to providing the data needed for this. the ticket describes providing a response indicating only the presence or absence of data, rather than how much data b) any variable annotated as temporal should be available in that variable picker.. i suppose we could treat the new endpoint (from the above link) like a more typical viz and add a constraints pattern to it c) i dont love a submit button, can we debounce or something? (not sure i fully understand the implications of what im suggesting). also, ideally the text indicating the selected range above the slider could also serve as an input.

how much more work do we think it is to make the bar from the mockups?

Thank you for your detailed information, @d-callan 👍 Honestly, it is not clear how to wire data-related one to the time filter component until I see the response from the backend, so I will wait for the backend to be ready.

As for the submit button, perhaps a kind of debounce may work without submit button as you mentioned. This may not be that clear to a user whether data request is made or not, so perhaps a loading icon or something like that needs to be displayed not to change the range. Will think about it. In addition, you also mentioned about selected range as an input, which I guess you meant a kind of input form to change values. As far as I know, the sliding bar, which is made of Visx's brush component, is uncontrolled component. Thus it is very hard to manage it from parent component like map. Not quite sure if the following would work without testing, but this may be feasible by moving the range part (and reset icon) into the time filter component (EzTimeFilter.tsx).

For your last comment, "make the bar", I guess you mean thin and thick horizontal bar style at mockup instead of current density plot. I thought that relevant data/response from the backend would include the format of x=time & y=value format, so in this case, it was arguably the easiest way to use Visx's xychart package and AreaClosed (like density plot) component so that x-axis tick & its label and y-axis values are shown with minimum efforts. From my understanding, there is no specific Visx component to display thin and thick horizontal bar style. This means that if we want to have such a bar style, then: 1) they should be made via SVGs manually; 2) this will also mean that x-axis should also be made via SVGs manually. That is, this would involve significant works to be implemented: basically this means that we need to make a specific plot component via d3 only.

@asizemore
Copy link
Member

On the subject of the bars, what if we just used histo data with all bars having height 1 if there is data and 0 if no data?
So if we get the binned data, for each bin we translate the counts to 0 or 1. Count becomes 1 if the original counts were 1 or higher, and stays at 0 if the original counts for that bin was 0. Then, we do a BarSeries instead of the AreaClosed?

That could mess with the axis a bit, but maybe worth a shot @moontrip ?

The other idea i came up with was doing basically ticks at every point where we have data. So we'd use LineSeries and use the x value of the data for the x1, x2 of each line, and then like 5 and 0 for the y1 and y2.

Anyways, curious if either of the above might be an easier way to get closer to the mockups!

@d-callan
Copy link
Member

thanks @asizemore. i do have a fairly strong preference to show presence/ absence of data as opposed to 'how much' data. if we can find a way to achieve that without a huge cost, that would be ideal.

@moontrip
Copy link
Contributor Author

Thank you for sharing your thoughts @asizemore 👍 Yes your suggestions can be relatively easier approach than mockup one. Looks like bar approach is more similar to the mockup. Will test it.

@moontrip
Copy link
Contributor Author

moontrip commented Jul 13, 2023

Hi @d-callan & @asizemore

I have tested scaled values that represent with (1) or without data (0). In Visx, in general scaleBand is used for Bar plot's x-axis scaling instead of scaleTime (for, e.g., lineplot/time series, etc.). The scaleBand automatically compute bandWidth based on data and plot size, however, it cannot be used for our purpose directly because slider filter does not recognize the x-axis values which is time-based, not bin/categorical. But I found that scaleTime can also be used for Bar plot by computing bandWidth manually, which resolves the aforementioned issue. The first screenshot is a demo version.

I have also tested lineplot (xychart) case with 0 and 1 values (2nd screenshot), but I guess you prefer the bar instead of having slopes between data and no data in the lineplot.

As a side note, I tested debouncing functionality to possibly be used for submitting selected range, and I could implement it with slider action.

ezTimeFilter-bar

ezTimeFilter-density

@d-callan
Copy link
Member

Thanks 👍 think you can make it shorter now? Possibly add a custom line annotation across the center of those bars?

@moontrip
Copy link
Contributor Author

Thank you for your comments @d-callan 👍 Can you please confirm my guess in the following? a) so making it shorter means the height of bars (i.e., plot height)? like 2/3 of current height? b) simply adding a horizontal center line in the bar plot?

@bobular
Copy link
Member

bobular commented Jul 17, 2023

I think the line plot (second screenshot above) looks great.

@d-callan
Copy link
Member

my issue w the line is primarily that it leaves it a bit ambiguous whether there is or isnt data for July 2009

@moontrip
Copy link
Contributor Author

moontrip commented Jul 17, 2023

@d-callan, @asizemore, (and @bobular)

I have addressed feedbacks, center line in the horizontal direction and shorter height. In addition, I figured out how to make the time filter plot be closer to the original mockup, #245. This is applied to both components' storybook and SAM, which a screenshot taken from SAM is attached FYI.

ez timefilter 04

@bobular
Copy link
Member

bobular commented Jul 18, 2023

Looks very nice DK!

@moontrip
Copy link
Contributor Author

Looks very nice DK!

Thanks @bobular 😃

@d-callan
Copy link
Member

beautiful! i wonder now if its possible to have the label above serve also as an input?

@moontrip
Copy link
Contributor Author

beautiful! i wonder now if its possible to have the label above serve also as an input?

Glad you like it @d-callan :) As for the label/input style, although I will need to investigate more, at this stage there are a couple of issues that need to be addressed. The biggest issue is that the slider, which is made of Visx brush, is uncontrolled component within my knowledge. Thus, there seems to be no explicit way to change the slider position by parent component (e.g., via props): in other words, even if an input range is made and the range values are changed, the slider position won't be changed, which is not good for UI/UX perspective.

Copy link
Member

@bobular bobular left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a suggestion how to make the Brush behave like it is controlled. It seems to work for me (I can control it from dev tools), but would like @dmfalke's view on this too!

First, set the initial position from EzTimeFilter's selectedRange prop

  const initialBrushPosition = useMemo(
    () => ({
      start: { x: xBrushScale(new Date(selectedRange.start)) },
      end: { x: xBrushScale(new Date(selectedRange.end)) },
    }),
    [selectedRange, xBrushScale]
  );

Then add a key to the Brush component

          <Brush
	    key={selectedRange.start + "/" + selectedRange.end}

If this (or another method) works, then it should be relatively straightforward to add the "window forward" and "window backwards" buttons (as well as the date-picker inputs from the "title").

paddingRight: '1.5em',
}}
>
<a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for putting the reset button outside the component?

To me it just adds complication though it has helped me understand forwardRef 🙂

Also, can we use a real button, not an <a> tag? In the storybook at least, clicking on the reset link makes everything go into fullscreen storybook mode.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, my (initial) intention to place the reset at parent level was for positioning. But I think that it may be better to place the reset and selected range label into the ez time filter component, and then adjust InputVariable component, which should be at parent component, via CSS positioning.

Yes, I also noticed that tag causes it at storybook. Will handle it, either change to button or something else.

@bobular
Copy link
Member

bobular commented Jul 19, 2023

ChatGPT3.5 seemed to reluctantly agree that the key approach was OK.
Here are the docs for Brush - as DK says - it seems totally uncontrollable.
https://airbnb.io/visx/docs/brush

@moontrip
Copy link
Contributor Author

@bobular Thank you for your examination 👍 I saw something similar approach somewhere: IIRC, we also used similar one at our Plotly (for thumbnail I suppose). Will check.

@bobular
Copy link
Member

bobular commented Aug 7, 2023

Hi @moontrip - did you run into problems making (faking) the controlled behaviour of the <Brush>?

Or have you not been able to get back to this due to time constraints?

@bobular
Copy link
Member

bobular commented Aug 7, 2023

Maybe @dmfalke could comment on my suggested use of the key prop?

@moontrip
Copy link
Contributor Author

moontrip commented Aug 7, 2023

Hi @moontrip - did you run into problems making (faking) the controlled behaviour of the <Brush>?

Or have you not been able to get back to this due to time constraints?

Hi @bobular 👋 I think that it may work but as you said, I didn't have time to test it due to time constraints. Actually I did use key approach for other purpose, attach/detach functionality of the filter to the top area and it seems to work fine: didn't commit the latest works yet as it needs some refactoring as well as I am dealing with other ticket (truncation one you commented alot 😄).

@dmfalke
Copy link
Member

dmfalke commented Aug 7, 2023

Maybe @dmfalke could comment on my suggested use of the key prop?

This will work, but it could be expensive. Changing the value of the key prop causes the component to unmount and mount again. I wonder if we can base the key value off of some state that is incremented only when the input changes? Something like this, roughly speaking:

function EZFilter(props: Props) {
  // Track how many times the input has changed by user 
  const [inputChangeCounter, setInputChangeCounter] = useState(0);
  // Generate brush key based on inputChangeCounter
  const brushKey = `iteration-${inputChangeCounter}`;

  // ...

  return (
    <>
      <DateInputs onChange={range => {
        // Update range value in analysis state
        updateRange(range);
        // Increment counter
        setInputChangeCounter(c => c + 1);
      }}/>
      <Brush key={brushKey}/>
    </>
}

@moontrip
Copy link
Contributor Author

moontrip commented Aug 8, 2023

Made lots of changes for ez time slider concerning UI and UX feedbacks. Summary is:

changes

  • changed the box style of the selected range: no hatched lines, use the same color with HistogramFilter, etc.
  • reduced some margins to be smaller overall
  • used dash instead of tilde between dates
  • removed no data line in the middle
  • used darker gray (used the same color (333) in floaters for no overlay)
  • removed x (close) in the top right
  • removed Reset button: removed forwardref
  • changed the name from Time Filter to Time Slider
  • static drawer mode (expand/shrink button)
    -- no interaction can be made under drawer mode (minimized)
    -- add Expand/Shrink button to switch static drawer mode
    -- reset to initial value under drawer mode

todo

  • data request: distribution
  • post processing response for the time slider data format
  • applying selected range to filters
  • adjusting input variable selector

A captured video is attached in the following:
https://github.com/VEuPathDB/web-monorepo/assets/12802305/4e018aac-7c82-4060-9834-abcaac3275b6

Copy link
Member

@dmfalke dmfalke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I think this code is looking nice and healthy. I made a few suggestions, which I think will make things a little bit easier to find and work with. Let me know what you think, @bobular

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see the additions in the file moved to a file specific to the ez time slider

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Sorted in 8b4c75f

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if the business logic in the component should be factored out into a new file, packages/libs/eda/src/lib/core/components/filter/EZTimeFilter.tsx. I would remove the DraggablePanel part of it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've already renamed it EZTimeFilter and removed the draggable. I haven't done any further refactoring as yet.

Comment on lines 102 to 107
timeSliderVariable: VariableDescriptor,
timeSliderSelectedRange: t.type({
start: t.string,
end: t.string,
}),
timeSliderActive: t.boolean,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see this all combined into one object, something like:

timeSliderConfig: {
  variable: VariableDescriptor,
  selectedRange: t.type({
    start: t.string,
    end: t.string,
  }),
  isActive: t.boolean,
}

Comment on lines 252 to 254
setTimeSliderVariable: useSetter('timeSliderVariable'),
setTimeSliderSelectedRange: useSetter('timeSliderSelectedRange'),
setTimeSliderActive: useSetter('timeSliderActive'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to my other comment, this could be

setTimeSliderConfig: useSetter('timeSliderconfig')

Copy link
Member

@dmfalke dmfalke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just made an observation: the date filters that get generated by the ez time filter are missing the time component, which causes a backend error, so that needs to be addressed somehow.

A related thought I just had is, perhaps appState can store a filter object? E.g.,

ezTimeFilterConfig: {
  isActive: boolean,
  filter?: Filter
}

Copy link
Member

@dmfalke dmfalke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran into a bug that will impact studies that don't have a temporal date variable.

Comment on lines 301 to 318
// take the first suitable variable from the filtered variable tree

// first find the first entity with some variables that passed the filter
const defaultTimeSliderEntity: StudyEntity | undefined = Array.from(
preorder(temporalVariableTree, (entity) => entity.children ?? [])
)
// not all `variables` are actually variables, so we filter to be sure
.filter(
(entity) =>
entity.variables.filter((variable) => Variable.is(variable))
.length > 0
)[0];

// then take the first variable from it
const defaultTimeSliderVariable: Variable | undefined =
defaultTimeSliderEntity.variables.filter(
(variable): variable is Variable => Variable.is(variable)
)[0];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran into a runtime bug with this code, when I was playing with constraints. For some reason, typescript is allowing you to assume that defaultTimeSliderEntity is defined on line 316, even thought it might be null. The suggested change below will address this, and I think it's also a little more performant (filter will iterate the entire array, while find and some will terminate after the first element for which the callback returns true).

Suggested change
// take the first suitable variable from the filtered variable tree
// first find the first entity with some variables that passed the filter
const defaultTimeSliderEntity: StudyEntity | undefined = Array.from(
preorder(temporalVariableTree, (entity) => entity.children ?? [])
)
// not all `variables` are actually variables, so we filter to be sure
.filter(
(entity) =>
entity.variables.filter((variable) => Variable.is(variable))
.length > 0
)[0];
// then take the first variable from it
const defaultTimeSliderVariable: Variable | undefined =
defaultTimeSliderEntity.variables.filter(
(variable): variable is Variable => Variable.is(variable)
)[0];
// take the first suitable variable from the filtered variable tree
// first find the first entity with some variables that passed the filter
const defaultTimeSliderEntity = Array.from(
preorder(temporalVariableTree, (node) => node.children ?? [])
).find((entity) => entity.variables.some(Variable.is));
// then take the first variable from it
const defaultTimeSliderVariable = defaultTimeSliderEntity?.variables.find(
Variable.is
);
return defaultTimeSliderEntity != null &&
defaultTimeSliderVariable != null
? {
entityId: defaultTimeSliderEntity.id,
variableId: defaultTimeSliderVariable.id,
}
: undefined;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Furthermore, when this function returns undefined, the MapAnalysisImpl component enters and infinite loop.

@bobular
Copy link
Member

bobular commented Sep 9, 2023

Hi @dmfalke - thanks again for your pre-review.

I've now refactored the timeSlider* props into timeSliderConfig and simplified quite a lot.

It's working the same as before, even though I haven't wrapped the setSelectedRange handler passed to EZTimeFilter in a useCallback. Was that OK?

@bobular bobular marked this pull request as ready for review September 9, 2023 15:36
Copy link
Member

@dmfalke dmfalke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is looking great. We should merge this asap so folks can see it.

@bobular bobular merged commit 88c719f into main Sep 11, 2023
1 check passed
@bobular bobular deleted the 245-ez-time-filter-component branch September 11, 2023 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Map: ez time filter
5 participants