Release/v0.21.3 (#3282)

* Refactor schemaControls, use selectField() when searching, cleaner schema documentation selection mode is default now for filed visibility * update count dependency to refelct when applying filter rules * add documentation links * better disable field handling - (un)select is broken - fixing next * improve select all control * uncheck subpaths even if they are disabled * show embedded document in selection mode * field doc type using tertiary text color * minor tweaks in color modal title style * review comments and fix include nested fields selection view * review comments, refactor selection row, and count fix * add first vitest for visibility, upgrade vitest packages * add basic test and refactor more * keep one embed doc field * no coverage when yarn test * add support for multi-select with string type * adds a Docker Hub Pulls badge * Typo * tweaking * lint * Add files via upload * Update brain.rst * Update index.rst * Update index.rst * Update index.rst * Update index.rst * Release v0.21.2 (#3251) (#3255) * update sidebar test, fix small regressions (#3250) * remove next index ref, fix string filter is matching default (#3249) * release notes * package bumps * disabled e2e * lint * fix mac arm64 * Teams v1.3.2 release note * rm e2e * add main * add icon * typo * first pass at milvus integration docs * adding milvus to user_guide/brain * adding to integrations index * adding milvus logo * square image, lower resolution * editing pass * consistency * bugfixes * disable field unit testing * group and video dataset unit tests disabled * toPatches disabling tests * toClips * label types test * more granular field selection when disabling label subfields * geolocation, heatmap, segmentation tests * complete disabling test * review comments * enable vector fields, disable top level id field, more grnular disabling of FrameSupportField and FrameNumberField * disable list of valid labels from visibility toggle * review comments * linting * editing pass * tweaks * cleanup * fixing capitalization * adding new vector integrations * bumping package versions * adding release notes * or count label tags (#3267) * has group slices only when group dataset (#3262) * Updating `Session.close()` (#3253) * kill server on session close * close desktop app * cleanup * lint * lint * is_open * rm remote warning * Embedded frame label fixes (#3256) * add frame cases to dynamic label tests * embedded frame label fixes * linting * adding a merge_sample() method * Minor typo fixes * adding test for one() method * updating release notes * tweak * documenting list bucket perms * Fix sidebar matching on label fields (#3270) * only matches * db_field bug and path fix * Suppress errors due to none fields (#3275) * don't throw error * add debug msg * import order * Fixing #3277 (#3279) * fixing filter_keypoints() bug * adding dynamic doc test * interactive typo fix for Dave * removing persistent dataset usage in unit tests * Sidebar filtering tests and fixes (#3280) * only matches * db_field bug and path fix * base image sample tests * cleanup * exclude bug * rm onlyMatch * frame and dynamic tests * adding coverage * keypoints fixes * cleanup * base image sample tests * cleanup * exclude bug * rm onlyMatch * frame and dynamic tests * adding coverage * keypoints fixes * cleanup * tweaks * exclude no only matches * add to release notes * whitespace * updating release notes * Minor fixes (#3283) * docs tweaks * documenting cache=True feature --------- Co-authored-by: manivoxel51 <mani@voxel51@gmail.com> Co-authored-by: Lanny W <lanzhenwang9@gmail.com> Co-authored-by: manivoxel51 <109545780+manivoxel51@users.noreply.github.com> Co-authored-by: Brian Moore <brian@voxel51.com> Co-authored-by: imanjra <ibrahim@voxel51.com> Co-authored-by: topher <topher@voxel51.com> Co-authored-by: Jacob Marks <jamarks13@gmail.com> Co-authored-by: brimoor <brimoor@umich.edu> Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com> Co-authored-by: Kishan Savant <ksavant577@gmail.com> Co-authored-by: Kacey <kacey@voxel51.com> Co-authored-by: Kishan Savant <66986430+NeoKish@users.noreply.github.com>
voxel51 · Jul 12, 2023 · b5ae567 · b5ae567
1 parent 20b0b6e
commit b5ae567
Show file tree

Hide file tree

Showing 51 changed files with 4,456 additions and 993 deletions.
diff --git a/README.md b/README.md
@@ -22,6 +22,7 @@ models**
 [![PyPI python](https://img.shields.io/pypi/pyversions/fiftyone)](https://pypi.org/project/fiftyone)
 [![PyPI version](https://badge.fury.io/py/fiftyone.svg)](https://pypi.org/project/fiftyone)
 [![Downloads](https://pepy.tech/badge/fiftyone)](https://pepy.tech/project/fiftyone)
+[![Docker Pulls](https://badgen.net/docker/pulls/voxel51/fiftyone?icon=docker&label=pulls)](https://hub.docker.com/r/voxel51/fiftyone/)
 [![Build](https://github.com/voxel51/fiftyone/workflows/Build/badge.svg?branch=develop&event=push)](https://github.com/voxel51/fiftyone/actions?query=workflow%3ABuild)
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
 [![Slack](https://img.shields.io/badge/Slack-4A154B?logo=slack&logoColor=white)](https://slack.voxel51.com)

diff --git a/app/package.json b/app/package.json
@@ -16,7 +16,7 @@
         "start": "yarn workspace @fiftyone/app start",
         "start-desktop": "yarn workspace FiftyOne start-desktop",
         "test": "yarn vitest run",
-        "test-ui": "yarn vitest --ui",
+        "test-ui": "yarn vitest --ui --coverage",
         "gen:schema": "strawberry export-schema fiftyone.server.app:schema > schema.graphql"
     },
     "devDependencies": {

diff --git a/app/packages/core/src/components/Filters/BooleanFieldFilter.tsx b/app/packages/core/src/components/Filters/BooleanFieldFilter.tsx
@@ -1,12 +1,10 @@
-import React from "react";
-
 import {
+  boolExcludeAtom,
+  boolIsMatchingAtom,
   booleanCountResults,
   booleanSelectedValuesAtom,
-  boolIsMatchingAtom,
-  boolOnlyMatchAtom,
-  boolExcludeAtom,
 } from "@fiftyone/state";
+import React from "react";
 import CategoricalFilter from "./categoricalFilter/CategoricalFilter";
 
 const BooleanFieldFilter = ({
@@ -27,7 +25,6 @@ const BooleanFieldFilter = ({
     <CategoricalFilter<{ value: boolean | null; count: number }>
       selectedValuesAtom={booleanSelectedValuesAtom({ path, modal })}
       isMatchingAtom={boolIsMatchingAtom({ path, modal })}
-      onlyMatchAtom={boolOnlyMatchAtom({ path, modal })}
       excludeAtom={boolExcludeAtom({ path, modal })}
       countsAtom={booleanCountResults({
         path,

diff --git a/app/packages/core/src/components/Filters/LabelFieldFilter.tsx b/app/packages/core/src/components/Filters/LabelFieldFilter.tsx
@@ -2,12 +2,11 @@ import React from "react";
 
 import {
   isMatchingAtom,
-  onlyMatchAtom,
   stringExcludeAtom,
   stringSelectedValuesAtom,
 } from "@fiftyone/state";
-import CategoricalFilter from "./categoricalFilter/CategoricalFilter";
 import { labelTagsCount } from "../Sidebar/Entries/EntryCounts";
+import CategoricalFilter from "./categoricalFilter/CategoricalFilter";
 
 const LabelTagFieldFilter = ({
   path,
@@ -27,7 +26,6 @@ const LabelTagFieldFilter = ({
     <CategoricalFilter<{ value: string | null; count: number }>
       selectedValuesAtom={stringSelectedValuesAtom({ modal, path })}
       excludeAtom={stringExcludeAtom({ modal, path })}
-      onlyMatchAtom={onlyMatchAtom({ modal, path })}
       isMatchingAtom={isMatchingAtom({ modal, path })}
       countsAtom={labelTagsCount({ modal, extended: false })}
       path={path}

diff --git a/app/packages/core/src/components/Filters/NumericFieldFilter.tsx b/app/packages/core/src/components/Filters/NumericFieldFilter.tsx
@@ -10,14 +10,14 @@ import styled from "styled-components";
 
 import * as fos from "@fiftyone/state";
 
-import RangeSlider from "../Common/RangeSlider";
-import Checkbox from "../Common/Checkbox";
-import { Button } from "../utils";
 import { DATE_FIELD, DATE_TIME_FIELD, FLOAT_FIELD } from "@fiftyone/utilities";
 import { formatDateTime } from "../../utils/generic";
-import withSuspense from "./withSuspense";
+import Checkbox from "../Common/Checkbox";
+import RangeSlider from "../Common/RangeSlider";
 import FieldLabelAndInfo from "../FieldLabelAndInfo";
+import { Button } from "../utils";
 import FilterOption from "./categoricalFilter/filterOption/FilterOption";
+import withSuspense from "./withSuspense";
 
 const NamedRangeSliderContainer = styled.div`
   margin: 3px;
@@ -135,11 +135,6 @@ const NumericFieldFilter = ({
     modal,
     defaultRange,
   });
-  const onlyMatchAtom = fos.numericOnlyMatchAtom({
-    path,
-    modal,
-    defaultRange,
-  });
   const values = useRecoilValue(
     fos.rangeAtom({
       modal,
@@ -149,7 +144,6 @@ const NumericFieldFilter = ({
     })
   );
   const setExcluded = excludeAtom ? useSetRecoilState(excludeAtom) : null;
-  const setOnlyMatch = onlyMatchAtom ? useSetRecoilState(onlyMatchAtom) : null;
   const setIsMatching = isMatchingAtom
     ? useSetRecoilState(isMatchingAtom)
     : null;
@@ -207,7 +201,6 @@ const NumericFieldFilter = ({
   const initializeSettings = () => {
     setFilter([null, null]);
     setExcluded && setExcluded(false);
-    setOnlyMatch && setOnlyMatch(true);
     setIsMatching && setIsMatching(!nestedField);
   };
 
@@ -304,7 +297,6 @@ const NumericFieldFilter = ({
             nestedField={nestedField}
             shouldNotShowExclude={false} // only boolean fields don't use exclude
             excludeAtom={excludeAtom}
-            onlyMatchAtom={onlyMatchAtom}
             isMatchingAtom={isMatchingAtom}
             valueName={field?.name ?? ""}
             path={path}

diff --git a/app/packages/core/src/components/Filters/StringFieldFilter.tsx b/app/packages/core/src/components/Filters/StringFieldFilter.tsx
@@ -1,12 +1,10 @@
-import React from "react";
 import * as fos from "@fiftyone/state";
-
 import {
   isMatchingAtom,
-  onlyMatchAtom,
   stringExcludeAtom,
   stringSelectedValuesAtom,
 } from "@fiftyone/state";
+import React from "react";
 import CategoricalFilter from "./categoricalFilter/CategoricalFilter";
 
 const StringFieldFilter = ({
@@ -27,7 +25,6 @@ const StringFieldFilter = ({
     <CategoricalFilter<{ value: string | null; count: number }>
       selectedValuesAtom={stringSelectedValuesAtom({ modal, path })}
       excludeAtom={stringExcludeAtom({ modal, path })}
-      onlyMatchAtom={onlyMatchAtom({ modal, path })}
       isMatchingAtom={isMatchingAtom({ modal, path })}
       countsAtom={fos.stringCountResults({
         modal,

diff --git a/app/packages/core/src/components/Filters/categoricalFilter/CategoricalFilter.tsx b/app/packages/core/src/components/Filters/categoricalFilter/CategoricalFilter.tsx
@@ -175,7 +175,6 @@ interface Props<T extends V = V> {
   selectedValuesAtom: RecoilState<T["value"][]>;
   excludeAtom: RecoilState<boolean>; // toggles select or exclude
   isMatchingAtom: RecoilState<boolean>; // toggles match or filter
-  onlyMatchAtom: RecoilState<boolean>; // toggles onlyMatch mode (omit empty samples)
   countsAtom: RecoilValue<{
     count: number;
     results: [T["value"], number][];
@@ -190,7 +189,6 @@ const CategoricalFilter = <T extends V = V>({
   countsAtom,
   selectedValuesAtom,
   excludeAtom,
-  onlyMatchAtom,
   isMatchingAtom,
   path,
   modal,
@@ -203,6 +201,7 @@ const CategoricalFilter = <T extends V = V>({
     : path.startsWith("_label_tags")
     ? "label tag"
     : name;
+
   const selectedCounts = useRef(new Map<V["value"], number>());
   const onSelect = useOnSelect(selectedValuesAtom, selectedCounts);
   const useSearch = getUseSearch({ modal, path });
@@ -213,7 +212,7 @@ const CategoricalFilter = <T extends V = V>({
 
   // id fields should always use filter mode
   const neverShowExpansion = field?.ftype?.includes("ObjectIdField");
-
+  if (countsLoadable.state === "hasError") throw countsLoadable.contents;
   if (countsLoadable.state !== "hasValue") return null;
   const { count, results } = countsLoadable.contents;
 
@@ -267,7 +266,6 @@ const CategoricalFilter = <T extends V = V>({
           selectedValuesAtom={selectedValuesAtom}
           excludeAtom={excludeAtom}
           isMatchingAtom={isMatchingAtom}
-          onlyMatchAtom={onlyMatchAtom}
           modal={modal}
           totalCount={count}
           selectedCounts={selectedCounts}

diff --git a/app/packages/core/src/components/Filters/categoricalFilter/Wrapper.tsx b/app/packages/core/src/components/Filters/categoricalFilter/Wrapper.tsx
@@ -8,18 +8,17 @@ import {
 
 import * as fos from "@fiftyone/state";
 
-import FilterOption from "./filterOption/FilterOption";
 import Checkbox from "../../Common/Checkbox";
 import { Button } from "../../utils";
 import { CHECKBOX_LIMIT, nullSort } from "../utils";
-import { isKeypointLabel, V } from "./CategoricalFilter";
+import { V, isKeypointLabel } from "./CategoricalFilter";
+import FilterOption from "./filterOption/FilterOption";
 
 interface WrapperProps {
   results: [V["value"], number][];
   selectedValuesAtom: RecoilState<V["value"][]>;
   excludeAtom: RecoilState<boolean>;
   isMatchingAtom: RecoilState<boolean>;
-  onlyMatchAtom: RecoilState<boolean>;
   color: string;
   totalCount: number;
   modal: boolean;
@@ -34,7 +33,6 @@ const Wrapper = ({
   selectedValuesAtom,
   excludeAtom,
   isMatchingAtom,
-  onlyMatchAtom,
   modal,
   path,
   selectedCounts,
@@ -44,7 +42,6 @@ const Wrapper = ({
   const [selected, setSelected] = useRecoilState(selectedValuesAtom);
   const selectedSet = new Set(selected);
   const setExcluded = excludeAtom ? useSetRecoilState(excludeAtom) : null;
-  const setOnlyMatch = onlyMatchAtom ? useSetRecoilState(onlyMatchAtom) : null;
   const setIsMatching = isMatchingAtom
     ? useSetRecoilState(isMatchingAtom)
     : null;
@@ -87,7 +84,6 @@ const Wrapper = ({
 
   const initializeSettings = () => {
     setExcluded && setExcluded(false);
-    setOnlyMatch && setOnlyMatch(true);
     setIsMatching && setIsMatching(!nestedField);
   };
 
@@ -144,7 +140,6 @@ const Wrapper = ({
               nestedField={nestedField}
               shouldNotShowExclude={shouldNotShowExclude}
               excludeAtom={excludeAtom}
-              onlyMatchAtom={onlyMatchAtom}
               isMatchingAtom={isMatchingAtom}
               valueName={name}
               color={color}

diff --git a/app/packages/core/src/components/Filters/categoricalFilter/filterOption/FilterOption.tsx b/app/packages/core/src/components/Filters/categoricalFilter/filterOption/FilterOption.tsx
@@ -1,27 +1,24 @@
-import React, { PropsWithChildren, useEffect } from "react";
-import styled from "styled-components";
-import { RecoilState, useRecoilState, useSetRecoilState } from "recoil";
 import FilterAltIcon from "@mui/icons-material/FilterAlt";
 import FilterAltOffIcon from "@mui/icons-material/FilterAltOff";
-import ImageIcon from "@mui/icons-material/Image";
 import HideImageIcon from "@mui/icons-material/HideImage";
+import ImageIcon from "@mui/icons-material/Image";
 import { IconButton } from "@mui/material";
-import { useSpring } from "framer-motion";
 import Color from "color";
+import React, { useEffect } from "react";
+import { RecoilState, useRecoilState, useSetRecoilState } from "recoil";
+import styled from "styled-components";
 
-import { useOutsideClick } from "@fiftyone/state";
 import { useTheme } from "@fiftyone/components/src/components/ThemeProvider";
 import Tooltip from "@fiftyone/components/src/components/Tooltip";
+import { useOutsideClick } from "@fiftyone/state";
 
-import { PopoutDiv } from "../../../utils";
-import Item from "./FilterItem";
 import { Popout } from "@fiftyone/components";
+import Item from "./FilterItem";
 
 interface Props {
   nestedField: string | undefined; // nested ListFields only ("detections")
   shouldNotShowExclude: boolean; // for BooleanFields
   excludeAtom: RecoilState<boolean>;
-  onlyMatchAtom: RecoilState<boolean>;
   isMatchingAtom: RecoilState<boolean>;
   valueName: string;
   color: string;
@@ -132,15 +129,13 @@ const FilterOption: React.FC<Props> = ({
   nestedField,
   shouldNotShowExclude,
   excludeAtom,
-  onlyMatchAtom,
   isMatchingAtom,
 }) => {
   const isLabelTag = path?.startsWith("_label_tags");
   const isSampleTag = path?.startsWith("tag");
 
   const [open, setOpen] = React.useState(false);
   const [excluded, setExcluded] = useRecoilState(excludeAtom);
-  const setOnlyMatch = onlyMatchAtom ? useSetRecoilState(onlyMatchAtom) : null;
   const setIsMatching = isMatchingAtom
     ? useSetRecoilState(isMatchingAtom)
     : null;
@@ -213,25 +208,21 @@ const FilterOption: React.FC<Props> = ({
   const onSelectFilter = () => {
     setExcluded && setExcluded(false);
     setIsMatching && setIsMatching(false);
-    setOnlyMatch && setOnlyMatch(true);
   };
 
   const onSelectNegativeFilter = () => {
     setExcluded && setExcluded(true);
     setIsMatching && setIsMatching(false);
-    setOnlyMatch && setOnlyMatch(false);
   };
 
   const onSelectMatch = () => {
     setExcluded && setExcluded(false);
     setIsMatching && setIsMatching(true);
-    setOnlyMatch && setOnlyMatch(true);
   };
 
   const onSelectNegativeMatch = () => {
     setExcluded && setExcluded(true);
     setIsMatching && setIsMatching(true);
-    setOnlyMatch && setOnlyMatch(true);
   };
 
   const children = (

diff --git a/app/packages/core/src/plugins/SchemaIO/components/DropdownView.tsx b/app/packages/core/src/plugins/SchemaIO/components/DropdownView.tsx
@@ -1,15 +1,46 @@
+import { Box, ListItemText, MenuItem, Select, Typography } from "@mui/material";
 import React from "react";
-import { Box, Typography, Select, MenuItem, ListItemText } from "@mui/material";
-import FieldWrapper from "./FieldWrapper";
-import autoFocus from "../utils/auto-focus";
 import { getComponentProps } from "../utils";
+import autoFocus from "../utils/auto-focus";
+import AlertView from "./AlertView";
+import FieldWrapper from "./FieldWrapper";
+
+const MULTI_SELECT_TYPES = ["string", "array"];
 
 export default function DropdownView(props) {
   const { onChange, schema, path, data } = props;
-  const { view = {}, type } = schema;
-  const { choices, placeholder = "", readOnly } = view;
+  const { default: defaultValue, view = {}, type } = schema;
+  const {
+    choices,
+    multiple: multiSelect,
+    placeholder = "",
+    separator = ",",
+    readOnly,
+  } = view;
+
+  if (multiSelect && !MULTI_SELECT_TYPES.includes(type))
+    return (
+      <AlertView
+        schema={{
+          view: {
+            label: `Unsupported type "${type}" for multi-select`,
+            description:
+              "Multi-select is supported for types " +
+              MULTI_SELECT_TYPES.join(", "),
+            severity: "error",
+          },
+        }}
+      />
+    );
 
-  const multiple = type === "array";
+  const isArrayType = type === "array";
+  const multiple = multiSelect || isArrayType;
+  const fallbackDefaultValue = multiple ? [] : "";
+  const rawDefaultValue = data ?? defaultValue ?? fallbackDefaultValue;
+  const computedDefaultValue =
+    multiple && !Array.isArray(rawDefaultValue)
+      ? rawDefaultValue.toString().split(separator)
+      : rawDefaultValue;
 
   const choiceLabels = choices.reduce((labels, choice) => {
     labels[choice.value] = choice.label;
@@ -21,7 +52,7 @@ export default function DropdownView(props) {
       <Select
         disabled={readOnly}
         autoFocus={autoFocus(props)}
-        defaultValue={data ?? schema.default ?? (multiple ? [] : "")}
+        defaultValue={computedDefaultValue}
         size="small"
         fullWidth
         displayEmpty
@@ -34,7 +65,14 @@ export default function DropdownView(props) {
           }
           return choiceLabels[value] || value;
         }}
-        onChange={(e) => onChange(path, e.target.value)}
+        onChange={(e) => {
+          const value = e.target.value;
+          const computedValue =
+            Array.isArray(value) && type !== "array"
+              ? value.join(separator)
+              : value;
+          onChange(path, computedValue);
+        }}
         multiple={multiple}
         {...getComponentProps(props, "select")}
       >