Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve source selection UX #6766

Merged
merged 35 commits into from Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
6615151
Add new source option styling for pasting from clipboard
hannahblair Dec 12, 2023
08a0416
prevent device selection cut off
hannahblair Dec 12, 2023
f99fa3a
Check for dupe sources in source selection
hannahblair Dec 13, 2023
5ec31b8
tweaks
hannahblair Dec 13, 2023
60dc45e
tweak
hannahblair Dec 13, 2023
9756b35
add image interaction test
hannahblair Dec 13, 2023
d6c1e93
more tests
hannahblair Dec 13, 2023
838a16f
improve light/dark mode color contrast
hannahblair Dec 13, 2023
352f714
Merge branch 'main' into improve-source-selection
hannahblair Dec 13, 2023
9feb0cc
add changeset
gradio-pr-bot Dec 13, 2023
dc1526f
remove unused prop
hannahblair Dec 13, 2023
8681c41
Merge branch 'improve-source-selection' of github.com:gradio-app/grad…
hannahblair Dec 13, 2023
b0be967
add no device found placeholder
hannahblair Dec 13, 2023
94f2d51
style tweak
hannahblair Dec 13, 2023
8bbc7ed
Merge branch 'main' into improve-source-selection
hannahblair Dec 14, 2023
5d24f01
allow pasting on click + add e2e test
hannahblair Dec 14, 2023
8e7426d
Merge branch 'improve-source-selection' of github.com:gradio-app/grad…
hannahblair Dec 14, 2023
d5dec81
Merge branch 'main' into improve-source-selection
hannahblair Dec 14, 2023
e44d909
Merge branch 'main' into improve-source-selection
hannahblair Dec 15, 2023
c9ed31a
Merge branch 'improve-source-selection' of github.com:gradio-app/grad…
hannahblair Dec 15, 2023
bf95184
Merge branch 'main' into improve-source-selection
hannahblair Dec 18, 2023
0397497
fix e2e tests
hannahblair Dec 18, 2023
3cdad9e
Merge branch 'improve-source-selection' of github.com:gradio-app/grad…
hannahblair Dec 18, 2023
a823d4f
formatting
hannahblair Dec 18, 2023
216b9e9
Merge branch 'main' into improve-source-selection
hannahblair Dec 18, 2023
c6689d0
add timeout to e2e test
hannahblair Dec 18, 2023
b8c695a
Merge branch 'improve-source-selection' of github.com:gradio-app/grad…
hannahblair Dec 18, 2023
81ca4a4
tweak
hannahblair Dec 19, 2023
e64cf2b
tweak test
hannahblair Dec 19, 2023
f28633e
change `getByLabel` to `getByText`
hannahblair Dec 19, 2023
87d3b61
value tweak
hannahblair Dec 19, 2023
7f75408
logic tweak
hannahblair Dec 19, 2023
59a64da
test
hannahblair Dec 19, 2023
b6d8f54
formatting
hannahblair Dec 19, 2023
1a98513
Merge branch 'main' into improve-source-selection
hannahblair Dec 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion js/app/src/lang/en.json
Expand Up @@ -110,6 +110,7 @@
"drop_csv": "Drop CSV Here",
"drop_file": "Drop File Here",
"drop_image": "Drop Image Here",
"drop_video": "Drop Video Here"
"drop_video": "Drop Video Here",
"paste_clipboard": "Paste from Clipboard"
}
}
42 changes: 26 additions & 16 deletions js/atoms/src/SelectSource.svelte
@@ -1,22 +1,28 @@
<script lang="ts">
import { Microphone, Upload, Video } from "@gradio/icons";
import { Microphone, Upload, Webcam, ImagePaste } from "@gradio/icons";

export let sources: string[];
export let active_source: string;
type sources = "upload" | "microphone" | "webcam" | "clipboard" | null;

export let sources: Partial<sources>[];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the use of Partial here and throughout this file? I don't believe it makes a difference in this context.

Suggested change
export let sources: Partial<sources>[];
export let sources: sources[];

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also would name the type differently if we also have a variable called sources.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial is needed because the SelectSource is used in Video, Audio and Image and sources[] isn't the same for them all - e.g. microphone isn't used by Image

export let active_source: Partial<sources>;
export let handle_clear: () => void = () => {};
export let handle_select: (source_type: Partial<sources>) => void = () => {};

async function handle_select_source(source: Partial<sources>): Promise<void> {
handle_clear();
active_source = source;
handle_select(source);
}
</script>

{#if sources.length > 1}
<span class="source-selection" data-testid="source-select">
{#if sources.includes("upload")}
<button
class="icon"
class:selected={active_source === "upload"}
class:selected={active_source === "upload" || !active_source}
aria-label="Upload file"
on:click={() => {
handle_clear();
active_source = "upload";
}}><Upload /></button
on:click={() => handle_select_source("upload")}><Upload /></button
>
{/if}

Expand All @@ -25,10 +31,8 @@
class="icon"
class:selected={active_source === "microphone"}
aria-label="Record audio"
on:click={() => {
handle_clear();
active_source = "microphone";
}}><Microphone /></button
on:click={() => handle_select_source("microphone")}
><Microphone /></button
>
{/if}

Expand All @@ -37,10 +41,16 @@
class="icon"
class:selected={active_source === "webcam"}
aria-label="Record video"
on:click={() => {
handle_clear();
active_source = "webcam";
}}><Video /></button
on:click={() => handle_select_source("webcam")}><Webcam /></button
>
{/if}
{#if sources.includes("clipboard")}
<button
class="icon"
class:selected={active_source === "clipboard"}
aria-label="Paste from clipboard"
on:click={() => handle_select_source("clipboard")}
><ImagePaste /></button
>
{/if}
</span>
Expand Down
16 changes: 12 additions & 4 deletions js/atoms/src/UploadText.svelte
@@ -1,7 +1,8 @@
<script lang="ts">
import type { I18nFormatter } from "@gradio/utils";
import { Upload as UploadIcon } from "@gradio/icons";
export let type: "video" | "image" | "audio" | "file" | "csv" = "file";
import { Upload as UploadIcon, ImagePaste } from "@gradio/icons";
export let type: "video" | "image" | "audio" | "file" | "csv" | "clipboard" =
"file";
export let i18n: I18nFormatter;
export let message: string | undefined = undefined;
export let mode: "full" | "short" = "full";
Expand All @@ -12,12 +13,19 @@
video: "upload_text.drop_video",
audio: "upload_text.drop_audio",
file: "upload_text.drop_file",
csv: "upload_text.drop_csv"
csv: "upload_text.drop_csv",
clipboard: "upload_text.paste_clipboard"
};
</script>

<div class="wrap">
<span class="icon-wrap" class:hovered><UploadIcon /> </span>
<span class="icon-wrap" class:hovered>
{#if type === "clipboard"}
<ImagePaste />
{:else}
<UploadIcon />
{/if}
</span>

{i18n(defs[type] || defs.file)}

Expand Down
14 changes: 8 additions & 6 deletions js/image/Index.svelte
Expand Up @@ -20,6 +20,8 @@
import type { LoadingStatus } from "@gradio/statustracker";
import { normalise_file } from "@gradio/client";

type sources = "upload" | "webcam" | "clipboard" | null;

export let elem_id = "";
export let elem_classes: string[] = [];
export let visible = true;
Expand Down Expand Up @@ -66,7 +68,7 @@
$: url && gradio.dispatch("change");

let dragging: boolean;
let active_tool: null | "webcam" = null;
let active_source: sources = null;
</script>

{#if !interactive}
Expand Down Expand Up @@ -124,7 +126,7 @@
/>

<ImageUploader
bind:active_tool
bind:active_source
bind:value
selectable={_selectable}
{root}
Expand All @@ -144,17 +146,17 @@
loading_status.status = "error";
gradio.dispatch("error", detail);
}}
on:click={() => gradio.dispatch("error", "bad thing happened")}
hannahblair marked this conversation as resolved.
Show resolved Hide resolved
on:error
{label}
{show_label}
{pending}
{streaming}
{mirror_webcam}
i18n={gradio.i18n}
>
{#if sources.includes("upload")}
<UploadText i18n={gradio.i18n} type="image" mode="short" />
{#if active_source === "upload" || !active_source}
<UploadText i18n={gradio.i18n} type="image" />
{:else if active_source === "clipboard"}
<UploadText i18n={gradio.i18n} type="clipboard" mode="short" />
{:else}
<Empty unpadded_box={true} size="large"><Image /></Empty>
{/if}
Expand Down
82 changes: 23 additions & 59 deletions js/image/shared/ImageUploader.svelte
Expand Up @@ -4,27 +4,20 @@
import { Image } from "@gradio/icons";
import type { SelectData, I18nFormatter } from "@gradio/utils";
import { get_coordinates_of_clicked_image } from "./utils";
import {
Webcam as WebcamIcon,
ImagePaste,
Upload as UploadIcon
} from "@gradio/icons";
import Webcam from "./Webcam.svelte";
import { Toolbar, IconButton } from "@gradio/atoms";

import { Upload } from "@gradio/upload";
import { type FileData, normalise_file } from "@gradio/client";
import ClearImage from "./ClearImage.svelte";
import { SelectSource } from "@gradio/atoms";

export let value: null | FileData;
export let label: string | undefined = undefined;
export let show_label: boolean;

export let sources: ("clipboard" | "webcam" | "upload")[] = [
"upload",
"clipboard",
"webcam"
];
type source_type = "upload" | "webcam" | "clipboard" | "microphone" | null;

export let sources: source_type[] = ["upload", "clipboard", "webcam"];
export let streaming = false;
export let pending = false;
export let mirror_webcam: boolean;
Expand All @@ -34,19 +27,25 @@

let upload: Upload;
let uploading = false;
export let active_tool: "webcam" | null = null;
export let active_source: source_type = null;

function handle_upload({ detail }: CustomEvent<FileData>): void {
value = normalise_file(detail, root, null);
dispatch("upload");
}

function handle_clear(): void {
value = null;
dispatch("clear");
dispatch("change", null);
}

async function handle_save(img_blob: Blob | any): Promise<void> {
pending = true;
const f = await upload.load_files([new File([img_blob], `webcam.png`)]);

value = f?.[0] || null;
if (!streaming) active_tool = null;
if (!streaming) active_source = null;

await tick();

Expand Down Expand Up @@ -78,31 +77,9 @@
}
}

const sources_meta = {
upload: {
icon: UploadIcon,
label: i18n("Upload"),
order: 0
},
webcam: {
icon: WebcamIcon,
label: i18n("Webcam"),
order: 1
},
clipboard: {
icon: ImagePaste,
label: i18n("Paste"),
order: 2
}
};

$: sources_list = sources.sort(
(a, b) => sources_meta[a].order - sources_meta[b].order
);

hannahblair marked this conversation as resolved.
Show resolved Hide resolved
$: {
if (sources.length === 1 && sources[0] === "webcam") {
active_tool = "webcam";
active_source = "webcam";
}
}

Expand All @@ -128,12 +105,6 @@
}
});
break;
case "webcam":
active_tool = "webcam";
break;
case "upload":
upload.open_file_upload();
break;
hannahblair marked this conversation as resolved.
Show resolved Hide resolved
default:
break;
}
Expand All @@ -153,7 +124,7 @@
{/if}
<div class="upload-container">
<Upload
hidden={value !== null || active_tool === "webcam"}
hidden={value !== null || active_source === "webcam"}
bind:this={upload}
bind:uploading
bind:dragging
Expand All @@ -163,11 +134,11 @@
{root}
disable_click={!sources.includes("upload")}
>
{#if value === null && !active_tool}
{#if value === null}
<slot />
{/if}
</Upload>
{#if active_tool === "webcam"}
{#if active_source === "webcam"}
<Webcam
on:capture={(e) => handle_save(e.detail)}
on:stream={(e) => handle_save(e.detail)}
Expand All @@ -192,27 +163,20 @@
{/if}
</div>
{#if sources.length > 1 || sources.includes("clipboard")}
<Toolbar show_border={!value?.url}>
{#each sources_list as source}
<IconButton
on:click={() => handle_toolbar(source)}
Icon={sources_meta[source].icon}
size="large"
label="{source}-image-toolbar-btn"
padded={false}
/>
{/each}
</Toolbar>
<SelectSource
{sources}
bind:active_source
handle_clear={() => handle_clear()}
handle_select={handle_toolbar}
/>
{/if}
</div>

<style>
/* .image-container {
height: auto;
} */
img {
width: var(--size-full);
height: var(--size-full);
object-fit: cover;
hannahblair marked this conversation as resolved.
Show resolved Hide resolved
}

.upload-container {
Expand Down