Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Add ui-react-liveness and FaceLivenessDetector component (#3677)
* fix: Update error handling (#679) * WIP * chore: update error state handling to only user onError, added onFailure callback * chore: move default error components to separate components (#680) * chore: move default error components to separate components * chore: update unit tests * fix: remove unused variable * refactor(liveness): move examples/liveness-next to examples/next (#681) * refactor(liveness): add liveness example to next dir * refactor(liveness): fix liveness example in next dir * refactor(liveness): remove examples/liveness-next * refactor(liveness): update README and env example Co-authored-by: MJ☔ <zhamujun@amazon.com> * fix: remove dotted line from liveness oval (#684) * fix(liveness): update the links after refactor the path (#685) Co-authored-by: MJ☔ <zhamujun@amazon.com> * chore: clean up app fixes, remove example logo (#686) * fix: set liveness width seed to 1.0 (#688) * fix: added message about multiple faces before countdown (#689) * fix: set default video bit rate to 1 Mbps (#690) * fix: handle errors from liveness stream with default server error messaging (#692) * fix: handle errors from liveness stream with default server error messaging * fix: unit tests * refactor(liveness): TS strict helper liveness (#693) * refactor(liveness): TS Strict for helper liveness * refactor(liveness): TS Strict for liveness helper * refactor(liveness): TS Strict for liveness helper * refactor(liveness): add isClientFreshnessColorSequence helper * refactor(liveness): remove exclude from getColorsSequencesFromSessionInformation Co-authored-by: MJ☔ <zhamujun@amazon.com> * fix(liveness): getColorsSequencesFromSessionInformation type (#695) Co-authored-by: MJ☔ <zhamujun@amazon.com> * fix: re-order stop video events to fix empty chunk getting sent before all video events are done (#697) * fix: add amplify user agent to rekognition streaming client (#698) * fix: Remove unused deps (#696) * fix: remove unused deps webrtc-adapater, aws-sdk-liveness, uuid * fix: add back in xstate dep * add yarn.lock * fix: unit tests * fix: fix unit test * fix: analytics events that were negative values (#699) * fix: more copy updates (#701) * fix: more copy updates * fix: unit tests * chore: update freeze stream to show paused video during uploading (#700) * add pauseVideo action * update freezeStream to not wait for animation end * fix: small ux issues for fnf precheck (#704) * fix: small ux issues for fnf precheck * fix: unit test * Revert "fix: set liveness width seed to 1.0 (#688)" (#708) This reverts commit 6be24a9. * test(liveness): set up tests for start screen (#687) Co-authored-by: MJ☔ <zhamujun@amazon.com> * fix: ensure that recording has started before continuing (#702) * testing: added changes to increase video size (not resolution) by 10%, testing this change in beta this has not been reviewed * fix: switch back to original timestamps (#710) * fix: switch back to original timestamps * remove unused method * Revert "testing: added changes to increase video size (not resolution) by 10%, testing this change in beta this has not been reviewed" (#712) This reverts commit 6d81f9f. * chore: update copy from UXSO (#713) * Update FPS message * update max to maximum --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: added mobile landscape warning (#714) * chore: added mobile landscape warning * add safari friendly orientation detection (#716) * add safari friendly orientation detection * Add isMobile check back * Add back 480 height to LivenessCheck --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * fix lint * fix unit tests --------- Co-authored-by: Heather Buchel <hbuchel@gmail.com> Co-authored-by: Heather Buchel <buchel@amazon.com> * test(liveness): add more e2e tests (#711) * test(liveness): add popover icon test * test(liveness): add liveness detector tests * test(liveness): add disable start screen tests * test(liveness): add With Custom Components tests * test(liveness): modify --------- Co-authored-by: MJ☔ <zhamujun@amazon.com> * chore: test new thresholds (#717) * fix: update facebounding box generation and oval matching thresholds (#718) * chore: add more unit test coverage for liveness (#694) * chore: add more unit test coverage for liveness * chore: update device maxtouchpoints to check full undefined * chore: added logging for beta (#719) * chore: more beta logs * fix: testing an updated fix for oval matching * chore: add even more logging for beta * fix: try testing reducing min face match count to 3 * fix: add offset to timestamps based on how long it takes to do initial blazeface detection (#722) * chore(ci): disable unneeded workflows (#724) * chore:update test app to match example app * fix: update min face match logic to be time based rather than count based * fix: updated unit test, rename function * fix: remove max resolution cap for mediaRecorder (#731) * fix: remove max resolution cap for mediaRecorder * fix: update unit tests * fix: set recording timeslices to 200ms to fix low score issue (#733) * fix: make mediaWidth/height logic more device agnostic (#727) * fix: close icon misaligned (#734) Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: add orientation utility tests (#735) * fix: set recording start timestamp to be the min of calculated record… (#736) * fix(liveness): stream-provider type (#737) Co-authored-by: MJ☔ <zhamujun@amazon.com> * chore: update face matching algorithm for liveness (#741) * fix: remove all the logging statements we added for debugging (#732) * chore: switch to using third party cdn for tfjs/blazeface (#742) * fix: remove analytics calls baked into liveness component (#740) * fix: remove analytics calls baked into liveness component * remove analytics flag * fix: fix linting * chore: update client-rekognitionstreaming to SDK v3 (#739) * chore: update magic numbers to use values from challenge config (#743) * chore: update magic numbers to use values from challenge config * chore: update to use calibrated pupil distance formula * chore(liveness): move screen brightness instruction to the top * chore: update copy per fnf ga and move more strings to use dictionary (#748) * chore: update FaceLivenessDetector API based on meeting, removed onsu… (#746) * chore: update FaceLivenessDetector API based on meeting, removed onsuccess/onfailure, combined onError and onPermissionDenied * chore: fix unnecessary optional chaining * chore: added options for overriding the tfjs and blazeface cdn urls (#750) * chore: added options for overriding the tfjs and blazeface cdn urls * Update packages/ui/src/helpers/liveness/blazeface-face-detection.ts Co-authored-by: Heather Buchel <hbuchel@gmail.com> * Update packages/ui/src/helpers/liveness/blazeface-face-detection.ts Co-authored-by: Heather Buchel <hbuchel@gmail.com> --------- Co-authored-by: Heather Buchel <hbuchel@gmail.com> * chore: add VideoEndTimestamp to client info (#749) * chore: add VideoEndTimestamp to client info * chore: separate stopVideo and empty chunk dispatch events, update tests * chore: setting up android, swift, and react liveness docs pages (#752) * fix: remove extra data selectos from liveness (#753) * fix: remove extra data selectos from liveness * fix: rename prop data-testid to testId * chore: use responsive sizing for video element and 4:3 resolution (#730) * fix: add missing await when stopping liveenss video (#756) * fix: use intrinsic videoWidth/videoHeight for video dims for correct face matching (#755) * chore: set video width/height dims to intrinsic width to fix face matching from blazeface * chore: remove useMediaDimensions --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: update api based on br review (#754) * chore: update api based on br review * chore: more api feedback fixes * fix: small comment fix and build fix * fix: update to onAnalysisComplete and config * fix(liveness): flip drawn oval and update sessionInfo to use normalized face bounding box (#757) * fix(liveness): flip oval matching logic as well (#758) * chore(liveness): set up publishing new ui-react-liveness package (#759) * chore(liveness): set up publishing new ui-react-liveness package * update package.json and gitignore * Update packages/react-liveness/package.json Co-authored-by: Caleb Pollman <cpollman@amazon.com> * Update packages/react-liveness/package.json Co-authored-by: Caleb Pollman <cpollman@amazon.com> * remove coma --------- Co-authored-by: Caleb Pollman <cpollman@amazon.com> * chore(liveness): try increase video time chunks to 1000ms (#760) * chore(liveness): try increase video time chunks to 1000ms * chore(liveness): reorder events to prevent pauseafter countdown * fix unit tests * add android and swift info to docs * chore: use full screen oval canvas on mobile, anchor video elements (#762) * chore: use class name to apply mobile styles * reposition oval drawing * change translation method * cleanup * chore: anchor video ui elements to video frame * class name cleanup * remove unused values/cleanup --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: add debug logs and send them to sample app through hug events (#763) * chore: add debug logs and send them to sample app through hug events * remove core mocking * chore(liveness): remove extraneous logging (#761) * fix: remove webgl dep * fix: fix breakpoint unit tests * fix: use main yarn.lock and set eslint config back to error on nullish coalescing * fix: fix import for react-countdown-circle-timer * fix: skip unit test for now, fix later * chore: update instruction texts from UX (#767) * Revert "chore: add debug logs and send them to sample app through hug events (#763)" This reverts commit 8e71060. * chore(react-liveness): add non-code files, setup jest and ts * chore(ui-react-liveness): migrate FaceLivenessDetector * fix snapshot * update FaceLivenessDetector imports * add top level react-liveness command * remove Liveness code from ui-react * add top level react-liveness command * chore: set opacity of camera view outside oval to 1.0 (#770) * port update from aws-amplify/amplify-ui-staging#770 * fix snapshot * fix: add missing react-liveness package to example app (#776) * chore: clean up + update liveness CSS class names (#764) * chore: flatten + cleanup classnames and styles * chore: change initial face distance threshold to be 0.34 (#778) * chore: change initial face distance threshold to be 0.34 * chore: move to constants folder and add comments explaining the numbers * chore: updated comment for more clarity * chore: try updating to .3 * add liveness quick start instructions and best practices * chore(liveness): add region prop to component (#779) * chore(liveness): add region prop to component * chore: update comment to not list out regions and instead add a link in future * chore(liveness): update threshold to 0.32 for testing * chore(liveness): text updates * chore: move instruction to top * chore(liveness): added border colors, adjusted colors for usability study * chore(liveness): add instrucitonal image * Update packages/react-liveness/src/components/FaceLivenessDetector/shared/Toast.tsx Co-authored-by: MJ Zhang <0618@users.noreply.github.com> * chore(liveness): update bullet index styling * chore: try adding test for mouseless navigation (#774) * chore: try adding test for keyboardless navigation * chore: updated to get begin check workign with enter button * chore(react-liveness): prepare for service code migration * chore: update instructional text * chore(ui-react-liveness): add styles exports * update quick start instructions and add credentials provider info * fix(ui-react-liveness): fix unit tests * remove console.warn spy * chore(ui-react-liveness): migrate service code * add customization info * Enable client debug logging (#789) * chore: re-add client logging to aid with session debugging * Fix copy/paste type in test to wrong videoRecorder path --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * feat: Add Match Progress Indicator for usability testing (#800) * fix: update 5s text to 8s (#801) * fix: set initial faceMatchPercentage to 25 to match formula (#802) Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: update api calls to match latest SDK changes (#803) * chore: update api calls to match latest SDK changes * chore: update to new version published in CA, rename OvalScaleFactors to OvalParameters --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * fix initial face timestamp by ensuring recorder has started (#798) * fix initial face timestamp by ensuring recoder has started * Add new timestamp calculation from Science * update comment * Use Math.round() on calculated timestamp --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: Add illustration components for start screen, minor default components refactor (#804) * Add illustration components for start screen * Fix test. Add background behind figure image to match illustration * Fix missing title tag * remove some unneeded imports * Add basic tests for new components * fix darkmode by using Text component around text elements --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: adding retry logic for loading CDN files * chore: update to use the retry logic from js core * fix: adding args * chore: update toasts during face matching to only show TOO_FAR, TOO_CLOSE toasts (#807) * Show progress indicator only when state matches too far * restrict when we show oval matching instructions to TOO_CLOSE, TOO_FAR, FACE_IDENTIFIED * Update toasts for matching state * update some comments, clean up code * revert changes in LivenessCameraModule, addressed in separate PR * add other face match state test scenarios during recording state --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * fix: only show match indicator during TOO_FAR, FACE_IDENTIFIED states (#808) Co-authored-by: Heather Buchel <buchel@amazon.com> * update android docs * chore: update face match timeout to 7 seconds (#811) * chore: update face match timeout to 7 seconds * update messaging to say 7 seconds instead of 8 --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: update illustration tests (#805) * chore: update tests for illustrations * add testId props for test * use nodeName and getByTitle * use findBy instead of queryBy --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * fix: move closer shouldn't show during MATCHED state (#810) Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: update copy from UX/copywriter (#812) * chore: copy updates * chore: more copy updates * fix oval instruction text --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * update dark mode theme colors * fix: error toast message should have dark text in darkmode (#813) Co-authored-by: Heather Buchel <buchel@amazon.com> * fix: use a css custom property for the match indicator transition (#814) Co-authored-by: Heather Buchel <buchel@amazon.com> * fix: uncenter illustrations (#815) Co-authored-by: Heather Buchel <buchel@amazon.com> * I18n updates (#787) * fix: overflow: hidden not working in safari on progress bar (#818) Co-authored-by: Heather Buchel <buchel@amazon.com> * Update test-internal-prs.yml (#819) * chore(ui-react-liveness): add size-limit test * remove extra file * update light mode theme colors * fix: 7 seconds string text (#823) Co-authored-by: Heather Buchel <buchel@amazon.com> * add link and update android versions * chore: remove client side logging (#826) Co-authored-by: Heather Buchel <buchel@amazon.com> * fix: Show Match indicator during CANT_IDENTIFY state during recording (#825) * fix: Show Match indicator during CANT_IDENTIFY state during recording * consolidate state check into showMatchIndicatorStates --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * add material3 dependency to setup * remove placeholder for theming image * test(liveness): add liveness build test (#824) * test(liveness): add nextjs build test * test(liveness): change build test nextjs size-limit * test(liveness): change version to latest * test(liveness): clean up code * test(liveness): clean up code * test(liveness): change version --------- Co-authored-by: MJ☔ <zhamujun@amazon.com> * add predictions dependency for liveness * chore: move error modal to default overridable component (#827) * chore: move error modal to default overridable component * chore: move error modal * chore: more fixes and string fixes * Update packages/react-liveness/src/components/FaceLivenessDetector/LivenessCheck/LivenessCameraModule.tsx Co-authored-by: Caleb Pollman <cpollman@amazon.com> --------- Co-authored-by: Caleb Pollman <cpollman@amazon.com> * remove instructions to add predictions plugin * add rekognition liveness links * remove unneeded dependencies * fix: remove liveness error modal from export (#835) * fix: remove liveness error modal from export * chore: remove more export, update snapshot * test(liveness): check CDN status (#830) * test(liveness): check CDN status * test(liveness): change CDN endpoint * test(liveness): add jsdelivr CDN back --------- Co-authored-by: MJ☔️ <zhamujun@amazon.comm> * chore(liveness): switch to public npm for client-rekognitionstreaming (#841) * chore: add tests for when MatchIndicator should show in LivenessCameraModule (#839) * chore: update tests for cases to show MatchIndicator * remove unnecesary import --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: add tests for MatchIndicator (#833) * chore: add tests for MatchIndicator * ensure progress bar can't be less than 0 or more than 100, add tests --------- Co-authored-by: Heather Buchel <buchel@amazon.com> * chore: update yarn.lock * chore: remove codeartifact and fix build issue * chore: fix docs and tests * chore: add social preview android/swift * chore: added web liveness quick start to docs (#834) * chore: added web liveness quick start to docs * Update docs/src/pages/[platform]/connected-components/liveness/quick-start-example.react.mdx Co-authored-by: Heather Buchel <hbuchel@gmail.com> * chore: merged with android branch * chore(liveness): add customization for react web * chore: update text * chore: added table of customizable components * Update docs/src/pages/[platform]/connected-components/liveness/quick-start-pull.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.i18n.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.start-view.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.theming.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.theming.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/quick-start-add.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/quick-start-add.react.mdx Co-authored-by: Scott Rees <6165315+reesscot@users.noreply.github.com> * chore: address comments * Update docs/src/pages/[platform]/connected-components/liveness/quick-start-pull.react.mdx * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.components.react.mdx Co-authored-by: Scott Rees <6165315+reesscot@users.noreply.github.com> * chore: address comments * Update docs/src/pages/[platform]/connected-components/liveness/quick-start-add.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.components.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/quick-start-add.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/quick-start-add.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * chore: address mor comments * chore: updated customization table with links * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.theming.react.mdx Co-authored-by: Scott Rees <6165315+reesscot@users.noreply.github.com> * chore: more comment updates --------- Co-authored-by: Heather Buchel <hbuchel@gmail.com> Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> Co-authored-by: Scott Rees <6165315+reesscot@users.noreply.github.com> * Liveness web docs examples (#847) * chore(liveness): added liveness example to start page * chore: added liveness docs examples * chore: fixed sidebar errors * Update docs/src/pages/[platform]/connected-components/liveness/customization/Customizationi18n.tsx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.components.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/customization/customization.theming.react.mdx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update docs/src/pages/[platform]/connected-components/liveness/QuickStartReact.tsx Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * chore: updated comments * address feedback --------- Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * chore: revert gh actions to match ui/main (#851) * chore(liveness): add iOS docs content (#837) * chore: update liveness streaming client (#855) * chore: update liveness streaming client * chore: remove CA install step * chore: remove extraneous github workflows from staging * chore: remove codeartifact gh action * chore: remove examples react-native changes * fix: clean up rn files and fix docs package.json * chore: add liveness to pr unit tests * fix(docs): hide install script for swift, fix install scripts format * fix(liveness): fix unit tests build for react-liveness * chore: add liveness e2e setup * chore: setup liveness e2e tests to run * chore: regenerate yarn.lock from main * Update packages/react-core/src/Authenticator/hooks/useAuthenticator/constants.ts Co-authored-by: Caleb Pollman <cpollman@amazon.com> * Update packages/react-core/src/Authenticator/hooks/useAuthenticator/constants.ts Co-authored-by: Caleb Pollman <cpollman@amazon.com> * chore: remove auth mock file * Create .changeset/brown-moles-grow.md * Update .changeset/brown-moles-grow.md Co-authored-by: Caleb Pollman <cpollman@amazon.com> * fix: workflow runs * chore: try fix e2e again * fix: docs snapshot * Update .github/workflows/reusable-e2e.yml Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * Update .github/workflows/reusable-e2e.yml Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> * chore: remove liveness env readme * chore: fix folder structure for liveness environment (#3679) * Move liveness backend to a nested folder * Remove unrelated file * Use us-east-1 * Update environments/liveness/package.json * chore(docs): switch all instaces of all to be angular|flutter|react|react-native|vue since we added android and swift * fix: remove unused livenessCard * fix: update docs snapshot test * fix: fix path for liveness env in next app * fix: also fix export in other places * fix: broken liveness e2e tests * fix: fix liveness e2e env * chore(docs): remove more default pages for android and swift * chore: switch liveness next app to use prod endpoint * chore: switch endpoint * chore: fix region for liveness creds * chore: fix more pr feedback * chore: pin version * update swift customization docs * Temporarily skip link checker * Revert "test(liveness): add liveness build test (#824)" This reverts commit 2e0a816. * chore: skip liveness canary * chore: fix ios docs build error --------- Co-authored-by: MJ Zhang <0618@users.noreply.github.com> Co-authored-by: MJ☔ <zhamujun@amazon.com> Co-authored-by: Heather Buchel <hbuchel@gmail.com> Co-authored-by: Heather Buchel <buchel@amazon.com> Co-authored-by: William Lee <43682783+wlee221@users.noreply.github.com> Co-authored-by: Ioana Brooks <68251134+ioanabrooks@users.noreply.github.com> Co-authored-by: Caleb Pollman <cpollman@amazon.com> Co-authored-by: Erica Eaton <eatoeric@amazon.com> Co-authored-by: Chenwei Zhang <40295569+zchenwei@users.noreply.github.com> Co-authored-by: Ayush Goyal <ayush987goyal@gmail.com> Co-authored-by: Chenwei Zhang <chenwz@amazon.com> Co-authored-by: wlee221 <willeea@amazon.com> Co-authored-by: Erica Eaton <67125657+eeatonaws@users.noreply.github.com> Co-authored-by: MJ☔️ <zhamujun@amazon.comm> Co-authored-by: Scott Rees <6165315+reesscot@users.noreply.github.com> Co-authored-by: Ian Saultz <52051793+atierian@users.noreply.github.com>
- Loading branch information
17 people
committed
Apr 10, 2023
1 parent
c3918d9
commit bc3fd6d
