StorageBrowser RFC

[reesscot]

Overview

StorageBrowser for Amazon S3 is an open source Amplify UI React component customers can add to their web applications to provide end-users with a simple interface for data stored in Amazon S3. Using StorageBrowser for S3, customers provide authorized end-users access to easily browse, download, and upload data in Amazon S3 from their own applications. StorageBrowser for S3 only displays the data that your end-users are authorized to access and automatically optimizes requests to deliver high throughput data transfer. You can control access to your data based on your end-user’s identity using AWS security and identity services or your own managed services.

StorageBrowser for S3 is currently in developer preview while we collect your feedback before making the component generally available. The API might change while in developer preview so we do not recommend using it in production until it is generally available. We will post updates to this RFC as we make changes to the component.

StorageBrowser for S3 has 3 main views:

1. Locations view: the initial view for users that show the root level S3 resources they have access to and their permissions (READ/READWRITE) for each S3 location. An S3 location is an S3 bucket or prefix, which could be defined by Amazon S3 Access Grants or defined manually.
2. Location detail view: a file-browser-like interface where users can browse files and folders in S3 and upload/download files
3. Location Action view: this view appears when users select an action like uploading files

1 Installation #

Fork the example repo:
https://github.com/reesscot/nextjs-storage-browser

OR:

You can install StorageBrowser for S3 from tagged versions of the @aws-amplify/ui-react-storage and aws-amplify packages. Add these dependencies to your package.json file:

  "dependencies": {
    "@aws-amplify/ui-react-storage": "storage-browser",
    "@aws-amplify/storage": "storage-browser",
    "aws-amplify": "storage-browser",
  }

Or add them via npm:

npm i --save @aws-amplify/ui-react-storage@storage-browser aws-amplify@storage-browser @aws-amplify/storage@storage-browser

We will publish StorageBrowser for S3 to the latest version of these packages once it is generally available.

2 Setting up authentication and authorization #

In order to show S3 locations and their contents to end users, you first need to set up your preferred authentication and authorization methods. There are 3 ways you can set up authentication/authorization with the storage browser component:

1. Amplify auth: If you are already using Amplify then this option lets you get started the fastest. It uses Amazon Cognito for authentication and IAM policies for authorization and access. And by using Amplify Gen 2, the access rules for users and groups can be customized.
2. AWS IAM Identity Center and S3 Access Grants: We recommend this option if you want to grant access on a per-S3-prefix basis to both IAM principals and directly to users or groups from your corporate directory. With S3 Access Grants capabilities, applications can request data from Amazon S3 on behalf of the current authenticated user. This means your applications no longer need to first map the user to an IAM principal. And when you use S3 Access Grants with IAM Identity Center trusted identity propagation, each AWS CloudTrail data event for S3 references the end user identity that accessed your data.
3. Custom auth: We recommend this option if you have your own identity and authorization service for authenticating and authorizing users in your application. To use this option, you will need to provide the list of S3 locations to display to the user and a mechanism for fetching scoped credentials for each location.

Amplify auth

Make sure you have an Amplify project started, by following the getting started guides: https://docs.amplify.aws/react/start/quickstart/. Then create an S3 bucket with access rules by defining a storage resource and adding authorization rules https://docs.amplify.aws/react/build-a-backend/storage/authorization/:

// amplify/storage/resource.ts

export const storage = defineStorage({
  name: 'myProjectFiles',
  access: (allow) => ({
    'public/*': [
      allow.guest.to(['read']),
      allow.authenticated.to(['read', 'write', 'delete']),
    ],
    'protected/{entity_id}/*': [
      allow.authenticated.to(['read']),
      allow.entity('identity').to(['read', 'write', 'delete'])
    ],
    'private/{entity_id}/*': [
      allow.entity('identity').to(['read', 'write', 'delete'])
    ]
  })
});

In your frontend code, create the StorageBrowser component with the Amplify Auth adapter:

import {
  createStorageBrowser,
  createAmplifyAuthAdapter,
} from '@aws-amplify/ui-react-storage/browser';
import '@aws-amplify/ui-react-storage/storage-browser-styles.css';

// these should match access patterns defined in amplify/storage/resource.ts
const defaultPrefixes = [
  'public/',
  (identityId: string) => `protected/${identityId}/`,
  (identityId: string) => `private/${identityId}/`,
];

export const { StorageBrowser } = createStorageBrowser({
  elements: elementsDefault, // replace to customize your UI
  config: createAmplifyAuthAdapter({
    options: {
      defaultPrefixes
    },
  }),
});

Or if you want to use Amplify Auth with prebuilt Amplify UI you can just import the default StorageBrowser component and render it:

import { StorageBrowser } from '@aws-amplify/ui-react-storage';

// these should match access patterns defined in amplify/storage/resource.ts
const defaultPrefixes = [
  'public/',
  (identityId: string) => `protected/${identityId}/`,
  (identityId: string) => `private/${identityId}/`,
];

export default function Example() {
  return (
    <StorageBrowser defaultPrefixes={defaultPrefixes} />
  )
}

AWS Identity Center and S3 Access Grants

1. Enable IAM Identity Center for your organization if you haven’t already.
2. Create a Trusted Token Issuer in IAM Identity Center. This construct will represent your external Identity Provider, like a Microsoft Entra ID tenant, within IAM Identity Center, enabling it to recognize identity tokens for your application’s authenticated users.
3. Create an Application in IAM Identity Center to represent your application as it interacts with IAM Identity Center on behalf of authenticated users.
4. Connect your Application to the Trusted Token Issuer so that your application will able to present identity tokens for authenticated users to IAM Identity Center.
5. Connect your Application to S3 Access Grants so that it can make requests to S3 Access Grants on behalf of authenticated users.
6. Associate your Application in Identity Center with the AWS account in which it will run. This step will allow your application to make the necessary requests to its corresponding IAM Identity Center Application.

For detailed setup instructions see this blog post: https://aws.amazon.com/blogs/storage/how-to-develop-a-user-facing-data-application-with-iam-identity-center-and-s3-access-grants/

Once you have IAM Identity Center and S3 Access Grants set up, in your React application you can use createManagedAuthAdapter to set up the auth rules and createStorageBrowser to create the StorageBrowser component:

import {
    createManagedAuthAdapter,
    createStorageBrowser,
    elementsDefault
} from '@aws-amplify/ui-react-storage/browser';
import '@aws-amplify/ui-react-storage/storage-browser-styles.css';

export const { StorageBrowser } = createStorageBrowser({
   elements: elementsDefault, // replace to customize your UI
   config: createManagedAuthAdapter({
        credentialsProvider: async () => {
            // return credentials object
        },
        // AWS `region` and `accountId`
        region: '',
        accountId: '',
        // call `onAuthStateChange` when end user auth state changes 
        // to clear sensitive data from the `StorageBrowser` state
        registerAuthListener: (onAuthStateChange) => {},
   })
});

Go to step 3

Custom authentication and authorization

You can also use StorageBrowser for S3 with your own authentication and authorization setup. Rather than calling createManagedAuthAdapter or createAmplifyAuthAdapter you can directly configure how StorageBrowser for S3 lists S3 locations and gets credentials for each location:

import { createStorageBrowser, elementsDefault } from '@aws-amplify/ui-react-storage/browser';
import '@aws-amplify/ui-react-storage/storage-browser-styles.css';

export const { StorageBrowser } = createStorageBrowser({
   elements: elementsDefault, // replace to customize your UI
   config: {
    listLocations: async (input = {}) => {
      const { nextToken, pageSize } = input;
      return {
        locations: [
          {
            bucketName: '[bucket name]',
            key: '', 
            region: 'us-east-1',
            type: 'BUCKET',
            permission: 'READWRITE',
            scope: 's3://[bucket name]',
          }
        ]
      }
    },
    getLocationCredentials: async ({ scope, permission }) => {
      // get credentials for specified scope and permission
      return {
        credentials: {
          accessKeyId: '',
          secretAccessKey: '',
          sessionToken: '',
          expiration: new Date(),
        }
      }
    },
    region: '',
    registerAuthListener: (onStateChange) => {

    }
  },
})

3 Rendering the StorageBrowser #

Now that you have created the StorageBrowser component using one of the 3 auth methods above, you can place the component into your app:

export default function Page() {
  return (
    <StorageBrowser />
  )
}

4 Customizing the StorageBrowser #

Composing

StorageBrowser for S3 has 3 main views that can be composed into your application if you want to move around parts of the component:

• LocationsView: initial screen showing a table with all S3 locations a user has access to
• LocationDetailView: file browsing view once the user has selected an S3 location
• LocationActionView: view rendered when the user initiates an action like uploading files

For example, to display the LocationsView (locations table) side-by-side with the LocationDetailView (file browser table) with the LocationActionView in a dialog:

const input = {} // `createStorageBrowser` input values
const { StorageBrowser, useControl } = createStorageBrowser(input);

function MyStorageBrowser() {
  const [{ selected }] = useControl('LOCATION_ACTIONS');

  return (
    <Flex>
      <Flex direction={'row'}>
        <StorageBrowser.LocationsView />
        <StorageBrowser.LocationDetailView />
      </Flex>
      {selected.type ? (
        <dialog open>
          <StorageBrowser.LocationActionView />
        </dialog>
      ) : null}
    </Flex>
  );
}

export default function App() {
  return (
    <StorageBrowser.Provider>
      <MyStorageBrowser />
    </StorageBrowser.Provider>
  )
}

Using a custom UI

createStorageBrowser improves bundle size by shipping without Amplify UI React Primitives, rendering basic HTML elements by default with only layout styling applied.

You can control all of the elements used in the StorageBrowser component by passing in elements to the createStorageBrowser function. Elements have a variant prop that is used to signify where that element is being used within the component tree of the StorageBrowser. For example, the Button element is used in many different places in the StorageBrowser UI, variant allows you control over how your button looks and behaves in different usage scenarios:

import { Button } from '@chakra-ui/react';
import { createStorageBrowser } from '@aws-amplify/ui-react-storage';
// layout styles
import '@aws-amplify/ui-react-storage/storage-browser-styles.css';

function MyButton({ variant, ...props }) {
  if (variant === 'table-data') {
    // return chakra ui `Button` link
    return <Button {...props} variant="link" />
  }
  return <Button {...props} />
}

const { StorageBrowser } = createStorageBrowser({
  elements: {
    Button: MyButton
  },
  //...
});

Here are all the elements that you can override:

• Button
• DefinitionList
• DefinitionTerm
• DefinitionDetail
• Heading
• Icon
• Input
• Label
• ListItem
• Nav
• OrderedList
• ProgressBar
• Span
• Table
• TableBody
• TableData
• TableHead
• TableHeader
• TableRow
• Text
• UnorderedList
• View

---

Feedback for dev preview #

We are looking for any and all feedback on StorageBrowser for S3. We would love to hear from you about bugs, aspects about the developer experience and APIs that could be improved, additional features and functionality you need, and more. We plan on addressing all of your feedback as it comes in.

Thank you to everyone that made this launch possible: @dbanksdesign @reesscot @calebpollman @timngyn @esauerbo @hbuchel @AllanZhengYP @jimblanc @ashwinkumar6 @Jordan-Nelson @haverchuck

[franck-chester]

looks awesome. I'm now desperate to see examples of validation / feedback of the upload. for example, after uploading a CSV, a table listing the rows, together with valid/invalid.

[soubhik]

Very nice, would be even more awesome if a simple search functionality based on s3 prefixes were added.

[tttol]

Awesome!
I created a sample application using the Storage Browser from an Amplify application in my local environment. However, there's one thing that's bothering me.

The attached image is a screenshot of my Amplify application.

In my environment, there’s a directory structure like this: Home/amplify-storagebrowserexa-storagebrowserexamplebuc-xxx/public/blog/2023. In this case, I feel the breadcrumb should naturally display like this:

Home /
amplify-storagebrowserexa-storagebrowserexamplebuc-xxx /
public /
blog /
2023

However, it’s currently being displayed like this, with public/ shown directly after amplify-storagebrowserexa-storagebrowserexamplebuc-xxx/:

Home /
amplify-storagebrowserexa-storagebrowserexamplebuc-xxx/public /
blog /
2023

Why is public being displayed on the same line as amplify-storagebrowserexa-storagebrowserexamplebuc-xxx?

[mkdev10]

Great feature, very impressed‼️Quick UI construction is amazing🚀
Feature requests:

• File search
• File and Folder deletion

Looking forward to the release!

[m-khan82]

Just set up the Storage Browser in my environment, and I'm loving it! a couple of questions:

1. Is it possible to have access to the bucket root level ? If not, is there a way to enable access to the bucket root level?

2. How can I update the amplifyconfiguration.json file to have access to multiple buckets? I am using Amplify Auth for Authorization

[naveeng2402]

Awesome package, can this use cdn(cloudfront) for file browsing and downloading and keep the bucket read access only on cdn and do we have multi bucket support

[vineetkala11]

Hello @reesscot ,

Great feature! Could you please provide an approximate target date for the release of the production-ready version?

[IVIURRAY]

Really nice RFC 👍 Looking forward to it!

Feature Request:

• Ability to peek into a S3 file / preview a files contents from the UI- (something like select * from s3_file limit 20) to get a feel for the contents of a file.

[scorobogaci]

Nice, looks super exciting
Few questions or feature requests

1. Does it work well with React Native as well ?
2. Does it support storage class for S3 (like it is configurable to which storage class the object to be uploaded)? Reference : https://aws.amazon.com/s3/storage-classes/
3. Any plans to support also delete operation (why not having all CRUD operations) ?

Thanks :)

[GowthamShanmugam]

AWS S3 SDK provides an "endpoint" parameter, can we configure that with this new UI browser ?

s3Client = new S3Client({
     region: '',
     endpoint: //s3 endpoint,
     credentials: {
        accessKeyId: //accessKeyId,
       secretAccessKey: //secretAccessKey,
   },
   });
}

[scorobogaci]

Hello, I have installed the StorageBrowser using
npm i --save @aws-amplify/ui-react-storage@storage-browser aws-amplify@storage-browser
which updated package.json like this

"@aws-amplify/ui-react-storage": "^0.0.0-storage-browser-670d987-20240905191734",
"aws-amplify": "^6.6.1-storage-browser.4949269.0",

However, not sure what these versions means, but the UI looks completely broken, including pagination, Breadcrumbs etc, see attached screenshots

The option suggested in the RFC above

  "dependencies": {
    "@aws-amplify/ui-react-storage": "storage-browser",
    "aws-amplify": "storage-browser",
  }

does not work at all, results in net::ERR_ABORTED 504 (Outdated Optimize Dep)
or the app is not building at all, failing with

✘ [ERROR] Missing "./storage-browser" specifier in "@aws-amplify/storage" package [plugin vite:dep-pre-bundle]

    node_modules/@aws-amplify/ui-react-storage/dist/esm/components/StorageBrowser/context/useGetCredentialsProvider.mjs:2:47:
      2 │ import { createLocationCredentialsStore } from '@aws-amplify/storage/storage-browser';
        ╵                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  This error came from the "onResolve" callback registered here:

    node_modules/vite/node_modules/esbuild/lib/main.js:1150:20:

Which version to use for latest UI that behaves like in the description of this RFC ?

Thank you

[scorobogaci]

Feedback !
Is there any benefit on presenting the folders like this

1. Why displaying the bucket name ? (For end users is this really useful for something ? )
2. While creating the folders with name like this eu-central-1:a078238c-b118-c091-b6b0-96f42f492fca it is really ugly, end users do not care about metadata (i guess that id uuid in there is the mapping with cognito user), but why displaying it ?
3. The eu-central-1:a078238c-b118-c091-b6b0-96f42f492fca as an example, is it some attribute on the user on Cognito (or is just some mapping that internally amplify uses ? )

Thank you

[scorobogaci]

Need some help guys, not able to run it :(
package.json

{
  "name": "amplify-vite-react-template",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "tsc && vite build",
    "lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
    "preview": "vite preview"
  },
  "dependencies": {
    "@aws-amplify/ui-react": "^6.2.0",
    "@aws-amplify/ui-react-storage": "storage-browser",
    "aws-amplify": "storage-browser",
    "react": "^18.2.0",
    "react-dom": "^18.2.0"
  },
  "devDependencies": {
    "@aws-amplify/backend": "^1.1.1",
    "@aws-amplify/backend-cli": "^1.2.4",
    "@types/react": "^18.2.66",
    "@types/react-dom": "^18.2.22",
    "@typescript-eslint/eslint-plugin": "^7.2.0",
    "@typescript-eslint/parser": "^7.2.0",
    "@vitejs/plugin-react": "^4.2.1",
    "aws-cdk": "^2.138.0",
    "aws-cdk-lib": "^2.138.0",
    "constructs": "^10.3.0",
    "esbuild": "^0.20.2",
    "eslint": "^8.57.0",
    "eslint-plugin-react-hooks": "^4.6.0",
    "eslint-plugin-react-refresh": "^0.4.6",
    "tsx": "^4.7.2",
    "typescript": "^5.4.5",
    "vite": "^5.2.0"
  }
}

Usage

import { Authenticator } from '@aws-amplify/ui-react'
import '@aws-amplify/ui-react/styles.css'
import {StorageBrowser} from "@aws-amplify/ui-react-storage";

function App() {

  const defaultPrefixes = [
    'public/',
    (identityId: string) => `protected/${identityId}/`,
    (identityId: string) => `private/${identityId}/`,
  ];

  return (
      <Authenticator>
        {({signOut, user}) => (
            <>
              <h1>{user?.signInDetails?.loginId}</h1>
              <StorageBrowser defaultPrefixes={defaultPrefixes}></StorageBrowser>
              <button onClick={signOut}>Sign out</button>
            </>
        )}
      </Authenticator>
  );
}

export default App;

amplify/storage/resource.ts

import {defineStorage} from "@aws-amplify/backend";

export const storage = defineStorage({
  name: 'myProjectFiles',
  access: (allow) => ({
    'public/*': [
      allow.guest.to(['read']),
      allow.authenticated.to(['read', 'write', 'delete']),
    ],
    'protected/{entity_id}/*': [
      allow.authenticated.to(['read']),
      allow.entity('identity').to(['read', 'write', 'delete'])
    ],
    'private/{entity_id}/*': [
      allow.entity('identity').to(['read', 'write', 'delete'])
    ]
  })
});

What I am missing ? What I am doing wrong ? Getting this error always when attempting to build/run the project :(

✘ [ERROR] Missing "./storage-browser" specifier in "@aws-amplify/storage" package [plugin vite:dep-pre-bundle]

    node_modules/vite/node_modules/esbuild/lib/main.js:1225:21:
      1225 │         let result = await callback({
           ╵                      ^

    at e (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:45978:25)
    at n (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:45978:627)
    at o (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:45978:1297)
    at resolveExportsOrImports (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:46599:18)
    at resolveDeepImport (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:46622:25)
    at tryNodeResolve (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:46387:16)
    at ResolveIdContext.resolveId (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:46137:19)
    at PluginContainer.resolveId (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:48952:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:66208:15
    at async file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:46888:28
    at async requestCallbacks.on-resolve (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:1225:22)
    at async handleRequest (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:647:11)

  This error came from the "onResolve" callback registered here:

    node_modules/vite/node_modules/esbuild/lib/main.js:1150:20:
      1150 │       let promise = setup({
           ╵                     ^

    at setup (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:46871:13)
    at handlePlugins (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:1150:21)
    at buildOrContextImpl (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:873:5)
    at Object.buildOrContext (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:699:5)
    at /Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:2032:68
    at new Promise (<anonymous>)
    at Object.context (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:2032:27)
    at Object.context (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:1874:58)
    at prepareEsbuildOptimizerRun (file:///Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/dist/node/chunks/dep-DyBnyoVI.js:50861:33)

  The plugin "vite:dep-pre-bundle" was triggered by this import

    node_modules/@aws-amplify/ui-react-storage/dist/esm/components/StorageBrowser/context/useGetCredentialsProvider.mjs:2:47:
      2 │ import { createLocationCredentialsStore } from '@aws-amplify/storage/storage-browser';
        ╵                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

7:44:04 PM [vite] error while updating dependencies:
Error: Build failed with 1 error:
node_modules/vite/node_modules/esbuild/lib/main.js:1225:21: ERROR: [plugin: vite:dep-pre-bundle] Missing "./storage-browser" specifier in "@aws-amplify/storage" package
    at failureErrorWithLog (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:1472:15)
    at /Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:945:25
    at /Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:1353:9
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Is this an error with vite ?

Error: Build failed with 1 error:
node_modules/@aws-amplify/ui-react-storage/dist/esm/components/StorageBrowser/context/useGetCredentialsProvider.mjs:2:47: ERROR: [plugin: vite:dep-pre-bundle] Missing "./storage-browser" specifier in "@aws-amplify/storage" package
    at failureErrorWithLog (/Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:1472:15)
    at /Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:945:25
    at /Users/ion/time-capsule/repository/amplify-vite-react-template/node_modules/vite/node_modules/esbuild/lib/main.js:1353:9
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  errors: [Getter/Setter],
  warnings: [Getter/Setter]
}

[scorobogaci]

@reesscot , do you have somewhere published a working version/project/template of this RFC please ?
Thanks a lot, versioning hell or whatever, just makes it really hard to test/play with this feature :(

[reesscot]

@scorobogaci It looks like you're getting the wrong versions of the dependencies somehow, can you try forcing the storage-browser version of the @aws-amplify/storage package:

npm install @aws-amplify/storage@storage-browser

Alternatively, you could try this example repo which we've been testing with on Next.js: https://github.com/reesscot/nextjs-storage-browser

[reesscot]

@tttol @scorobogaci In regards to the styling questions, can you tell me what order you are imported the CSS in? Also, do you have any other CSS styling that might be conflicting in your app?

[scorobogaci]

@scorobogaci It looks like you're getting the wrong versions of the dependencies somehow, can you try forcing the storage-browser version of the @aws-amplify/storage package:

npm install @aws-amplify/storage@storage-browser

Alternatively, you could try this example repo which we've been testing with on Next.js: https://github.com/timngyn/nextjs-storage-browser/tree/main

Perfect, thanks a lot @reesscot , explicitly adding @aws-amplify/storage@storage-browser fixed the build, and not the app is building as expected. Thanks a lot

[scorobogaci]

import '@aws-amplify/ui-react-storage/storage-browser-styles.css';

thanks for the hint @reesscot . I have imported the styles in the component directly import '@aws-amplify/ui-react-storage/storage-browser-styles.css'; and all works as in the description of the RFC :)

Final simple component looks like this and works fine 👍

import { Authenticator } from '@aws-amplify/ui-react'
import '@aws-amplify/ui-react/styles.css'
import '@aws-amplify/ui-react-storage/storage-browser-styles.css';
import {StorageBrowser} from "@aws-amplify/ui-react-storage";

function App() {

  const defaultPrefixes = [
    'public/',
    (identityId: string) => `protected/${identityId}/`,
    (identityId: string) => `private/${identityId}/`,
  ];

  return (
      <Authenticator>
        {({signOut, user}) => (
            <>
              <h1>Hi {user?.signInDetails?.loginId}</h1>
              <StorageBrowser defaultPrefixes={defaultPrefixes}></StorageBrowser>
              <button onClick={signOut}>Sign out</button>
            </>
        )}
      </Authenticator>
  );
}

export default App;

[scorobogaci]

Why when using createStorageBrowser getting such error ?

Reference :

import {
    createStorageBrowser,
    createAmplifyAuthAdapter, elementsDefault,
} from '@aws-amplify/ui-react-storage/browser';
import '@aws-amplify/ui-react-storage/storage-browser-styles.css';

const defaultPrefixes = [
    (identityId: string) => `files/${identityId}/`,
];

export const { StorageBrowser,useControl } = createStorageBrowser({
    elements: elementsDefault,
    config: createAmplifyAuthAdapter({
        options: {
            defaultPrefixes
        },
    }),
});

Amplify.configure(outputs); is being invoked in main.tsx

import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App.tsx";
import { Amplify } from "aws-amplify";
import outputs from "../amplify_outputs.json";

Amplify.configure(outputs);

ReactDOM.createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

Also, amplify_outputs.json has been updated with the json downloaded from Amplify Console

[scorobogaci]

https://github.com/timngyn/nextjs-storage-browser/tree/main

Sample is not reachable, 404

[reesscot]

@scorobogaci Sorry about that, I just made this example repo and was able to get it running and deployed to Amplify Hosting.
https://github.com/reesscot/nextjs-storage-browser

[scorobogaci]

@reesscot , I see you've edited a bit the setup by adding

  "dependencies": {
    "@aws-amplify/ui-react-storage": "storage-browser",
    "@aws-amplify/storage": "storage-browser",
    "aws-amplify": "storage-browser",
  }

explicitly @aws-amplify/storage just so you know adding this package has serious issue with generation of the amplify_outputs.json when using CI of Amplify Gen 2. After you add the storage definition and do the deployment, the amplify_output.json is not getting updated with the storage (only after a day it does) :(

Man, can you please review this setup and this tag/RFC so that is work with other components of the Amplify please ? It is becoming a nightmare already

[reesscot]

@scorobogaci Can you please try out the example repo app?
https://github.com/reesscot/nextjs-storage-browser
I tested it this morning and was able to fully deploy it to Amplify Hosting. It includes a package-lock.json file will will ensure you get the correct dependencies to test out the Storage Browser.

The @aws-amplify/storage package will have no effect on the amplify_outputs.json file. It's just a client side JS library and does not generate backend resources. The @aws-amplify/backend-cli

What issues are you having with deploying changes? When I've had issues with deploying changes to the storage definitions, it's usually been a missing command in the amplify.yml file. Specifically check to make sure you have this line: https://github.com/reesscot/nextjs-storage-browser/blob/main/amplify.yml#L7

[scorobogaci]

@scorobogaci Can you please try out the example repo app? https://github.com/reesscot/nextjs-storage-browser I tested it this morning and was able to fully deploy it to Amplify Hosting. It includes a package-lock.json file will will ensure you get the correct dependencies to test out the Storage Browser.

The @aws-amplify/storage package will have no effect on the amplify_outputs.json file. It's just a client side JS library and does not generate backend resources. The @aws-amplify/backend-cli

What issues are you having with deploying changes? When I've had issues with deploying changes to the storage definitions, it's usually been a missing command in the amplify.yml file. Specifically check to make sure you have this line: https://github.com/reesscot/nextjs-storage-browser/blob/main/amplify.yml#L7

Alright mate, the issue is ... once you add the package "@aws-amplify/storage": "storage-browser" breaks the generation of the amplify_outputs.json, the storage definition won't be added after deployment once "@aws-amplify/storage": "storage-browser" is added. More than that, by breaking this thing, even data (amplify client) won't generate anymore :(
Please validate it, there are incompatibilities with "@aws-amplify/storage": "storage-browser", validate it please.
So, once this package is added, the storage definition for example

export const storage = defineStorage({
  name: 'myProjectFiles',
  access: (allow) => ({
    'public/*': [
      allow.guest.to(['read']),
      allow.authenticated.to(['read', 'write', 'delete']),
    ],
    'protected/{entity_id}/*': [
      allow.authenticated.to(['read']),
      allow.entity('identity').to(['read', 'write', 'delete'])
    ],
    'private/{entity_id}/*': [
      allow.entity('identity').to(['read', 'write', 'delete'])
    ]
  })
});

won't be added to amplify_outputs.json

[scorobogaci]

@scorobogaci Can you please try out the example repo app? https://github.com/reesscot/nextjs-storage-browser I tested it this morning and was able to fully deploy it to Amplify Hosting. It includes a package-lock.json file will will ensure you get the correct dependencies to test out the Storage Browser.
The @aws-amplify/storage package will have no effect on the amplify_outputs.json file. It's just a client side JS library and does not generate backend resources. The @aws-amplify/backend-cli
What issues are you having with deploying changes? When I've had issues with deploying changes to the storage definitions, it's usually been a missing command in the amplify.yml file. Specifically check to make sure you have this line: https://github.com/reesscot/nextjs-storage-browser/blob/main/amplify.yml#L7

Alright mate, the issue is ... once you add the package "@aws-amplify/storage": "storage-browser" breaks the generation of the amplify_outputs.json, the storage definition won't be added after deployment once "@aws-amplify/storage": "storage-browser" is added. More than that, by breaking this thing, even data (amplify client) won't generate anymore :( Please validate it, there are incompatibilities with "@aws-amplify/storage": "storage-browser", validate it please. So, once this package is added, the storage definition for example

export const storage = defineStorage({
  name: 'myProjectFiles',
  access: (allow) => ({
    'public/*': [
      allow.guest.to(['read']),
      allow.authenticated.to(['read', 'write', 'delete']),
    ],
    'protected/{entity_id}/*': [
      allow.authenticated.to(['read']),
      allow.entity('identity').to(['read', 'write', 'delete'])
    ],
    'private/{entity_id}/*': [
      allow.entity('identity').to(['read', 'write', 'delete'])
    ]
  })
});

won't be added to amplify_outputs.json

ignore this please, I forgot to add storage do backend definition, ignore me please. Sorry, will re-try now

[ghost]

Keep up the GOOD work @reesscot

[VishnuM17]

@reesscot Thanks for all the support. I have a doubt, is it possible to use this component to build an app that serves the purpose of an S3 browser where users can access multiple buckets based on their access.

[koiker]

@reesscot Thanks for all the support. I have a doubt, is it possible to use this component to build an app that serves the purpose of an S3 browser where users can access multiple buckets based on their access.

Yes, you can use S3 Access Grants to create grants for users and groups. The component will list those buckets or prefixes as locations and the user will be able to navigate them.

[OSobky]

@reesscot & @koiker thank you for this needed component!

I would request to enable also for CEPH buckets to be used with your opensource component. This will require enable custom endpoints & force pathstyle. I see you already have this enabled for internal usage/development, would be great it is also enabled for StorgaBrowser component.

[ineplyueva]

Is it possible to deploy storage-browser self-hosted and connect to my ceph cluster or is only Amazon services required?

[OSobky]

@ineplyueva , yes it would be possible if it is enabled through the component configurations (Custom authentication and authorization). Currently the component configs doesn't accept customEndpoint or forcePathStyle. As this functionality is only available for internal development.

Hey @reesscot & @koiker, any feedback or comments on this would be highly appreciated.

[mhoroschun]

@reesscot Thanks for all the support. I have a doubt, is it possible to use this component to build an app that serves the purpose of an S3 browser where users can access multiple buckets based on their access.

Yes, you can use S3 Access Grants to create grants for users and groups. The component will list those buckets or prefixes as locations and the user will be able to navigate them.

Are there any examples of how to make this work? Getting S3 Access Grants set up is OK, but then once that's done there seems like there is quite a bit of logic required to do the token exchange? The example above says...

    credentialsProvider: async () => {
        // return credentials object
    },

[NBIX-Jingtao-Liu]

Do you have any plan to support vuejs soon?

[SyoujirouAbe]

Could someone tell how to use Storage Browser for S3 for an S3 bucket that was not created with Amplify?
Authentication for my app is handled using Amplify UI's Auth (Cognito).

[KaranamLokesh]

Could someone tell how to use Storage Browser for S3 for an S3 bucket that was not created with Amplify? Authentication for my app is handled using Amplify UI's Auth (Cognito).

I added storage using backend for the bucket i have created without Amplify, followed this doc to enable CORS on my bucket https://sto-bro-docs.ddzh7dd1jott2.amplifyapp.com/react/connected-components/storage/storage-browser
const backend = defineBackend({
auth
});

backend.addOutput({
storage: {
aws_region: "region",
bucket_name: "bucket",
},
});

[KaranamLokesh]

I'm using amplify auth and custom bucket for setting up storage browser, I'm trying to use multiple buckets. I have tried setting the storage using backend, getting only the first bucket

backend.addOutput({
  storage: {
    aws_region: "region",
    bucket_name: "bucket1",
    buckets: [
      {
        name: "test-bucket", 
        bucket_name: "bucket1", 
        aws_region: "region" 
      },
      {
        name: "test-bucket-2",
        bucket_name: "bucket2",
        aws_region: "region"
      }
    ]
  },
});

I also tried to add storage without using amplify backend, in the amplify configuration, but only getting the first bucket

import { Amplify } from "aws-amplify"

Amplify.configure({
  Auth: {
    Cognito: {
      userPoolId: "<your-cognito-user-pool-id>",
      userPoolClientId: "<your-cognito-user-pool-client-id>",
      identityPoolId: "<your-cognito-identity-pool-id>",
      loginWith: {
        email: true,
      },
      signUpVerificationMethod: "code",
      userAttributes: {
        email: {
          required: true,
        },
      },
      allowGuestAccess: true,
      passwordFormat: {
        minLength: 8,
        requireLowercase: true,
        requireUppercase: true,
        requireNumbers: true,
        requireSpecialCharacters: true,
      },
    },
  },
  storage: {
    aws_region: "region",
    bucket_name: "bucket1",
    buckets: [
      {
        name: "test-bucket", 
        bucket_name: "bucket1", 
        aws_region: "region" 
      },
      {
        name: "test-bucket-2",
        bucket_name: "bucket2",
        aws_region: "region"
      }
    ]
  },
})

Can someone point me on how to use storage browser with multiple custom buckets?

Thanks

[ashwinkumar6]

@KaranamLokesh, yes this is possible.

1. Configuring multiple buckets on storage browser using gen2:
Adding multi-bucket using latest gen2 produces amplify_outputs.json in the following format.

  "storage": {
    "aws_region": "region",
    "bucket_name": "bucket1",
    "buckets": [
      {
        "name": "test-bucket1",
        "bucket_name": "bucket1",
        "aws_region": "region",
        "paths": {
          "public/*": {
            "guest": ["get", "list", "write", "delete"],
            "authenticated": ["write", "get", "list", "delete"]
          }
        }
      },
      {
        "name": "test-bucket2",
        "bucket_name": "bucket1",
        "aws_region": "region",
        "paths": {
          "public/*": {
            "guest": ["get", "list", "write", "delete"],
            "authenticated": ["write", "get", "list", "delete"]
          }
        }
      }
    ]
  },

You can follow the existing Add multiple storage buckets to your app using AWS Amplify doc with no additional change to generate the above.

You would notice a new paths key, this is required for amplify auth

2. Configuring multiple buckets on storage browser (buckets created outside gen2):
The custom bucket needs to be configured in the same format as above, i.e with paths.

example:

import amplifyconfig from "./amplify_outputs.json";
Amplify.configure(amplifyconfig);

// configure custom bucket
const amplifyConfig = Amplify.getConfig();
Amplify.configure({
  ...amplifyConfig,
  Storage: {
    ...amplifyConfig.Storage,
    S3: {
      ...amplifyConfig.Storage?.S3,
      buckets: {
        ...amplifyConfig.Storage?.S3.buckets,
        "storage-browser-custom": {
          bucketName: "storage-browser-custom-bucket",
          region: "region",
+          paths: {
+           "customPublic/*": {
+              guest: ["get", "list", "write", "delete"],
+              authenticated: ["write", "get", "list", "delete"]
            },
          },
        },
      },
    },
  },
});

Note: Also please make sure necessary permission is added to Amplify cognito's auth/unauth IAM roles to access this custom bucket. You can find the auth/unauth IAM roles gen2 created under Cognito console -> Identity pools -> select your Identity pool -> user access.

[SyoujirouAbe]

Is it possible to control access on existing S3 depending on the Cognito user group, without using amplify backend？
I'm trying to configure 'paths' param like below, but I can only configure 'guest'/'authenticated' level so far.

Amplify.configure({
  Auth: {
    Cognito: {
      userPoolId: "xxx",
      userPoolClientId: "xxx",
      identityPoolId: "xxx",
    },
  },
  Storage: {
    S3: {
      bucket: 'my-sample-bucket',
      region: 'us-east-1',
      buckets: {
        "first-bucket": {
          bucketName: 'my-sample-bucket',
          region: 'us-east-1',
          paths: {
            "private/": {
              // ↓ working.
              "guest": ["read"],
              "authenticated": ["read", "write"],

              // ↓ not working. "admin" is a cognito user group name.
              "admin": ["read", "write", "delete"]
            }
          }
        }
      }
    },
  },
});

If I can control access depending on the custom attribute of the Cognito, it's perfect...