Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update download apis to use path #4561

Merged
merged 49 commits into from
Mar 25, 2024
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
7df1e46
chore: add `StoragePath`
Jordan-Nelson Mar 11, 2024
a69e1f9
chore: add `S3PathResolver`
Jordan-Nelson Mar 11, 2024
d9a44f9
chore: update `getUrl` and `getProperties` API
Jordan-Nelson Mar 11, 2024
075824d
chore: update S3UploadTask
Jordan-Nelson Mar 11, 2024
e7f3dbe
chore: fix tests
Jordan-Nelson Mar 12, 2024
7665485
chore: validate storage path
Jordan-Nelson Mar 12, 2024
669f1e5
chore: add tests for S3PathResolver
Jordan-Nelson Mar 12, 2024
2ede7fa
feat: Gen 2 Upload APIs (#4542)
khatruong2009 Mar 13, 2024
174d65f
chore: refactor StoragePath (#4544)
Jordan-Nelson Mar 13, 2024
b238ea6
chore: remove `key` from public APIs
Jordan-Nelson Mar 13, 2024
3bf3669
chore: update tests to adjust for using path
khatruong2009 Mar 14, 2024
a30ec9d
chore: fix lint issues
Jordan-Nelson Mar 15, 2024
9f9e40b
chore: fix formatting
Jordan-Nelson Mar 15, 2024
b03df43
chore: fix test
Jordan-Nelson Mar 15, 2024
8ef34be
chore: remove lint ignored
Jordan-Nelson Mar 15, 2024
408df98
chore: update comment
Jordan-Nelson Mar 15, 2024
3155206
chore: remove unused method
Jordan-Nelson Mar 15, 2024
c3d40aa
chore: update download apis to use path
khatruong2009 Mar 15, 2024
2172ee8
Merge branch 'feat/storage-gen-2' into feat/storage-gen-2-download-apis
khatruong2009 Mar 15, 2024
de1d2e2
chore: skip tests other than main
Jordan-Nelson Mar 15, 2024
28d8819
chore: run dart format on both files that are receiving analyze warnings
khatruong2009 Mar 15, 2024
aed1e31
chore: used dart fix to add trailing commas
khatruong2009 Mar 15, 2024
7207fd1
chore: fixed formatting
khatruong2009 Mar 15, 2024
5961b72
chore: run format on test files
khatruong2009 Mar 15, 2024
c1da20c
chore: fix formatting in integration tests
khatruong2009 Mar 15, 2024
c98a08a
chore: update generated code
Jordan-Nelson Mar 18, 2024
8722ad8
chore: add leading `/`
Jordan-Nelson Mar 18, 2024
74420c2
Merge branch 'feat/storage-gen-2' into feat/storage-gen-2-download-apis
khatruong2009 Mar 18, 2024
31cbaea
chore: remove access level from e2e test
khatruong2009 Mar 18, 2024
6c615e3
fix(auth): device metadata migration (#4503)
Jordan-Nelson Mar 11, 2024
255855f
fix(auth): Allow retries with verifyTotpSetup() (#4532)
Equartey Mar 13, 2024
bd4bfe2
chore(version): cherry pick Bump version (#4545)
Equartey Mar 14, 2024
941b192
fix(auth): verifyTotp throw EnableSoftwareTokenMfaException (#4558)
Equartey Mar 15, 2024
188a256
chore(version): cherry pick Bump version (#4560)
Equartey Mar 18, 2024
eabcd53
chore: Update BUG-REPORT.yaml (#4568)
Jordan-Nelson Mar 19, 2024
fdce4a2
fix: correct `package_info_plus` version constraint (#4583)
Jordan-Nelson Mar 21, 2024
da10dda
feat(storage): copy and move APIs (#4569)
Jordan-Nelson Mar 21, 2024
e54c549
chore: update download apis to use path
khatruong2009 Mar 15, 2024
1ddc587
Merge branch 'feat/storage-gen-2' into feat/storage-gen-2-download-apis
khatruong2009 Mar 22, 2024
304da8c
chore: remove duplicate code from rebase
khatruong2009 Mar 22, 2024
761c494
chore: removed random text file
khatruong2009 Mar 22, 2024
df3fe46
chore: remove commented code
khatruong2009 Mar 22, 2024
a5c93a4
chore: fix failing tests
Jordan-Nelson Mar 24, 2024
90f873a
chore: update options, tests
Jordan-Nelson Mar 24, 2024
250df23
chore: update options classes
Jordan-Nelson Mar 24, 2024
eb8eafa
chore: add back temp dir
Jordan-Nelson Mar 25, 2024
69c1881
chore: fix integ test
Jordan-Nelson Mar 25, 2024
0d300f2
chore: add skip back to io specific test
Jordan-Nelson Mar 25, 2024
49ff753
chore: skip test set up in configs other than main
Jordan-Nelson Mar 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -91,43 +91,43 @@ class StorageCategory extends AmplifyCategory<StoragePluginInterface> {
}

/// {@template amplify_core.amplify_storage_category.download_data}
/// Downloads bytes of object specified by [key] into memory with optional
/// Downloads bytes of object specified by [path] into memory with optional
/// [onProgress] and [StorageDownloadDataOptions], and returns a
/// [StorageDownloadDataOperation].
///
/// Ensure you are managing the data in memory properly to avoid unexpected
/// memory leaks.
/// {@endtemplate}
StorageDownloadDataOperation downloadData({
required String key,
required StoragePath path,
void Function(StorageTransferProgress)? onProgress,
StorageDownloadDataOptions? options,
}) {
return identifyCall(
StorageCategoryMethod.downloadData,
() => defaultPlugin.downloadData(
key: key,
path: path,
onProgress: onProgress,
options: options,
),
);
}

/// {@template amplify_core.amplify_storage_category.download_file}
/// Downloads the object specified by [key] to [localFile] with optional
/// Downloads the object specified by [path] to [localFile] with optional
/// [onProgress] and [StorageDownloadFileOptions], and returns a
/// [StorageDownloadFileOperation].
/// {@endtemplate}
StorageDownloadFileOperation downloadFile({
required String key,
required StoragePath path,
required AWSFile localFile,
void Function(StorageTransferProgress)? onProgress,
StorageDownloadFileOptions? options,
}) {
return identifyCall(
StorageCategoryMethod.downloadFile,
() => defaultPlugin.downloadFile(
key: key,
path: path,
localFile: localFile,
onProgress: onProgress,
options: options,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ abstract class StoragePluginInterface extends AmplifyPluginInterface {

/// {@macro amplify_core.amplify_storage_category.download_data}
StorageDownloadDataOperation downloadData({
required String key,
required StoragePath path,
void Function(StorageTransferProgress)? onProgress,
StorageDownloadDataOptions? options,
}) {
Expand All @@ -49,7 +49,7 @@ abstract class StoragePluginInterface extends AmplifyPluginInterface {

/// {@macro amplify_core.amplify_storage_category.download_file}
StorageDownloadFileOperation downloadFile({
required String key,
required StoragePath path,
required AWSFile localFile,
void Function(StorageTransferProgress)? onProgress,
StorageDownloadFileOptions? options,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ import 'package:amplify_core/amplify_core.dart';
class StorageDownloadDataRequest {
/// {@macro amplify_core.storage.download_data_request}
const StorageDownloadDataRequest({
required this.key,
required this.path,
this.options,
});

/// Key of the object to download.
final String key;
/// Path of the object to download.
final StoragePath path;

/// Configurable options of the [StorageDownloadDataRequest].
final StorageDownloadDataOptions? options;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ import 'package:amplify_core/amplify_core.dart';
class StorageDownloadFileRequest {
/// {@macro amplify_core.storage.download_file_request}
const StorageDownloadFileRequest({
required this.key,
required this.path,
required this.localFile,
this.options,
});

/// Key of the object to download.
final String key;
/// Path of the object to download.
final StoragePath path;

/// Reference to the local file that download the object to.
final AWSFile localFile;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,11 @@ void testTransferAcceleration({
test('S3DataPayload ${entry.key}', () async {
final dataPayload = entry.value;
final result = await Amplify.Storage.downloadData(
key: dataPayload.targetKey,
options: StorageDownloadDataOptions(
accessLevel: dataPayload.targetAccessLevel,
pluginOptions: const S3DownloadDataPluginOptions(
path: StoragePath.fromString(
'/${dataPayload.targetAccessLevel}/${dataPayload.targetKey}',
),
options: const StorageDownloadDataOptions(
pluginOptions: S3DownloadDataPluginOptions(
useAccelerateEndpoint: true,
),
),
Expand Down Expand Up @@ -156,9 +157,10 @@ void testTransferAcceleration({
const end = 5 * 1024 + 12;
final awsFile = entry.value;
final result = await Amplify.Storage.downloadData(
key: awsFile.targetKey,
path: StoragePath.fromString(
'${awsFile.targetAccessLevel}/${awsFile.targetKey}',
),
options: StorageDownloadDataOptions(
accessLevel: awsFile.targetAccessLevel,
pluginOptions: S3DownloadDataPluginOptions(
useAccelerateEndpoint: true,
bytesRange: S3DataBytesRange(start: start, end: end),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -320,13 +320,14 @@ void main() {
});

testWidgets(
skip: true,
// skip: true,
khatruong2009 marked this conversation as resolved.
Show resolved Hide resolved
'download object as bytes data in memory with access level private'
' for the currently signed in user', (WidgetTester tester) async {
final result = await Amplify.Storage.downloadData(
key: testObjectKey3,
path: StoragePath.withIdentityId(
(identityId) => '/private/$identityId/$testObjectKey3',
),
options: const StorageDownloadDataOptions(
accessLevel: StorageAccessLevel.private,
pluginOptions: S3DownloadDataPluginOptions(
getProperties: true,
),
Expand All @@ -338,15 +339,16 @@ void main() {
});

testWidgets(
skip: true,
// skip: true,
'download a range of bytes of an object with access level private'
' for the currently signed in user', (WidgetTester tester) async {
const start = 5 * 1024;
const end = 5 * 1024 + 12;
final result = await Amplify.Storage.downloadData(
key: testObjectKey3,
path: StoragePath.withIdentityId(
(identityId) => '/private/$identityId/$testObjectKey3',
),
options: StorageDownloadDataOptions(
accessLevel: StorageAccessLevel.private,
pluginOptions: S3DownloadDataPluginOptions(
getProperties: true,
bytesRange: S3DataBytesRange(
Expand All @@ -366,7 +368,7 @@ void main() {
});

testWidgets(
skip: true,
// skip: true,
'download file with access level private for the currently signed in user',
(WidgetTester tester) async {
final s3Plugin =
Expand All @@ -377,10 +379,11 @@ void main() {

final result = await s3Plugin
.downloadFile(
key: testObjectKey3,
path: StoragePath.withIdentityId(
(identityId) => '/private/$identityId/$testObjectKey3',
),
localFile: localFile,
options: const StorageDownloadFileOptions(
accessLevel: StorageAccessLevel.private,
pluginOptions: S3DownloadFilePluginOptions(
getProperties: true,
),
Expand Down Expand Up @@ -618,14 +621,15 @@ void main() {
});

testWidgets(
skip: true,
// skip: true,
'download data of object with access level protected and a target'
' identity id for the currently signed user',
(WidgetTester tester) async {
final result = await Amplify.Storage.downloadData(
key: testObjectKey2,
path: StoragePath.withIdentityId(
(identityId) => '/protected/$identityId/$testObjectKey2',
),
options: StorageDownloadDataOptions(
accessLevel: StorageAccessLevel.protected,
pluginOptions: S3DownloadDataPluginOptions.forIdentity(
user1IdentityId,
getProperties: true,
Expand All @@ -637,14 +641,15 @@ void main() {
});

testWidgets(
skip: true,
// skip: true,
'download data of object (belongs to other user) with access level'
' private for the currently signed user',
(WidgetTester tester) async {
final operation = Amplify.Storage.downloadData(
key: testObjectKey3,
path: StoragePath.withIdentityId(
(identityId) => '/private/$identityId/$testObjectKey3',
),
options: const StorageDownloadDataOptions(
accessLevel: StorageAccessLevel.private,
pluginOptions: S3DownloadDataPluginOptions(
getProperties: true,
),
Expand Down
4 changes: 2 additions & 2 deletions packages/storage/amplify_storage_s3/example/lib/main.dart
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ class _HomeScreenState extends State<HomeScreen> {
final filepath = '${documentsDir.path}/$key';
try {
await Amplify.Storage.downloadFile(
key: key,
path: StoragePath.fromString(key),
localFile: AWSFile.fromPath(filepath),
onProgress: (p0) => _logger
.debug('Progress: ${(p0.transferredBytes / p0.totalBytes) * 100}%'),
Expand All @@ -196,7 +196,7 @@ class _HomeScreenState extends State<HomeScreen> {
Future<void> downloadFileWeb(String key) async {
try {
await Amplify.Storage.downloadFile(
key: key,
path: StoragePath.fromString(key),
localFile: AWSFile.fromPath(key),
onProgress: (p0) => _logger
.debug('Progress: ${(p0.transferredBytes / p0.totalBytes) * 100}%'),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,8 @@ Future<void> downloadDataOperation() async {

final s3Plugin = Amplify.Storage.getPlugin(AmplifyStorageS3Dart.pluginKey);
final downloadDataOperation = s3Plugin.downloadData(
key: key,
// key: key,
khatruong2009 marked this conversation as resolved.
Show resolved Hide resolved
path: StoragePath.fromString(key),
options: StorageDownloadDataOptions(
accessLevel: accessLevel,
pluginOptions: const S3DownloadDataPluginOptions(
Expand Down Expand Up @@ -251,7 +252,8 @@ Future<void> downloadFileOperation() async {

final s3Plugin = Amplify.Storage.getPlugin(AmplifyStorageS3Dart.pluginKey);
final downloadFileOperation = s3Plugin.downloadFile(
key: key,
// key: key,
path: StoragePath.fromString(key),
localFile: localFile,
options: StorageDownloadFileOptions(
accessLevel: accessLevel,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ class AmplifyStorageS3Dart extends StoragePluginInterface

@override
S3DownloadDataOperation downloadData({
required String key,
required StoragePath path,
StorageDownloadDataOptions? options,
void Function(S3TransferProgress)? onProgress,
}) {
Expand All @@ -232,21 +232,20 @@ class AmplifyStorageS3Dart extends StoragePluginInterface
);

final s3Options = StorageDownloadDataOptions(
accessLevel: options?.accessLevel,
pluginOptions: s3PluginOptions,
);

final bytes = BytesBuilder();
final downloadTask = storageS3Service.downloadData(
key: key,
path: path,
options: s3Options,
onProgress: onProgress,
onData: bytes.add,
);

return S3DownloadDataOperation(
request: StorageDownloadDataRequest(
key: key,
path: path,
options: options,
),
result: downloadTask.result.then(
Expand All @@ -263,7 +262,7 @@ class AmplifyStorageS3Dart extends StoragePluginInterface

@override
S3DownloadFileOperation downloadFile({
required String key,
required StoragePath path,
required AWSFile localFile,
void Function(S3TransferProgress)? onProgress,
StorageDownloadFileOptions? options,
Expand All @@ -273,11 +272,10 @@ class AmplifyStorageS3Dart extends StoragePluginInterface
defaultPluginOptions: const S3DownloadFilePluginOptions(),
);
options = StorageDownloadFileOptions(
accessLevel: options?.accessLevel,
pluginOptions: s3PluginOptions,
);
return download_file_impl.downloadFile(
key: key,
path: path,
localFile: localFile,
options: options,
s3pluginConfig: s3pluginConfig,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ S3DownloadFileOperation downloadFile({

return S3DownloadFileOperation(
request: StorageDownloadFileRequest(
key: key,
// TODO(khatruong2009): update to use path from customer
path: StoragePath.fromString(key),
localFile: localFile,
options: options,
),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,11 @@ import 'package:amplify_core/amplify_core.dart';
import 'package:amplify_storage_s3_dart/amplify_storage_s3_dart.dart';
import 'package:amplify_storage_s3_dart/src/storage_s3_service/storage_s3_service.dart';
import 'package:meta/meta.dart';
import 'package:path/path.dart' as path;

/// The io implementation of `downloadFile` API.
@internal
S3DownloadFileOperation downloadFile({
required String key,
required StoragePath path,
required AWSFile localFile,
required StorageDownloadFileOptions options,
required S3PluginConfig s3pluginConfig,
Expand All @@ -27,7 +26,6 @@ S3DownloadFileOperation downloadFile({
final s3PluginOptions = options.pluginOptions as S3DownloadFilePluginOptions;
final targetIdentityId = s3PluginOptions.targetIdentityId;
final downloadDataOptions = StorageDownloadDataOptions(
accessLevel: options.accessLevel,
pluginOptions: targetIdentityId == null
? S3DownloadDataPluginOptions(
getProperties: s3PluginOptions.getProperties,
Expand All @@ -41,17 +39,14 @@ S3DownloadFileOperation downloadFile({
);

final downloadDataTask = storageS3Service.downloadData(
key: key,
path: path,
options: downloadDataOptions,
// Ensure destination file is writable. Exception thrown in the check
// will be forwarded to the Future, downloadDataTask.result below
preStart: () async {
destinationPath = await _ensureDestinationWritable(localFile);
tempFile = File(
path.join(
await appPathProvider.getTemporaryPath(),
'amplify_storage_s3_temp_${uuid()}',
),
uuid(),
);
sink = tempFile.openWrite(mode: FileMode.append);
},
Expand All @@ -78,7 +73,7 @@ S3DownloadFileOperation downloadFile({

return S3DownloadFileOperation(
request: StorageDownloadFileRequest(
key: key,
path: path,
localFile: localFile,
options: options,
),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import 'package:amplify_storage_s3_dart/src/storage_s3_service/storage_s3_servic

/// Interface of the platform implementation of the `downloadFile` API.
S3DownloadFileOperation downloadFile({
required String key,
required StoragePath path,
required AWSFile localFile,
required StorageDownloadFileOptions options,
required S3PluginConfig s3pluginConfig,
Expand Down