Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error while creating folder after deleting it from gdrive #29

Closed
gcilinskas opened this issue Jul 16, 2021 · 6 comments
Closed

Error while creating folder after deleting it from gdrive #29

gcilinskas opened this issue Jul 16, 2021 · 6 comments

Comments

@gcilinskas
Copy link

gcilinskas commented Jul 16, 2021 •
edited
Loading

Steps to replicate:

  1. Create folder Storage::disk('google')->createDir('test');
  2. Delete 'test' folder in Google drive manually(delete from trash also)
  3. Create same folder Storage::disk('google')->createDir('test');

Locally works, but throws this error on live servers (AWS EC2 Instance Ubuntu)

[2021-07-16 06:59:58] staging.ERROR: Unknown exception encountered while retrieving product or dispatching asset download jobs: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "notFound",
    "message": "File not found: 1255h34g2AtWNgO4wjJPzzfHlxh_yIIyA.",
    "locationType": "parameter",
    "location": "fileId"
   }
  ],
  "code": 404,
  "message": "File not found: 1255h34g2AtWNgO4wjJPzzfHlxh_yIIyA."
 }
}
 {"asin":"B01LSUQSB0","marketplace":"us","exception":"[object] (Google\\Service\\Exception(code: 404): {
 \"error\": {
  \"errors\": [
   {
    \"domain\": \"global\",
    \"reason\": \"notFound\",
    \"message\": \"File not found: 1255h34g2AtWNgO4wjJPzzfHlxh_yIIyA.\",
    \"locationType\": \"parameter\",
    \"location\": \"fileId\"
   }
  ],
  \"code\": 404,
  \"message\": \"File not found: 1255h34g2AtWNgO4wjJPzzfHlxh_yIIyA.\"
 }
}
 at /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Http/REST.php:128)
[stacktrace]
#0 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Http/REST.php(103): Google\\Http\\REST::decodeHttpResponse()
#1 [internal function]: Google\\Http\\REST::doExecute()
#2 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Task/Runner.php(182): call_user_func_array()
#3 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Http/REST.php(66): Google\\Task\\Runner->run()
#4 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Client.php(898): Google\\Http\\REST::execute()
#5 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Service/Resource.php(238): Google\\Client->execute()
#6 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient-services/src/Drive/Resource/Files.php(101): Google\\Service\\Resource->call()
#7 /var/www/html/contentx/staging/releases/20211507_122037/vendor/masbug/flysystem-google-drive-ext/src/GoogleDriveAdapter.php(1282): Google\\Service\\Drive\\Resource\\Files->create()
#8 /var/www/html/contentx/staging/releases/20211507_122037/vendor/masbug/flysystem-google-drive-ext/src/GoogleDriveAdapter.php(573): Masbug\\Flysystem\\GoogleDriveAdapter->createDirectory()
#9 /var/www/html/contentx/staging/releases/20211507_122037/vendor/league/flysystem/src/Filesystem.php(263): Masbug\\Flysystem\\GoogleDriveAdapter->createDir()
#10 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Filesystem/FilesystemAdapter.php(787): League\\Flysystem\\Filesystem->createDir()
#11 /var/www/html/contentx/staging/releases/20211507_122037/app/Jobs/GetAsinDataJob.php(95): Illuminate\\Filesystem\\FilesystemAdapter->__call()
#12 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\\Jobs\\GetAsinDataJob->handle()
#13 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#14 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#15 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#16 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#17 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(128): Illuminate\\Container\\Container->call()
#18 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Bus\\Dispatcher->Illuminate\\Bus\\{closure}()
#19 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#20 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(132): Illuminate\\Pipeline\\Pipeline->then()
#21 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(120): Illuminate\\Bus\\Dispatcher->dispatchNow()
#22 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Queue\\CallQueuedHandler->Illuminate\\Queue\\{closure}()
#23 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#24 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(122): Illuminate\\Pipeline\\Pipeline->then()
#25 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(70): Illuminate\\Queue\\CallQueuedHandler->dispatchThroughMiddleware()
#26 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php(98): Illuminate\\Queue\\CallQueuedHandler->call()
#27 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(428): Illuminate\\Queue\\Jobs\\Job->fire()
#28 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(378): Illuminate\\Queue\\Worker->process()
#29 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(172): Illuminate\\Queue\\Worker->runJob()
#30 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(117): Illuminate\\Queue\\Worker->daemon()
#31 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(101): Illuminate\\Queue\\Console\\WorkCommand->runWorker()
#32 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\\Queue\\Console\\WorkCommand->handle()
#33 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#34 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#35 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#36 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#37 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Console/Command.php(136): Illuminate\\Container\\Container->call()
#38 /var/www/html/contentx/staging/releases/20211507_122037/vendor/symfony/console/Command/Command.php(299): Illuminate\\Console\\Command->execute()
#39 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Console/Command.php(121): Symfony\\Component\\Console\\Command\\Command->run()
#40 /var/www/html/contentx/staging/releases/20211507_122037/vendor/symfony/console/Application.php(978): Illuminate\\Console\\Command->run()
#41 /var/www/html/contentx/staging/releases/20211507_122037/vendor/symfony/console/Application.php(295): Symfony\\Component\\Console\\Application->doRunCommand()
#42 /var/www/html/contentx/staging/releases/20211507_122037/vendor/symfony/console/Application.php(167): Symfony\\Component\\Console\\Application->doRun()
#43 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Console/Application.php(92): Symfony\\Component\\Console\\Application->run()
#44 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\\Console\\Application->run()
#45 /var/www/html/contentx/staging/releases/20211507_122037/artisan(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#46 {main}
@erikn69
Copy link
Contributor

erikn69 commented Jul 16, 2021 •
edited
Loading

This package is using a little cache trick, if you delete a folder in Google drive manually, that folder still remains in cache even though it no longer exists in google drive (if you are using a long running script, because on every new instance it starts the connection and cache), maybe the function Storage::disk('google')->getDriver()->getAdapter()->clearCache(); before createDir will be useful to you (maybe with catch, only when there is these exception).

flysystem-google-drive-ext/src/GoogleDriveAdapter.php

Lines 257 to 263 in aa0b678

public function clearCache()
{
$this->cachedPaths = [];
$this->requestedIds = [];
$this->cacheFileObjects = [];
$this->cacheHasDirs = [];
}

@masbug is this a bug?, maybe i can make a fix

@gcilinskas
Copy link
Author

Storage::disk('google')->getDriver()->getAdapter()->clearCache();
This one helped, thanks!
Just for information, Since I use queued jobs, I had to call this clearCache command not only once before creating directories, but also in every job class that was using google drive upload

@gcilinskas gcilinskas reopened this Jul 16, 2021
@erikn69 erikn69 mentioned this issue Jul 16, 2021
Merged
@erikn69
Copy link
Contributor

erikn69 commented Jul 16, 2021 •
edited
Loading

On limitation's section

Using display paths as identifiers for folders and files requires them to be unique. Unfortunately Google Drive allows users to create files and folders with same (displayed) names. In such cases when unique path cannot be determined this adapter chooses the oldest (first) instance

Concurrent use of same Google Drive might lead to unexpected problems due to heavy caching of file/folder identifiers and file objects.

So, for concurrent threads of queued jobs you have to do this workaround with cache, otherwise you get those errors, or when create a directory duplicate(created at the same time with same name) and you put a unique file or folder inside, the adapter will be able to reach it properly because it chooses the oldest (first), so, create directory, uncache the folder, then put file inside
I make a PR with uncacheFolder function, if it is merged it be available on the next release

$disk=\Storage::disk('google');    
$disk->createDir('test/testa');
//manually delete without adapter or other job create it first 
$dir=$disk->getDriver()->getAdapter()->getMetadata('test/testa');
$disk->getDriver()->getAdapter()->getService()->files->delete($dir['id']); 
//uncache
$disk->getDriver()->getAdapter()->uncacheFolder('test/testa'); 
// refresh id and put file
$disk->put('test/testa/file.txt', 'Contents1');

@masbug
Copy link
Owner

masbug commented Jul 19, 2021

@gcilinskas
As @erikn69 pointed out, extensive object caching breaks concurrency. The code of this adapter assumes that it's operating alone on a selected drive.
One way to partially work around this would be to disable all caching in the code (which would slow everything down considerably)...but that's only half of the fix - there's still the issue of mixing write, read, delete, create operations and you'd still have race conditions to solve.
Perhaps you can just change the approach and not work on the same folder from multiple threads/jobs/queues? Or just not delete the folder after you're done with it (I assume you use one job to fill the folder and another to consume what's in it and then delete the folder)?

@gcilinskas
Copy link
Author

Yeah, manual human interaction with gdrive is possible on our side so removal of folders might be at random times. We are fetching data from api and saving its assets to gdrive in structured folders hierarchy.

clearCache method helped for now, performance for gdrive upload is not crucial for us. Regarding to racing conditions we did some workarounds with sleep functions, currently seems to work

We don't have options to change approaches in usage of gdrive folders

@masbug
Copy link
Owner

masbug commented Jul 22, 2021

Ok. So I guess you're doing something similar to:

lRetry:
try {
  ... small set of operations on the drive or a complete set with error mitigation
} catch(\Throwable $e) {
  sleep(rand()%4);
  $adapter->clearCache();
  if($retry-->0)
     goto lRetry;
}

@masbug masbug closed this as completed Jul 22, 2021
@philyuska philyuska mentioned this issue Jan 13, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@erikn69 @masbug @gcilinskas