This is a CLI tool to download shared files and folders from Google Drive. For large file, the resumable download can be also run.
We have already known that the shared files on Google Drive can be downloaded without the authorization. But when the size of file becomes large (about 40MB), it requires a little ingenuity to download the file. It requires to access 2 times to Google Drive. At 1st access, it retrieves a cookie and a code for downloading. At 2nd access, the file is downloaded using the cookie and code. I created this process as a CLI tool.
This method uses API key.
There are sometimes the situation for downloading files in a shared folder. But I couldn't find the CLI applications for downloading files in the shared folder. So I implemented this. But when in order to retrieve the file list from the shared file, Drive API is required to be used. In order to use Drive API, it is required to use OAuth2, Service account and API key. So I selected to use API key which is the simplest way. This CLI tool can retrieve the file list in the shared folder using API key and download all files in the shared folder. From version 1.2.2, you can retrieve only files with the specific mimeType from the folder.
This method uses API key.
At a recent proposal, I knew the requirement of the resumable download of shared file. So I implemented this.
Download an executable file of goodls from the release page and import to a directory with path.
or
Use go get.
$ go install github.com/tanaikech/goodls@latest
GO111MODULE=on
The image used for this demonstration was created by k3-studio
You can use this just after you download or install goodls. You are not required to do like OAuth2 process.
$ goodls -u [URL of shared file on Google Drive]
- Help
$ goodls --help
- Options
-e
- Extension of output file. This is for only Google Docs (Spreadsheet, Document, Presentation). Default is
pdf
. Whenms
is used, the shared Google Docs can be downloaded as Microsoft Docs. - Sample :
$ goodls -u https://docs.google.com/document/d/#####/edit?usp=sharing -e txt
- Extension of output file. This is for only Google Docs (Spreadsheet, Document, Presentation). Default is
-f
- Filename of file which is output. When this was not used, the original filename on Google Drive is used.
- Sample :
$ goodls -u https://docs.google.com/document/d/#####/edit?usp=sharing -e txt -f sample.txt
- URL is like below.
- In the case of Google Docs (Spreadsheet, Document, Slides)
https://docs.google.com/spreadsheets/d/#####/edit?usp=sharing
https://docs.google.com/document/d/#####/edit?usp=sharing
https://docs.google.com/presentation/d/#####/edit?usp=sharing
- In the case of except for Google Docs
https://drive.google.com/file/d/#####/view?usp=sharing
- In the case of webContentLink
https://drive.google.com/uc?export=download&id=###
- In the case of Google Docs (Spreadsheet, Document, Slides)
If you have a file including URLs, you can input the URL data using standard input and pipe as follows. If wrong URL is included, the URL is skipped.
$ cat sample.txt | goodls
or
$ goodls < sample.txt
sample.txt
https://docs.google.com/spreadsheets/d/#####/edit?usp=sharing
https://docs.google.com/document/d/#####/edit?usp=sharing
https://docs.google.com/presentation/d/#####/edit?usp=sharing
When you download shared files from Google Drive, please confirm whether the files are shared.
When above structure is downloaded, the command is like below. At that time, the folder ID is the folder ID of "sampleFolder1".
Files are downloaded from the shared folder. In this demonstration, the fake folder ID and API key are used.
In order to use this, please retrieve API key as the following flow.
-
Login to Google.
-
Access to https://console.cloud.google.com/?hl=en.
-
Click select project at the right side of "Google Cloud Platform" of upper left of window.
- If you cannot see the project, please try to access to https://console.cloud.google.com/cloud-resource-manager?hl=en. You can also create new project at there. - When you created new project there, please click the left of "Google Cloud Platform". You can see it like 3 horizontal lines. By this, a side bar is opened. At there, please select "API & Services" -> "Library". After this, please check the following flow from
Click "Library" at left side.
.
- If you cannot see the project, please try to access to https://console.cloud.google.com/cloud-resource-manager?hl=en. You can also create new project at there. - When you created new project there, please click the left of "Google Cloud Platform". You can see it like 3 horizontal lines. By this, a side bar is opened. At there, please select "API & Services" -> "Library". After this, please check the following flow from
-
Click "NEW PROJECT"
- Input "Project Name".
- Click "CREATE".
- Open the created project.
- Click "Enable APIs and get credentials like keys".
- Click "Library" at left side.
- Input "Drive API" in "Search for APIs & Services".
- Click "Google Drive API".
- Click "ENABLE".
- Back to https://console.cloud.google.com/?hl=en.
- Click "Enable APIs and get credentials like keys".
- Click "Credentials" at left side.
- Click "Create credentials" and select API key.
- Copy the API key. You can use this API key.
- When API key is registered to the environment variable. When
GOODLS_APIKEY
as the environment variable is set API key, goodls uses API key from the environment variable. - If API key is used with the option at the command line, the priority of option is higher than the environment variable.
For example, in the case of bash, please add a following code to .bashrc.
export GOODLS_APIKEY=### your API key ###
When the URL of shared folder is https://drive.google.com/drive/folders/#####?usp=sharing
, you can download all files in the folder by the following command.
$ goodls -u https://drive.google.com/drive/folders/#####?usp=sharing -key [APIkey]
- Project files cannot be downloaded by API key. If you want to download the project files, you can download them by ggsrun, because ggsrun uses OAuth2.
- This new function uses the Go library of go-getfilelist.
- When the option of
--NoProgres
,-np
is used, the progress information is not seen. This is a silent mode. - If the files which are tried to be downloaded are existing, an error occurs. But when you use the option
--overwrite
and--skip
, the files are overwritten and skipped, respectively. - If you use the option
-m
, you can retrieve only files with the specific mimeType from the folder. For example, when you want to download PDF and PNG files, please run like below.$ goodls -u [URL] -key [APIkey] -m "application/pdf,image/png"
When you want to retrieve the information of file and folder, you can do it as follows.
$ goodls -u https://docs.google.com/spreadsheets/d/#####/edit?usp=sharing -key [APIkey] -i
$ goodls -u https://drive.google.com/drive/folders/#####?usp=sharing -key [APIkey] -i
When you use this option, at first, please retrieve API key. About how to retrieve API key, you can see at here.
When you want to download 100 MBytes of the shared file, you can use the following command.
$ goodls -u [URL of shared file on Google Drive] -key [APIkey] -r 100m
- Please use the option
-r
. In this sample,100m
means to download 100 MBytes of the shared file.- If you want to download 1 GB, please use
-r 1g
. - If you use
-r 1000000
, 1 MByte of the file will be able to be downloaded.
- If you want to download 1 GB, please use
You can see the actual running of this option at the following demonstration movie.
In this demonstration, the following command is run 3 times.
$ goodls -u https://drive.google.com/drive/folders/abcdefg?usp=sharing -key htjklmn -r 80m
- At 1st run, the data of 0 - 80 MBytes is downloaded.
- You can see
New download
at "Current status".
- You can see
- At 2nd run, the data of 80 - last is downloaded.
- You can see
Resumable download
at "Current status".
- You can see
- At 3rd run, the download has already been done. So the checksum is shown.
- You can see
Download has already done.
at "Current status".
- You can see
- Reason that API key is used for this.
- When it accesses to the shared file without the authorization, the file size and md5checksum cannot be retrieved. So in order to use Drive API, I adopted to use API key.
- Reason that the download size is inputted every time.
- When this option is run 1 time, 1 quota is used for Drive API. So I adopted this way.
-
I want to download shared projects from user's Google Drive.
- You can download shared projects using ggsrun.
- ggsrun can also download shared files from other user's Google Drive using Drive API which needs the access token.
-
I want to download all files including the standalone projects from the shared folder and own folder.
- You can achieve it using ggsrun.
-
I want to use this with "Dockerfile build".
- In this case, it has already been mentioned by rahulpowar at tanaikech#15. Please check it.
If you have any questions and commissions for me, feel free to tell me.
-
v1.0.0 (January 10, 2018)
- Initial release.
-
v1.0.1 (January 11, 2018)
- In order to download several files, a datafile including URLs using Standard Input and Pipe have gotten to be able to be inputted.
-
v1.0.2 (May 10, 2018)
- Files with large size has gotten to be able to be used.
- In order to download files with large size (several gigabytes), files are saved by chunks.
- Files with large size has gotten to be able to be used.
-
v1.0.3 (September 4, 2018)
- When the files are downloaded, the progress of downloading got to be able to be displayed.
- This demonstration can be seen at Demo.
- If the new option of
--np
is used, the progress is not displayed.
- When the files are downloaded, the progress of downloading got to be able to be displayed.
-
v1.1.0 (November 4, 2018)
- By using API key, files from the shared folder got to be able to be downloaded while keeping the folder structure.
- This demonstration can be seen at Demo.
- By using API key, the information of shared file and folder can be also retrieved.
- About the option of
--extension
and-e
, when-e ms
is used, Google Docs (Document, Spreadsheet, Slides) are converted to Microsoft Docs (Word, Excel, Powerpoint), respectively.
- By using API key, files from the shared folder got to be able to be downloaded while keeping the folder structure.
-
v1.1.1 (November 13, 2018)
1. Version of [go-getfilelist](https://github.com/tanaikech/go-getfilelist) was updated. Because the structure of ``drive.File`` got to be able to be used, I also updated this application.
-
v1.2.0 (November 24, 2018)
- By using API key, the shared large files can be run the resumable download.
- This demonstration can be seen at Demo.
- By using API key, the shared large files can be run the resumable download.
-
v1.2.1 (November 25, 2018)
- API key got to be able to be used by an environment variable. When
GOODLS_APIKEY
as the environment variable is set API key, goodls uses API key from the environment variable.
- API key got to be able to be used by an environment variable. When
-
v1.2.2 (December 12, 2018)
- When files are downloaded from a specific folder, it got to be able to select mimeType. By this, files with the specific mimeType in the specific folder can be retrieved. For this, I updated the go library go-getfilelist.
$ goodls -u [URL] -key [APIkey] -m "application/pdf,image/png"
- When files are downloaded from a specific folder, it got to be able to select mimeType. By this, files with the specific mimeType in the specific folder can be retrieved. For this, I updated the go library go-getfilelist.
-
v1.2.3 (October 31, 2019)
- An option for inputting the directory to save the downloaded files was added. When this option is not used, the files are saved to the current working directory.
$ goodls -u [URL] -d [directory]
- An option for inputting the directory to save the downloaded files was added. When this option is not used, the files are saved to the current working directory.
-
v1.2.4 (January 3, 2020)
- It seems that the specification of
github.com/urfave/cli
was changed by the update of https://github.com/urfave/cli. By this, whengo get -u github.com/tanaikech/goodls
is run, an error occurred. So this error was removed.
- It seems that the specification of
-
v1.2.5 (January 29, 2020)
- An option for selecting whether the top directory is created was added.
$ goodls -u [URL] --notcreatetopdirectory
or$ goodls -u [URL] -ntd
- When this option is NOT used (default situation), when a folder including sub-folders is downloaded, the top folder which is downloaded is created as the top directory under the working directory. When this option is used, the top directory is not created and all files and sub-folders under the top folder are downloaded under the working directory.
- This feature request was implemented.
- An option for selecting whether the top directory is created was added.
-
v1.2.6 (February 23, 2020)
- When the files are downloaded from the shared folder, if an error occurs, the download was stopped. In this thread, a request for skipping this error was proposed. This request was implemented at v1.2.6. Please try the option of
--skiperror
like below.$ goodls -u https://drive.google.com/drive/folders/###?usp=sharing --skiperror
- When the files are downloaded from the shared folder, if an error occurs, the download was stopped. In this thread, a request for skipping this error was proposed. This request was implemented at v1.2.6. Please try the option of
-
v1.2.7 (August 21, 2020)
- As the URL for downloading the files,
webContentLink
was added. So from this version, the URL ofhttps://drive.google.com/uc?export=download&id=###
got to be able to be used. This is the request from this thread.
- As the URL for downloading the files,
-
v1.2.8 (February 17, 2022)
- Recently, it seems that the specification the process for downloading the shared file on Google Drive has been changed. So I updated goodls for reflecting this. The usage of goodls is not changed.
-
v2.0.0 (February 25, 2022)
- By changing the specification of methods,
drive.New()
andtransport.APIKey
were deprecated. By this, I updated go-getfilelist. In this version, I used this updated library to goodls. And also,drive.NewService()
is used instead ofdrive.New()
.
- By changing the specification of methods,
-
v2.0.1 (February 26, 2022)
- A bug for the resumable download was removed.
-
v2.0.2 (February 24, 2023)
- Modified go.mod and go.sum.