Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add -R -z in curl #15

Merged
merged 2 commits into from
Apr 28, 2020
Merged

add -R -z in curl #15

merged 2 commits into from
Apr 28, 2020

Conversation

htyeim
Copy link
Contributor

@htyeim htyeim commented Apr 27, 2020

Add two parameters in curl command.

I think using timestemp of a RemoteFile might be better for RemoteFiles.jl package.

The description in curl.haxx.se

-R, --remote-time

When used, this will make curl attempt to figure out the timestamp of the remote file, and if that is available make the local file get that same timestamp. 

-z, --time-cond <time>

(HTTP FTP) Request a file that has been modified later than the given time and date, or one that has been modified before that time. The <date expression> can be all sorts of date strings or if it doesn't match any internal ones, it is taken as a filename and tries to get the modification date (mtime) from <file> instead. See the curl_getdate(3) man pages for date expression details.

Start the date expression with a dash (-) to make it request for a document that is older than the given date/time, default is a document that is newer than the specified date/time.

If this option is used several times, the last one will be used.

The timestamp could be found as follow:

curl -I https://curl.haxx.se/

HTTP/2 200 
server: Apache
x-frame-options: SAMEORIGIN
last-modified: Mon, 27 Apr 2020 02:13:49 GMT
etag: "21e7-5a43c43fc71bf"
cache-control: max-age=60
expires: Mon, 27 Apr 2020 02:15:10 GMT
x-content-type-options: nosniff
content-security-policy: default-src 'self' www.fastly-insights.com; style-src 'unsafe-inline' 'self'
strict-transport-security: max-age=31536000; includeSubDomains;
content-type: text/html
via: 1.1 varnish
accept-ranges: bytes
date: Mon, 27 Apr 2020 03:19:49 GMT
via: 1.1 varnish
age: 23
x-served-by: cache-bma1641-BMA, cache-tyo19944-TYO
x-cache: HIT, MISS
x-cache-hits: 1, 0
x-timer: S1587957589.889921,VS0,VE1073
vary: Accept-Encoding
content-length: 8679

Last-Modified in HTTP standard could be found in w3.org

14.29 Last-Modified
The Last-Modified entity-header field indicates the date and time at which the origin server believes the variant was last modified.

       Last-Modified  = "Last-Modified" ":" HTTP-date
An example of its use is

       Last-Modified: Tue, 15 Nov 1994 12:45:26 GMT
The exact meaning of this header field depends on the implementation of the origin server and the nature of the original resource. For files, it may be just the file system last-modified time. For entities with dynamically included parts, it may be the most recent of the set of last-modify times for its component parts. For database gateways, it may be the last-update time stamp of the record. For virtual objects, it may be the last time the internal state changed.

An origin server MUST NOT send a Last-Modified date which is later than the server's time of message origination. In such cases, where the resource's last modification would indicate some time in the future, the server MUST replace that date with the message origination date.

An origin server SHOULD obtain the Last-Modified value of the entity as close as possible to the time that it generates the Date value of its response. This allows a recipient to make an accurate assessment of the entity's modification time, especially if the entity changes near the time that the response is generated.

HTTP/1.1 servers SHOULD send Last-Modified whenever feasible.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 93.125% when pulling 28cb731 on htyeim:master into daa641a on helgee:master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage remained the same at 93.125% when pulling 28cb731 on htyeim:master into daa641a on helgee:master.

@helgee
Copy link
Owner

helgee commented Apr 27, 2020

LGTM, thanks! Could you please revert the unrelated whitespace changes? Keyword arguments are usually written without surrounding spaces, e.g. foo(bar; baz=0).

@htyeim
Copy link
Contributor Author

htyeim commented Apr 28, 2020

I am sorry, but I don't know how to revert them. I use vscode and the file was auto formated when I saved it.

@helgee
Copy link
Owner

helgee commented Apr 28, 2020

I am sorry, but I don't know how to revert them. I use vscode and the file was auto formated when I saved it.

No problem. I took care of it 👍

@helgee helgee merged commit f4fe1ac into helgee:master Apr 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants