/curl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add microsecond precise timers for measuring various intervals #2495

Closed
wants to merge 1 commit into
base: master
from

Conversation

Reviewers

@bagder bagder

@jay jay

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@pprindeville @bagder @jay
@pprindeville
Copy link
Contributor

pprindeville commented Apr 14, 2018

Work in progress.

Trying to make the definition of CURLINFO_TOTAL_TIME_T, et al be conditional on whether CURL_SIZEOF_CURL_OFF_T == 8 or not.

CURL_SIZEOF_CURL_OFF_T is defined in lib/curl_setup.h but that's not included in curl/curl.h... investigating a work-around.

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch 2 times, most recently from da3f93c to cd37983 Apr 15, 2018

@bagder

This comment has been minimized.

Sign in to view
Copy link
Member

bagder commented Apr 16, 2018 

../../include/curl/curl.h:2530:5: warning: "CURL_SIZEOF_CURL_OFF_T" is not defined [-Wundef]
 #if CURL_SIZEOF_CURL_OFF_T == 8
     ^

CURL_SIZEOF_CURL_OFF_T is not defined by the global public curl headers so you can't rely on that in there! But that I don't think you should add ifdef-conditional options to begin with so just remove the ifdef...

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch from cd37983 to 941d7bb Apr 16, 2018

@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented Apr 16, 2018

But that I don't think you should add ifdef-conditional options to begin with so just remove the ifdef...

Done.
@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented Apr 19, 2018

Any other blockers?
@bagder

This comment has been minimized.

Sign in to view
Copy link
Member

bagder commented Apr 19, 2018

The CI builds are still red so I haven't really checked out the details.

progressfunc.c:52:3: error: conversion to ‘double’ from ‘curl_off_t’ may alter its value [-Werror=conversion]
   if((curtime - myp->lastruntime) >= MINIMAL_PROGRESS_FUNCTIONALITY_INTERVAL) {
   ^
progressfunc.c:53:5: error: conversion to ‘double’ from ‘curl_off_t’ may alter its value [-Werror=conversion]
     myp->lastruntime = curtime;
     ^

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch 2 times, most recently from ada1661 to 558174d Apr 19, 2018

@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented Apr 19, 2018

The CI builds are still red so I haven't really checked out the details.

Saw some errors, yeah, but I thought they were pre-existing. But you're right, the above was introduced by my changes. Fixing it now.

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch from 558174d to ab269cb Apr 20, 2018

@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented Apr 20, 2018

Also failed the symbols-in-versions test (1119). Fixed that.

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch from ab269cb to 6ebc305 Apr 20, 2018

@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented Apr 20, 2018

@bagder Does Travis build each commit in a PR separately or just the last one?

I fixed the issue that you pointed out above, then it built but failed elsewhere (with the missing versioning of the new symbols) so I fixed that... now it's back to failing at the above place. Weird.

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch 2 times, most recently from eb924bf to 924da0b Apr 20, 2018

@bagder

bagder reviewed Apr 20, 2018
View changes

Copy link
Member

bagder left a comment

I think this looks great, I only have a few comments and questions to ponder.
docs/examples/progressfunc.c Outdated

/* under certain circumstances it may be desirable for certain functionality
to only run every N seconds, in order to do this the transaction time can
be used */
if((curtime - myp->lastruntime) >= MINIMAL_PROGRESS_FUNCTIONALITY_INTERVAL) {
if(((curtime - myp->lastruntime) / 1000000) >= MINIMAL_PROGRESS_FUNCTIONALITY_INTERVAL) {

This comment has been minimized.

Sign in to view
Copy link
@bagder

bagder Apr 20, 2018

Member

Wouldn't this look nicer if instead the constant MINIMAL_PROGRESS_FUNCTIONALITY_INTERVAL was multiplied with 1000000 ?
lib/getinfo.c Outdated
@@ -281,6 +281,30 @@ static CURLcode getinfo_offt(struct Curl_easy *data, CURLINFO info,
*param_offt = (data->progress.flags & PGRS_UL_SIZE_KNOWN)?
data->progress.size_ul:-1;
break;
#if CURL_SIZEOF_CURL_OFF_T == 8

This comment has been minimized.

Sign in to view
Copy link
@bagder

bagder Apr 20, 2018

Member

Why limit these to 64bit only?

This comment has been minimized.

Sign in to view
Copy link
@pprindeville

pprindeville Apr 20, 2018

Contributor

Okay, since it's a relative timestamp and not absolute, sure, I'll make it 32-bit as well.
include/curl/curl.h Outdated
((off_t) / 1000000L), (long)((off_t) % 1000000L)

#define CURL_FORMAT_TIME_TUPLE \
CURL_FORMAT_CURL_OFF_T ".%ld"

This comment has been minimized.

Sign in to view
Copy link
@bagder

bagder Apr 20, 2018

Member

These might be handy for an application, but they're also very basic and for very specific and narrow use cases. Does it really add anything to provide them in the public header?

To me they feel more like handy macros for our test suite.

This comment has been minimized.

Sign in to view
Copy link
@pprindeville

pprindeville Apr 20, 2018

Contributor

Well, since other format specifiers were exposed in system.h I was trying to follow that model.
tests/libtest/lib500.c
curl_easy_getinfo(curl, CURLINFO_CONNECT_TIME, &time_connect);
curl_easy_getinfo(curl, CURLINFO_PRETRANSFER_TIME, &time_pretransfer);
curl_easy_getinfo(curl, CURLINFO_STARTTRANSFER_TIME,
curl_easy_getinfo(curl, CURLINFO_NAMELOOKUP_TIME_T, &time_namelookup);

This comment has been minimized.

Sign in to view
Copy link
@bagder

bagder Apr 20, 2018

Member

Since you limited the code to extract this time to 64 bit only, won't this code fail hard on 32 bit systems?

This comment has been minimized.

Sign in to view
Copy link
@pprindeville

pprindeville Apr 20, 2018

Contributor

Made it 32-bit also.
docs/libcurl/symbols-in-versions Outdated
@@ -206,9 +206,11 @@ CURLHEADER_SEPARATE 7.37.0
CURLHEADER_UNIFIED 7.37.0
CURLINFO_ACTIVESOCKET 7.45.0
CURLINFO_APPCONNECT_TIME 7.19.0
CURLINFO_APPCONNECT_TIME_T 7.60.0

This comment has been minimized.

Sign in to view
Copy link
@bagder

bagder Apr 20, 2018

Member

Since the feature window has closed now for 7.60.0 and this is a change, we can assume this change can land in 7.61.0.

This comment has been minimized.

Sign in to view
Copy link
@pprindeville

pprindeville Apr 20, 2018

Contributor

Changed... the Travis testing is kicking my ass, though... Can't get it to pass the tests 1139 and 1140.

This comment has been minimized.

Sign in to view
Copy link
@jay

jay Apr 21, 2018

Member

I don't see it in the most recent commit. I see it fails because of this:

lib500.c: In function ‘test’:
lib500.c:115:112: error: expected identifier before ‘%’ token
           fprintf(moo, "namelookup vs connect: %" CURL_FORMAT_CURL_OFF_T ".%06ld " %" CURL_FORMAT_CURL_OFF_T ".%06ld\n",

there's a quote in middle of the ".%06ld " %" which it looks like causes a slew of syntax errors

This comment has been minimized.

Sign in to view
Copy link
@pprindeville

pprindeville Apr 21, 2018

Contributor

Fixed

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch 6 times, most recently from 5f0e463 to 678268f Apr 20, 2018

@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented Apr 21, 2018

Not understanding what's needed here:

test 1139...[Verify that all libcurl options have man pages]

perl  returned 7, when expecting 0
 exit FAILED
== Contents of files in the log/ dir after test 1139
=== Start of file stderr1139
 Missing CURLINFO_APPCONNECT_TIME_T.3
 Missing CURLINFO_CONNECT_TIME_T.3
 Missing CURLINFO_NAMELOOKUP_TIME_T.3
 Missing CURLINFO_PRETRANSFER_TIME_T.3
 Missing CURLINFO_REDIRECT_TIME_T.3
 Missing CURLINFO_STARTTRANSFER_TIME_T.3
 Missing CURLINFO_TOTAL_TIME_T.3
=== End of file stderr1139
test 1140...[Verify the nroff of man pages]

perl  returned 7, when expecting 0
 exit FAILED
== Contents of files in the log/ dir after test 1140
=== Start of file stderr1140
 error: ./../docs/libcurl/curl_easy_getinfo.3:65: referring to non-existing man page CURLINFO_TOTAL_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:71: referring to non-existing man page CURLINFO_NAMELOOKUP_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:77: referring to non-existing man page CURLINFO_CONNECT_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:83: referring to non-existing man page CURLINFO_APPCONNECT_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:89: referring to non-existing man page CURLINFO_PRETRANSFER_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:95: referring to non-existing man page CURLINFO_STARTTRANSFER_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:101: referring to non-existing man page CURLINFO_REDIRECT_TIME_T.3
=== End of file stderr1140
@jay

This comment has been minimized.

Sign in to view
Copy link
Member

jay commented Apr 21, 2018

because it's not documented, you need man pages for each. how is CURLINFO_APPCONNECT_TIME different from CURLINFO_APPCONNECT_TIME_T
@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented Apr 21, 2018

Can we have the *TIME and *TIME_T versions share the same pages?

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch 4 times, most recently from f5e2890 to d33b20f Apr 23, 2018

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch 2 times, most recently from 5d33f05 to 62a1d5a Apr 28, 2018

@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented May 1, 2018

I think the PR is good to go. @jay @bagder?
@pprindeville
add microsecond precise timers for measuring various intervals 
We overload the curl_off_t, if it's 64-bits wide, for extending
the API to allow retrieving times in microseconds (which is how
they're stored internally).
Loading status checks…
450fc14

@pprindeville pprindeville force-pushed the pprindeville:add-integer-times branch from 62a1d5a to 450fc14 May 1, 2018

@jay

This comment has been minimized.

Sign in to view
Copy link
Member

jay commented May 2, 2018

I'll check it again it looks like you did more work on it. Even if approved it can't be added until the next feature window, in 2 weeks.
@pprindeville

This comment has been minimized.

Sign in to view
Copy link
Contributor

pprindeville commented May 16, 2018

When does the window reopen?
@bagder

This comment has been minimized.

Sign in to view
Copy link
Member

bagder commented May 17, 2018

At release, which was yesterday!
@bagder

This comment has been minimized.

Sign in to view
Copy link
Member

bagder commented May 17, 2018

Thanks!

@bagder bagder closed this in ce2140a May 17, 2018

@lock lock bot locked as resolved and limited conversation to collaborators Aug 15, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.