add microsecond precise timers for measuring various intervals #2495

pprindeville · 2018-04-14T02:13:35Z

Work in progress.

Trying to make the definition of CURLINFO_TOTAL_TIME_T, et al be conditional on whether CURL_SIZEOF_CURL_OFF_T == 8 or not.

CURL_SIZEOF_CURL_OFF_T is defined in lib/curl_setup.h but that's not included in curl/curl.h... investigating a work-around.

bagder · 2018-04-16T15:05:01Z

../../include/curl/curl.h:2530:5: warning: "CURL_SIZEOF_CURL_OFF_T" is not defined [-Wundef]
 #if CURL_SIZEOF_CURL_OFF_T == 8
     ^

CURL_SIZEOF_CURL_OFF_T is not defined by the global public curl headers so you can't rely on that in there! But that I don't think you should add ifdef-conditional options to begin with so just remove the ifdef...

pprindeville · 2018-04-16T23:38:13Z

But that I don't think you should add ifdef-conditional options to begin with so just remove the ifdef...

Done.

pprindeville · 2018-04-19T18:23:43Z

Any other blockers?

bagder · 2018-04-19T21:16:57Z

The CI builds are still red so I haven't really checked out the details.

progressfunc.c:52:3: error: conversion to ‘double’ from ‘curl_off_t’ may alter its value [-Werror=conversion]
   if((curtime - myp->lastruntime) >= MINIMAL_PROGRESS_FUNCTIONALITY_INTERVAL) {
   ^
progressfunc.c:53:5: error: conversion to ‘double’ from ‘curl_off_t’ may alter its value [-Werror=conversion]
     myp->lastruntime = curtime;
     ^

pprindeville · 2018-04-19T23:48:24Z

The CI builds are still red so I haven't really checked out the details.

Saw some errors, yeah, but I thought they were pre-existing. But you're right, the above was introduced by my changes. Fixing it now.

pprindeville · 2018-04-20T15:13:47Z

Also failed the symbols-in-versions test (1119). Fixed that.

pprindeville · 2018-04-20T17:22:03Z

@bagder Does Travis build each commit in a PR separately or just the last one?

I fixed the issue that you pointed out above, then it built but failed elsewhere (with the missing versioning of the new symbols) so I fixed that... now it's back to failing at the above place. Weird.

bagder

I think this looks great, I only have a few comments and questions to ponder.

bagder · 2018-04-20T20:36:28Z

docs/examples/progressfunc.c


  /* under certain circumstances it may be desirable for certain functionality
     to only run every N seconds, in order to do this the transaction time can
     be used */
-  if((curtime - myp->lastruntime) >= MINIMAL_PROGRESS_FUNCTIONALITY_INTERVAL) {
+  if(((curtime - myp->lastruntime) / 1000000) >= MINIMAL_PROGRESS_FUNCTIONALITY_INTERVAL) {


Wouldn't this look nicer if instead the constant MINIMAL_PROGRESS_FUNCTIONALITY_INTERVAL was multiplied with 1000000 ?

bagder · 2018-04-20T20:36:28Z

lib/getinfo.c

@@ -281,6 +281,30 @@ static CURLcode getinfo_offt(struct Curl_easy *data, CURLINFO info,
    *param_offt = (data->progress.flags & PGRS_UL_SIZE_KNOWN)?
      data->progress.size_ul:-1;
    break;
+#if CURL_SIZEOF_CURL_OFF_T == 8


Why limit these to 64bit only?

Okay, since it's a relative timestamp and not absolute, sure, I'll make it 32-bit as well.

bagder · 2018-04-20T20:36:28Z

include/curl/curl.h

+	((off_t) / 1000000L), (long)((off_t) % 1000000L)
+
+#define CURL_FORMAT_TIME_TUPLE \
+	CURL_FORMAT_CURL_OFF_T ".%ld"


These might be handy for an application, but they're also very basic and for very specific and narrow use cases. Does it really add anything to provide them in the public header?

To me they feel more like handy macros for our test suite.

Well, since other format specifiers were exposed in system.h I was trying to follow that model.

bagder · 2018-04-20T20:36:28Z

tests/libtest/lib500.c

-        curl_easy_getinfo(curl, CURLINFO_CONNECT_TIME, &time_connect);
-        curl_easy_getinfo(curl, CURLINFO_PRETRANSFER_TIME, &time_pretransfer);
-        curl_easy_getinfo(curl, CURLINFO_STARTTRANSFER_TIME,
+        curl_easy_getinfo(curl, CURLINFO_NAMELOOKUP_TIME_T, &time_namelookup);


Since you limited the code to extract this time to 64 bit only, won't this code fail hard on 32 bit systems?

Made it 32-bit also.

bagder · 2018-04-20T20:36:28Z

docs/libcurl/symbols-in-versions

@@ -206,9 +206,11 @@ CURLHEADER_SEPARATE             7.37.0
 CURLHEADER_UNIFIED              7.37.0
 CURLINFO_ACTIVESOCKET           7.45.0
 CURLINFO_APPCONNECT_TIME        7.19.0
+CURLINFO_APPCONNECT_TIME_T      7.60.0


Since the feature window has closed now for 7.60.0 and this is a change, we can assume this change can land in 7.61.0.

Changed... the Travis testing is kicking my ass, though... Can't get it to pass the tests 1139 and 1140.

I don't see it in the most recent commit. I see it fails because of this:

lib500.c: In function ‘test’: lib500.c:115:112: error: expected identifier before ‘%’ token fprintf(moo, "namelookup vs connect: %" CURL_FORMAT_CURL_OFF_T ".%06ld " %" CURL_FORMAT_CURL_OFF_T ".%06ld\n",

there's a quote in middle of the ".%06ld " %" which it looks like causes a slew of syntax errors

pprindeville · 2018-04-21T05:23:50Z

Not understanding what's needed here:

test 1139...[Verify that all libcurl options have man pages]

perl  returned 7, when expecting 0
 exit FAILED
== Contents of files in the log/ dir after test 1139
=== Start of file stderr1139
 Missing CURLINFO_APPCONNECT_TIME_T.3
 Missing CURLINFO_CONNECT_TIME_T.3
 Missing CURLINFO_NAMELOOKUP_TIME_T.3
 Missing CURLINFO_PRETRANSFER_TIME_T.3
 Missing CURLINFO_REDIRECT_TIME_T.3
 Missing CURLINFO_STARTTRANSFER_TIME_T.3
 Missing CURLINFO_TOTAL_TIME_T.3
=== End of file stderr1139
test 1140...[Verify the nroff of man pages]

perl  returned 7, when expecting 0
 exit FAILED
== Contents of files in the log/ dir after test 1140
=== Start of file stderr1140
 error: ./../docs/libcurl/curl_easy_getinfo.3:65: referring to non-existing man page CURLINFO_TOTAL_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:71: referring to non-existing man page CURLINFO_NAMELOOKUP_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:77: referring to non-existing man page CURLINFO_CONNECT_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:83: referring to non-existing man page CURLINFO_APPCONNECT_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:89: referring to non-existing man page CURLINFO_PRETRANSFER_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:95: referring to non-existing man page CURLINFO_STARTTRANSFER_TIME_T.3
 error: ./../docs/libcurl/curl_easy_getinfo.3:101: referring to non-existing man page CURLINFO_REDIRECT_TIME_T.3
=== End of file stderr1140

jay · 2018-04-21T05:47:08Z

because it's not documented, you need man pages for each. how is CURLINFO_APPCONNECT_TIME different from CURLINFO_APPCONNECT_TIME_T

pprindeville · 2018-04-21T15:32:49Z

Can we have the *TIME and *TIME_T versions share the same pages?

pprindeville · 2018-05-01T21:52:55Z

I think the PR is good to go. @jay @bagder?

We overload the curl_off_t, if it's 64-bits wide, for extending the API to allow retrieving times in microseconds (which is how they're stored internally).

jay · 2018-05-02T07:16:01Z

I'll check it again it looks like you did more work on it. Even if approved it can't be added until the next feature window, in 2 weeks.

pprindeville · 2018-05-16T19:33:23Z

When does the window reopen?

bagder · 2018-05-17T07:36:49Z

At release, which was yesterday!

bagder · 2018-05-17T11:41:41Z

Thanks!

pprindeville force-pushed the add-integer-times branch 2 times, most recently from da3f93c to cd37983 Compare Apr 15, 2018

pprindeville force-pushed the add-integer-times branch from cd37983 to 941d7bb Compare Apr 16, 2018

pprindeville force-pushed the add-integer-times branch 2 times, most recently from ada1661 to 558174d Compare Apr 19, 2018

pprindeville force-pushed the add-integer-times branch from 558174d to ab269cb Compare Apr 20, 2018

pprindeville force-pushed the add-integer-times branch from ab269cb to 6ebc305 Compare Apr 20, 2018

pprindeville force-pushed the add-integer-times branch 2 times, most recently from eb924bf to 924da0b Compare Apr 20, 2018

bagder reviewed Apr 20, 2018

View changes

pprindeville force-pushed the add-integer-times branch 6 times, most recently from 5f0e463 to 678268f Compare Apr 21, 2018

pprindeville force-pushed the add-integer-times branch 4 times, most recently from f5e2890 to d33b20f Compare Apr 28, 2018

pprindeville force-pushed the add-integer-times branch 2 times, most recently from 5d33f05 to 62a1d5a Compare Apr 28, 2018

add microsecond precise timers for measuring various intervals

450fc14

We overload the curl_off_t, if it's 64-bits wide, for extending the API to allow retrieving times in microseconds (which is how they're stored internally).

pprindeville force-pushed the add-integer-times branch from 62a1d5a to 450fc14 Compare May 1, 2018

bagder closed this in ce2140a May 17, 2018

lock bot locked as resolved and limited conversation to collaborators Aug 15, 2018

add microsecond precise timers for measuring various intervals #2495

add microsecond precise timers for measuring various intervals #2495

pprindeville commented Apr 14, 2018

bagder commented Apr 16, 2018

pprindeville commented Apr 16, 2018

pprindeville commented Apr 19, 2018

bagder commented Apr 19, 2018

pprindeville commented Apr 19, 2018

pprindeville commented Apr 20, 2018

pprindeville commented Apr 20, 2018

bagder left a comment

bagder Apr 20, 2018

bagder Apr 20, 2018

pprindeville Apr 20, 2018

bagder Apr 20, 2018

pprindeville Apr 20, 2018

bagder Apr 20, 2018

pprindeville Apr 20, 2018

bagder Apr 20, 2018

pprindeville Apr 20, 2018

jay Apr 21, 2018

pprindeville Apr 21, 2018

pprindeville commented Apr 21, 2018

jay commented Apr 21, 2018

pprindeville commented Apr 21, 2018

pprindeville commented May 1, 2018

jay commented May 2, 2018

pprindeville commented May 16, 2018

bagder commented May 17, 2018

bagder commented May 17, 2018

add microsecond precise timers for measuring various intervals #2495

add microsecond precise timers for measuring various intervals #2495

Conversation

pprindeville commented Apr 14, 2018

bagder commented Apr 16, 2018

pprindeville commented Apr 16, 2018

pprindeville commented Apr 19, 2018

bagder commented Apr 19, 2018

pprindeville commented Apr 19, 2018

pprindeville commented Apr 20, 2018

pprindeville commented Apr 20, 2018

bagder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pprindeville commented Apr 21, 2018

jay commented Apr 21, 2018

pprindeville commented Apr 21, 2018

pprindeville commented May 1, 2018

jay commented May 2, 2018

pprindeville commented May 16, 2018

bagder commented May 17, 2018

bagder commented May 17, 2018