[PSA] Make packet arrival time readable in standard way

[jafingerhut]

This is useful for things like INT, and other features.

I think it might be non-controversial to make at least the time that the packet arrived at a system, or perhaps the time that it started being parsed, available in a standard location, e.g. a metadata field. It would probably be nice if a separate value were also available in egress processing, indicating the time the packet started egress processing (after any queueing delay spent in the PRE queue was finished).

Details that we could try to standardize across implementations, or at least have constants defined in the architecture that let an implementation know what the values are for a particular implementation:

• number of bits in the time stamp
• rate that the time stamp increments (e.g. increments one time every clock cycle at 1.5 GHz, or 1.2 GHz, etc. whatever the clock rate of a device is). This seems difficult to make uniform across implementations if the precision will be in clock cycles. An application that did not need such high precision can easily discard least significant bits, but converting to exact multiples of microseconds or milliseconds would not be possible if implementations varied in this way, unless they also permitted a way to do the conversion, e.g. allowed multiplication and/or division by arbitrary constants, or had a built-in conversion function specifically for converting to a particular set of useful units required by some standards.

[jnfoster]

I agree.

[vgurevich]

+1

In terms of details, I think we should define a standard type, e.g. timestamp_t (or whatever, pending the discussion on naming convention) that each vendor can define as bit.

I am not sure whether standardizing N or the precision will be feasible, but, as @jafingerhut suggested, we can agree that an implementation should provide a couple of functions, to convert a platform-specific time interval to nanoseconds and back and that's probably all that's needed.

[jnfoster]

I know Hakim Weatherspoon at Cornell has thought about this issue (his group had a demo at the May 2017 P4 Workshop). We could ping him for his input.

[jafingerhut]

@vgurevich Note that the P4_16 spec explicitly allows an implementation to only permit multiplication and division by compile-time constants that are a power of 2 (i.e. equivalent to shifts of a bit vector). That is not enough to permit one to convert nanoseconds to microseconds, for example. If some RFC somewhere suggests or requires having time values in units that are not power-of-2 related to nanoseconds, getting microseconds could not be done portably.

[vgurevich]

@jafingerhut

Perhaps, I was not specific enough: an implementation should provide a couple of extern functions that would perform the conversion. I do understand that we do not have multiplication/division. I thought that's what you suggested in the last sentence as "built-in conversion functions".

[jafingerhut]

@vgurevich Agreed. You said that only converting to/from nanoseconds might be enough. I am suggesting that it might not be enough to be useful for existing standards. I don't have a particular standard in mind, but can look for one if we need motivation.

[hanw]

The counter values (timestamp) also need to be synchronized among multiple devices (and to an external reference clock) for them to be meaningful. AFAIK, the synchronization is done using the clock counter incrementing at local clock speed. Maybe we should consider PTP as a motivating example.

[jafingerhut]

@hanw I know I often advocate for more stuff to be controllable from P4 rather than less, but I would suggest that whether synchronization between devices like PTP is performed or not should be implementation-specific, i.e. a PSA implementation is not required to implement time synchronization between devices. Cycle-granularity time stamps can be useful for some applications even without tight time synchronization. Not all applications, I realize, but some.

For example, exporting Netflow/sFlow records from a switch can be useful, even if its timestamps are not tightly synchronized with surrounding devices.

[vgurevich]

I hear you, @jafingerhut

I would think that providing either function (i.e. unit2ns() or unit2us()) to work at runtime would probably be very difficult. In other words, if your ingress timestamp comes in some platform-specific units, but your protocol forces you to output that (or a difference of two timestamps) in microseconds or nanoseconds, then you would probably be screwed unless you can really do high-precision division/multiplication. That type of an extern can probably be only optional.

However, I think, a more common case is when you are trying to encode a timeout or some other time difference constant so that you can compare a time interval against it. That conversion therefore can be done at compile-time by the compiler or at run-time by the control-plane API. That one should be equally easy to do for nanoseconds or microseconds (or anything else) and that's what I do not mind mandating.

[hanw]

@jafingerhut At a risk of side-tracking the original question, I believe PTP can be implemented in P4, if PSA exposes the capability to update the local free-running counter with a P4 program. To me, exposing read/write capability to the local free-running clock counter is sufficient to both timestamping and clock synchronization applications.

[cc10512]

I am concerned of prescribing a certain bit precision for the timestamps, since it may make the mapping to targets harder. Shouldn't this be a target specific type? That is, a type that needs to be defined by the target backend that implements PSA.

Sounds like if we define this as an extern it will be part of the library of externs we discussed on Monday. However, I would like to avoid introducing "optional" externs. Targets should be defining architectures derived from PSA for externs that are not in PSA but are defined in the library. For PSA the externs are either supported or not.

[jafingerhut]

@hanw At the further risk of additional sidetracking, I would guess that implementing the time synchronization methods you are thinking of will require multiplication and/or division by arbitrary quantities that are not powers of 2, and a particular ASIC that is near and dear to both you and @vgurevich might not be able to do those, from his comments. (or if it can do them, it is because you implement it with many ALU shift and add operations that implement multiplication/division)

I am sure some P4 implementations will be able to do such calculations at packet forwarding time, but not all.

[jafingerhut]

First draft of a proposed addition to the PSA spec for timestamps is available on this pull request: #366

[jafingerhut]

The latest master version of the PSA draft document as of the time of writing this command has a "Timestamps" section proposing how timestamps work. They currently do not propose any way to convert these implementation-specific timestamps to some external time reference or units like nanoseconds, microseconds, etc. as so far such proposals have not met with any kind of general agreement. A separate issue/pull request for such a proposal, if it is desired as part of the PSA, is welcome.