PTP TIMESTAMPING TEST TOOL OVERVIEW
===================================

ptp_timestamping_test is a standalone tool that can be used independently of DEP
itself to do a definitive check of a network driver's timestamping capabilities.

After installing DEP, this tool should ALWAYS be run before starting DEP for the
first time. This is to ensure that the driver(s) for the selected network interface(s)
are able to properly timestamp all PTP event packets using the required timestamping
mode (i.e. hardware, PTPv1-only hardware or software).

If problems are found with the driver(s) when timestamping in a given mode, DEP
should NOT be started in that mode, as otherwise PTP will either fail to start
or will experience synchronisation issues.

In such a case, in order to run DEP:

- To continue using the given mode, the driver(s) must be updated and the tool
  rerun until problems are no longer reported
- Or, another timestamping mode that shows no problems must be used instead


CONTENTS
========

1. Test types
2. Test requirements
   a. Preliminary check
   b. Full test
3. Running tests
   a. Via dep_check.sh
   b. Running manually
4. Understanding test results
   a. Preliminary check
   b. Full test
   c. Full test output examples
5. Using event + error logging
6. Test behaviour and duration
   a. Test packet receive and send behaviour
   b. Test duration


1. TEST TYPES
=============

There are two types of tests that can be done with the tool:

1) A preliminary check of whether a driver actually supports configuring a
   particular timestamping mode. This test tries to create a network socket
   configured to use the selected mode. The success or failure of this test
   is a much more reliable indicator of configuration support than the output
   of 'ethtool -T', as a driver's reported claims are not always 100% accurate.
   Nevertheless, as a guide to the user the tool gathers and displays the
   equivalent information shown via 'ethtool -T'.

2) A full test. This uses a configured network socket to send and receive
   all PTP event packet types, and checks that they all get timestamped. 

A full test is the only real way to determine if timestamping is fully operational.
A simple preliminary check is NOT a substitute for this.

However, if a timestamping mode fails a preliminary check it means that mode
CANNOT be used (unless the driver can be updated/rectified).


2. TEST REQUIREMENTS
====================

The tool must always be run as root.

a. Preliminary check
--------------------

Running a preliminary check has no extra requirements. This check can be done
at any time.

b. Full test
------------

The device being tested must be connected to a Dante network. And, in addition 
to this device:

- At least two Dante devices (one leader, and at least one follower) must be
  present on the network
- If AES67 and/or site/domain unicast clocking are going to be used, at least
  two PTPv2-capable devices (again, one leader and at least one follower) must
  be present:
  * At least one of these must be a third-party (i.e. non-Dante) device
  * Two Dante devices running in AES67 mode are NOT a suitable substitute, as
    without a third-party device there will be no PTPv2 follower devices

Points of note:

- ALL devices (both Dante and third-party) MUST be in the unmanaged domain (i.e.
  in the default subdomain for PTPv1, and in domain number 0 for PTPv2)
- The tool assumes that all PTP traffic is multicast. Therefore, any follower
  devices configured to use unicast delay requests will not be visible and thus
  cannot be used
- If only one device of each PTPv1/PTPv2 type is present (as a leader), the
  tool can still be run but it will report that the test is incomplete
- Running the test will NOT cause clock synchronisation to be disrupted on the
  network


3. RUNNING TESTS
================

a. Via dep_check.sh
-------------------

dep_check.sh will use the tool to do preliminary checks of all three timestamping
modes for the interface(s) configured in dante.json. Upon completion, it will
display a complete command line(s) (with parameters based on the settings in
dante.json) that the user can run to do a full test(s) for:

- The mode in dante.json, and/or
- The recommended mode (if different to the above), based on the check results

If the timestamping mode set in dante.json fails a preliminary check, dep_check.sh
will show a complete command line the user can run to see the full output of the
check if desired.


b. Running manually
-------------------

Running the tool with the -h option will show how the tool can be used.

As stated above dep_check.sh provides complete command lines for convenience,
however manual usage is fairly straightforward:

- The value for -i is the interface under "network" in dante.json
- The timestamping mode can be set as required
- If "dsaTaggedPackets" in dante.json is set to true, -dsa must be specified
- If -dsa is used along with either -hw or -hwv1, a value must be supplied
  for -hwi. This is the entry under "clock.hardwareInterfaces" in dante.json
  that corresponds to the value supplied for -i
- By default, the tool will run a full test. Specifying -c will result in the
  tool running only a preliminary check


4. UNDERSTANDING TEST RESULTS
=============================

a. Preliminary check
--------------------

The result will be a simple pass or fail, depending on whether a network socket
could be configured with the chosen settings.

The tool will also display the timestamping capabilities reported by the driver
(the same ones shown via 'ethtool -T') and indicate whether these match what the
user wants. HOWEVER, even if the reported capabilities do not match the check
still proceeds to do a network socket setup by explaining that it MAY succeed-
in spite of the mismatch.

b. Full test
------------

For timestamping to be deemed operational, the driver MUST be able to timestamp:

- Both PTP event packet types (SYNC and DELAY_REQ)
- In both directions

Furthermore, in order to be able to run DEP:

- To use Dante audio, PTPv1 timestamping must fully work
- To use AES67 with a third-party device and to enable site/domain unicast
  clocking, PTPv2 timestamping must also work

The test output provides a terse summary of whether timestamping for a particular
version of PTP was:

- All OK
- Found to have errors for a particular packet type + timestamping direction
- Not (completely) tested due to lack of devices present

Refer to the next section for some example full test outputs.

The tool will display detailed results for the packet version + type(s) that
encountered any timestamping errors. These results provide specific information
on the number of packets sent/received, the number successfully timestamped and
how many attempts resulted in errors. This can be useful when debugging a network
driver. If however, yet more specific details are required about the types of
errors and/or the relative times at which packet events + timestamping operations
occur, the logging option can be used. This is discussed further below.

c. Full test output examples
----------------------------

On a device with a working driver, assuming both PTPv1 and PTPv2 leaders and
followers are present the output will look like the following:

    # ./ptp_timestamping_test -i eno2 -hw
    Using interface eno2, with hardware timestamping
    Checking PHC resolution...
    Checking PHC read speed... 4283ns (approx.)

    Testing PTP timestamping...

    Testing v1 SYNC packets... OK
    Testing v1 DELAY_REQ packets... OK
    Testing v2 SYNC packets... OK
    Testing v2 DELAY_REQ packets... OK


    TEST SUMMARY
    ============

    PTPv1 all OK
    PTPv2 all OK

Points of note:

- When using hardware timestamping, the first thing the tool does is perform some
  measurements of the PHC (PTP Hardware Clock)
- The approximate read speed of the PHC should be noted. NICs that perform
  timestamping at the PHY instead of the MAC layer will exhibit very large values
  here (greater than about 100,000ns), and these are typically unsuitable for use
  with DEP as the slow read times will adversely affect its PTP accuracy

The outputs below were produced on a board which has problems with hardware
timestamping (but not software timestamping). Also:

- The Dante network used had two devices
- One of the devices had AES67 turned on (and so there was a PTPv2 leader
  present, but no PTPv2 followers)

Software timestamping result:

    # ./ptp_timestamping_test -i eth1
    Using interface eth1, with software timestamping

    Testing PTP timestamping...

    Testing v1 SYNC packets... OK
    Testing v1 DELAY_REQ packets... OK
    Testing v2 SYNC packets... OK
    Testing v2 DELAY_REQ packets... none detected


    TEST SUMMARY
    ============

    PTPv1 all OK
    PTPv2 only partially tested - no follower devices detected

Hardware timestamping result:

    # ./ptp_timestamping_test -i eth1 -hw
    Using interface eth1, with hardware timestamping
    Checking PHC resolution...
    Checking PHC read speed... 3928ns (approx.)

    Testing PTP timestamping...

    Testing v1 SYNC packets... OK
    Testing v1 DELAY_REQ packets... Tx problems
    Testing v2 SYNC packets... OK
    Testing v2 DELAY_REQ packets... none detected


    TEST SUMMARY
    ============

    PTPv1 SYNCs OK, errors found with DELAY_REQs - details below
    PTPv2 only partially tested - no follower devices detected


    DETAILED RESULTS
    ================

    v1 DELAY_REQ
    ------------
    Rx packet receive limit: 5
    Rx packets received: 5
    Rx packets timestamped: 5
    Rx packet timestamping errors: 0
    Rx packets from additional followers received: 0
    Rx packets from additional followers timestamped: 0
    Rx packet timestamping errors for additional followers: 0

    Tx packets sent: 5
    Tx packets timestamped: 0
    Tx packets with slow timestamps: 0
    Tx packet timestamping errors: 0

The detailed results show that:

- All Rx packets (up to the default receive limit of 5) from the follower the
  test chose were successfully timestamped
- No other followers were present (and so DELAY_REQs from those were not received)
- No Tx packets were timestamped, HOWEVER there were no errors. This is an
  indication that the driver did not attempt to timestamp any outgoing v1 DELAY_REQs

For some drivers, timestamping operations do indeed take place BUT result in
errors. The following detailed results are from a board whose driver produces
these:

    DETAILED RESULTS
    ================

    v1 SYNC
    -------
    Rx packet receive limit: 20
    Rx packets received: 20
    Rx packets timestamped: 0
    Rx packet timestamping errors: 20
    Rx FOLLOW-UPs received: 20
    Multiple leaders detected: no

    Tx packets sent: 20
    Tx packets timestamped: 20
    Tx packets with slow timestamps: 0
    Tx packet timestamping errors: 0

These details show that:

- All 20 (the default receive limit) SYNC packets were indeed received, however
  none were timestamped
- Timestamping was attempted each time, but always resulted in an error
- THere were no issues timestamping outgoing packets

The details in a SYNC report also include network information that may be of
interest to the user:

- Most PTP leaders issue FOLLOW-UP packets after each SYNC. These do not need
  to be timestamped, however the tool listens for these and checks that they
  indeed come from the leader sending the SYNCs. If no FOLLOW-UPs are received,
  this may indicate either a problematic leader OR the presence of a leader
  using the one-step rather than the more common two-step synchronisation method
- If the tool detects more than one leader on the network, it will be indicated
  here


5. USING EVENT + ERROR LOGGING
==============================

While the full test outputs above point to which timestamping operations fail to
work, in some cases (e.g. if trying to debug a network driver) a detailed timeline
of packet and timestamping events, along with the specific errors that occurred,
can be useful.

By using the -l option along with a file to log to, the tool will produce its
normal output but also place all events into that file. The logs will contain:

- The start and end of each test
- Every packet receive and send
- Every successful timestamp read (Rx/Tx), along with the timestamp value
- If an error reading a timestamp occurs, a description of the error (ancillary
  data truncated, no timestamp information, insufficient timestamp data)
- If timestamping is not taking place at all (as in the example above, for Tx),
  the log will be missing these events

Each line also starts with a timestamp (seconds and nanoseconds), which is the
value of CLOCK_REALTIME at the moment an event or error was logged.

NOTE: When running a hardware timestamping test, you will notice that the first
test in the log is a PTPv1 "scratch" test. This is a throwaway test done at the
start that, on some devices, will exhibit odd Tx timestamping and/or errors.
A scratch test is always done because in some cases a driver will only start
reporting correct timestamps after a few initial socket and timestamp operation
failures. This way, the actual tests of interest are not affected. The scratch
test logs can be safely ignored (although initial socket behaviour after setup
may be of interest to some).


6. TEST BEHAVIOUR AND DURATION
==============================

a. Test packet receive and send behaviour
-----------------------------------------

The test uses packet receive limits for each PTP event packet type (for both
protocol versions). By default, these are:

- 20 SYNCs from the first (and ideally only) leader seen
- 5 DELAY_REQs from the first follower detected

When transmitting packets:

- For a SYNC test, each received SYNC is copied and sent out
- For a DELAY_REQ, packet sends are throttled if required so that not less than
  0.25 seconds can elapse before a DELAY_REQ is sent. The sent packet is a copy
  of the last received DELAY_REQ. DELAY_REQ tests count packets from the first
  follower detected but otherwise receive and timestamp DELAY_REQs from any and
  all followers on the network

Each test only ends when:

- The receive limit has been reached, and
  - For SYNCs, that same number has been sent out
  - For DELAY_REQs, a minimum of that same number has been sent
- Or, the test times out waiting for a packet to arrive
  - For SYNCs, the test will wait 2 seconds before timing out
  - for DELAY_REQs, this figure is 8 seconds (because it can be up to 7.5 seconds
    between packet arrivals)
- Or, a socket error occurs (these should NOT occur unless there is a system
  issue or network connectivity is suddenly lost)

NOTE: If the test sees no SYNC packets for a particular PTP version, it will
automatically skip the DELAY_REQ test for that version as, in the absence of a
leader, there will not be followers sending DELAY_REQs. If no SYNCs are detected
despite the device being on a populated Dante network (or one with PTPv2 devices),
it may be the case that multicast PTP traffic is not being sent to this device.
If this happens, the network and/or switch(es) should be checked to ensure that
the device receives multicast PTP.

b. Test duration
----------------

At the default receive limits, a test will last about 25 seconds typically:

- SYNCs from Dante leaders are normally sent every 0.25 seconds
- DELAY_REQs from Dante followers are sent at varying intervals, but on average
  are around 4-5 seconds

Note: third-party AES67 devices may have their own packet send intervals

To run a shorter or longer test, the -nsy and -ndr tool options can be used:

- If a driver is known to have working timestamping for a particular packet
  type but not another, the limit for the working type can be reduced
- Setting the limit to 0 will skip that packet type entirely (and the full test
  output will say so, and also warn of an incomplete test)
- On the other hand, it may be the case that a driver only begins to exhibit
  problems after running for a while. In this case, the limits can be increased
  using the send intervals above as a rough guide for determining the approximate
  test duration
