Datalogging

Contents

Datalogging#

Usage of the SDK watchables as described in Accessing variables is done through polling and the update rate is limited by the bandwidth of the communication link between the server and the device (Serial, TCP, CAN, etc). Therefore, using Python to monitor a value may not reliably capture a rapid event in the firmware, as the interval between two polling events could span several milliseconds.

Datalogging (or Embedded Graphs) addresses this issue by instructing the device to record a specific set of watchables in a circular buffer until a certain event occurs. This effectively turns the embedded device into a scope.

The condition that stop the circular acquisition is called the Trigger condition and may be of many type. Once the trigger condition is met, the acquisition finishes (either immediately or after a specified amount of time) and the data is returned to the server to be saved into a database. A client can then download and display that data under the form of a graph.

Configuring the datalogger#

The first step before configuring the datalogger is knowing what the device is capable of in terms of datalogging. We need information such as :

  • Is datalogging even supported?

  • What are the sampling rates?

  • What is the size of the buffer?

  • How many signal can I log?

  • etc

This information can be obtained using the ScrutinyClient.get_datalogging_capabilities method, which returns a DataloggingCapabilities object.

ScrutinyClient.get_datalogging_capabilities()[source]#

Gets the device capabilities in terms of datalogging. This information includes the available sampling rates, the datalogging buffer size, the data encoding format and the maximum number of signals.

Raises:

OperationFailure – If the request to the server fails

Returns:

The datalogging capabilities

Return type:

DataloggingCapabilities

class scrutiny.sdk.datalogging.DataloggingCapabilities[source]#

(Immutable struct) Tells what the device is able to achieve in terms of datalogging

buffer_size: int#

Size of the datalogging buffer

encoding: DataloggingEncoding#

The encoding of data

max_nb_signal: int#

Maximum number of signal per acquisition (including time if measured)

sampling_rates: List[SamplingRate]#

List of available sampling rates

Configuring and arming the datalogger is possible with the ScrutinyClient.start_datalog method which takes a sdk.DataloggingConfig as sole argument.

ScrutinyClient.start_datalog(config)[source]#

Requires the device to make a datalogging acquisition based on the given configuration

Parameters:

config (DataloggingConfig) – The datalogging configuration including sampling rate, signals to log, trigger condition and operands, etc.

Raises:

  • OperationFailure – If the request to the server fails

  • ValueError – Bad parameter value

  • TypeError – Given parameter not of the expected type

Returns:

A DataloggingRequest handle that can provide the status of the acquisition process and used to fetch the data.

Return type:

DataloggingRequest

class scrutiny.sdk.datalogging.DataloggingConfig[source]#

A datalogging acquisition configuration. Contains all the configurable parameters for an acquisition

__init__(sampling_rate, decimation=1, timeout=0.0, name='')[source]#

Creates an instance of DataloggingConfig

Parameters:

  • sampling_rate (int | SamplingRate) – The acquisition sampling rate. Can be the sampling rate ID in the device or an instance of SamplingRate gotten from the DataloggingCapabilities returned by ScrutinyClient.get_datalogging_capabilities

  • decimation (int) – The decimation factor that reduces the effective sampling rate

  • timeout (float) – Timeout to the acquisition. After the datalogger is armed, it will forcefully trigger after this amount of time. 0 means no timeout

  • name (str) – Name of the configuration. Save into the database for reference

Raises:

  • TypeError – Given parameter not of the expected type

  • ValueError – Given parameter has an invalid value

add_axis(name)[source]#

Adds a Y axis to the acquisition.

Parameters:

name (str) – The name of the axis, for display purpose.

Returns:

An AxisDefinition object that can be assigned to a signal when calling add_signal()

Return type:

AxisDefinition

add_signal(signal, axis, name=None)[source]#

Adds a signal to the acquisition

Parameters:
Raises:

  • IndexError – Invalid axis index

  • ValueError – Bad parameter value

  • TypeError – Given parameter not of the expected type

Return type:

None

configure_trigger(condition, operands=None, position=0.5, hold_time=0)[source]#

Configure the required conditions to fire the trigger event

Parameters:

  • condition (TriggerCondition) – The type of condition used for triggering the acquisition.

  • operands (List[WatchableHandle | float | str] | None) – List of operands. Each operands can be a constant number (float), the path to a variable/rpv/alias (str) or a WatchableHandle given by ScrutinyClient.watch(). The number of operands depends on the trigger condition

  • position (float) – Position of the trigger event in the datalogging buffer. Value from 0 to 1, where 0 is leftmost, 0.5 middle and 1 rightmost.

  • hold_time (float) – Time in seconds that the trigger condition must evaluate to true before firing the trigger event.

Raises:

  • ValueError – Bad parameter value

  • TypeError – Given parameter not of the expected type

Return type:

None

configure_xaxis(axis_type, signal=None, name=None)[source]#

Configures the X-Axis

Parameters:
Raises:

  • ValueError – Bad parameter value

  • TypeError – Given parameter not of the expected type

Return type:

None

class scrutiny.sdk.datalogging.TriggerCondition[source]#

(Enum) The type of trigger condition to use.

AlwaysTrue = 'true'#

Always true. Triggers immediately after being armed

ChangeMoreThan = 'cmt'#

X=(Operand1[n]-Operand1[n-1]); |X| > |Operand2| && sign(X) == sign(Operand2)

Equal = 'eq'#

Operand1 == Operand2

GreaterOrEqualThan = 'get'#

Operand1 >= Operand2

GreaterThan = 'gt'#

Operand1 > Operand2

IsWithin = 'within'#

|Operand1 - Operand2| < |Operand3|

LessOrEqualThan = 'let'#

Operand1 <= Operand2

LessThan = 'lt'#

Operand1 < Operand2

NotEqual = 'neq'#

Operand1 != Operand2

class scrutiny.sdk.datalogging.XAxisType[source]#

(Enum) Represent a type of X-Axis that a user can select

IdealTime = 'ideal_time'#

Time deduced from the sampling frequency. Does not require space in the datalogging buffer. Only available for fixed frequency loops

Indexed = 'index'#

No signal will be captured for the X-Axis and the returned X-Axis data will be the index of the samples

MeasuredTime = 'measured_time'#

Time measured by the device. Requires space for a 32 bits value in the datalogging buffer

Signal = 'signal'#

X-Axis is an arbitrary signal, not time.

class scrutiny.sdk.datalogging.AxisDefinition[source]#

(Immutable struct) Represent an axis

axis_id: int#

A unique ID used to identify the axis

name: str#

The name of the axis. Used for display

class scrutiny.sdk.datalogging.SamplingRate[source]#

(Immutable struct) Represent a sampling rate supported by the device

identifier: int#

The unique identifier of the sampling rate. Matches the embedded device index in the loop array set in the configuration

name: str#

Name for display

class scrutiny.sdk.datalogging.DataloggingRequest[source]#

Handle to a request for a datalogging acquisition. Gets updated by the client and reflects the actual status of the acquisition

wait_for_completion(timeout=None)[source]#

Wait for the acquisition to be triggered and extracted by the server. Once this is done, the acquisition_reference_id will not be None anymore and its value will point to the database entry storing the data.

Params timeout:

Maximum wait time in seconds. Waits forever if None

Raises:

  • TimeoutException – If the acquisition does not complete in less than the specified timeout value

  • OperationFailure – If an error happened that prevented the acquisition to successfully complete

Parameters:

timeout (float | None) –

Return type:

None

fetch_acquisition(timeout=None)[source]#

Download and returns an acquisition data from the server. The acquisition must be complete

Params timeout:

Timeout to get a response by the server in seconds. Uses the default timeout value if None

Raises:

  • TimeoutException – If the server does not respond in time

  • OperationFailure – If the acquisition is not complete or if an error happens while fetching the data

Returns:

The DataloggingAcquisition object containing the acquired data

Parameters:

timeout (float | None) –

Return type:

DataloggingAcquisition

wait_and_fetch(timeout=None, fetch_timeout=None)[source]#

Do successive calls to wait_for_completion() & fetch_acquisition() and return the acquisition

Params timeout:

Timeout given to wait_for_completion()

Params fetch_timeout:

Timeout given to fetch_acquisition()

Raises:
Returns:

The DataloggingAcquisition object containing the acquired data

Parameters:

  • timeout (float | None) –

  • fetch_timeout (float | None) –

Return type:

DataloggingAcquisition

property acquisition_reference_id: str | None#

The unique ID used to fetch the acquisition data from the server. Value is set only if request is completed and succeeded. None otherwise

Example#

with client.connect(hostname, port):
    var1 = client.watch('/a/b/var1')
    var2 = client.watch('/a/b/var2')

    config = sdk.datalogging.DataloggingConfig(sampling_rate=0, decimation=1, timeout=0, name="MyGraph")
    config.configure_trigger(sdk.datalogging.TriggerCondition.GreaterThan, [var1, 3.14159], position=0.75, hold_time=0)
    config.configure_xaxis(sdk.datalogging.XAxisType.MeasuredTime)
    axis1 = config.add_axis('Axis 1')
    axis2 = config.add_axis('Axis 2')
    config.add_signal(var1, axis1, name="MyVar1")
    config.add_signal(var2, axis1, name="MyVar2")
    config.add_signal('/a/b/alias_rpv1000', axis2, name="MyAliasRPV1000")

    request = client.start_datalog(config)

    timeout = 60
    print(f"Embedded datalogger armed. Waiting for MyVar1 >= 3.14159...")
    try:
        request.wait_for_completion(timeout)    # Wait for the trigger condition to be fulfilled
    except sdk.exceptions.TimeoutException:
        print(f'Timed out while waiting')

    if request.completed:   # Will be False if timed out
        if request.is_success:
            acquisition = request.fetch_acquisition()
            filename = "my_acquisition.csv"
            acquisition.to_csv(filename)
            print(f"Acquisition [{acquisition.reference_id}] saved to CSV format in {filename}")
        else:
            print(f"The datalogging acquisition failed. Reason: {request.failure_reason}")

Reading an acquisition after completion#

A DataloggingAcquisition represent what the datalogger has captured. It contains multiple data series, some acquired by the device and some generated by the server (like the ideal time data series crafted from the sampling frequency).

The content of a data series is converted to double-precision floating point values so they can be more easily plotted or manipulated,

Note

Double precision (64 bits) floating point values have a mantissa of 52 bits, therefore, only 64bits integers greater than 2^52 may lose precision during that conversion. The choice of auto-converting the values seemed to offer more advantages than disadvantage in the majority of real use cases.

Data series are tied to an axis. There can be a single X-Axis and multiple Y-Axis.

A DataloggingAcquisition can be obtained by calling either

Once a DataloggingAcquisition is obtained, the DataloggingAcquisition.to_csv() can be used to export the data

class scrutiny.core.datalogging.DataloggingAcquisition[source]#

Represent an acquisition of multiple signals

reference_id: str#

ID used to reference the acquisition in the storage

firmware_id: str#

Firmware ID of the device on which the acquisition has been taken

acq_time: datetime#

Time at which the acquisition has been taken

xdata: DataSeries#

The series of data that represent the X-Axis

name: str | None#

A display name associated with the acquisition for easier management

ydata: List[DataSeriesWithAxis]#

List of data series acquired

trigger_index: int | None#

Sample index of the trigger

firmware_name: str | None#

The firmware name taken from the metadata of the SFD loaded when the acquisition was made. None if it is not available

to_csv(filename)[source]#

Export a DataloggingAcquisition content to a csv file

Parameters:

filename (str) – The file to write to

Return type:

None

class scrutiny.core.datalogging.DataSeriesWithAxis[source]#

(Immutable struct) Dataseries tied to an axis definition

series: DataSeries#

The dataseries containing the acquisition data

axis: AxisDefinition#

The Y-Axis to which the dataseries is bound to

class scrutiny.core.datalogging.DataSeries[source]#

A data series is a series of measurement represented by a series of 64 bits floating point value

name: str#

The name of the data series. Used for display

logged_element: str#

The server element that was the source of the data. Path to a variable, alias or RPV (Runtime Published Value)

data: List[float]#

The data stored as a list of 64 bits float

Fetching an acquisition from the database#

The server maintain a local sqlite database of all the acquisition captured. In most use case relevant for this SDK, a user will want to download an acquisition that just got triggered, but it is also possible to browse the database and download past acquisitions (which is also possible through the CLI)

Each acquisition has a unique ID called the reference_id. On can list the acquisition available by calling ScrutinyClient.list_stored_datalogging_acquisitions and once the reference_id is known, it can be passed to ScrutinyClient.read_datalogging_acquisition()

ScrutinyClient.list_stored_datalogging_acquisitions(timeout=None)[source]#

Gets the list of datalogging acquisition stored in the server database

Parameters:

timeout (float | None) – The request timeout value. The default client timeout will be used if set to None Defaults to None

Raises:

OperationFailure – If fetching the list fails

Returns:

A list of database entries, each one representing an acquisition in the database with reference_id as its unique identifier

Return type:

List[DataloggingStorageEntry]

ScrutinyClient.read_datalogging_acquisition(reference_id, timeout=None)[source]#

Reads a datalogging acquisition from the server storage identified by its reference ID

Parameters:

  • reference_id (str) – The acquisition unique ID

  • timeout (float | None) – The request timeout value. The default client timeout will be used if set to None Defaults to None

Raises:

OperationFailure – If fetching the acquisition fails

Returns:

An object containing the acquisition, including the data, the axes, the trigger index, the graph name, etc

Return type:

DataloggingAcquisition

class scrutiny.sdk.datalogging.DataloggingStorageEntry[source]#

(Immutable struct) Represent an entry in datalogging storage

reference_id: str#

Database ID used to uniquely identified this acquisition

firmware_id: str#

Firmware ID of the device that took the acquisition

name: str#

Name of the acquisition. For display purpose

timestamp: datetime#

Date/Time at which the acquisition was captured

firmware_metadata: SFDMetadata | None#

The metadata of the firmware used by the device if available

Example#

with client.connect(hostname, port):
    entries = client.list_stored_datalogging_acquisitions()
    print(f"The server has {len(entries)} acquisition stored")
    for entry in entries:
        dt = entry.timestamp.strftime(r"%Y-%m-%d %H:%M:%S")
        print(f"[{entry.reference_id}] {entry.name} taken on {dt}")
    assert (len(entries) > 0, "Cannot fetch first datalogging acquisition")
    acquisition = client.read_datalogging_acquisition(entries[0].reference_id)   # Read the first one
    print(f"Acquisition {acquisition.name} [{acquisition.reference_id}] downloaded and has {len(acquisition.ydata)} signals on the Y-Axis. ")
    acquisition.to_csv("myfile.csv")
Contents