# Monitoring

Monitoring is a process of checking if the observed behavior of a component conforms to a behavioral specification. Behavioral specifications can be interface models and component models. In case of interface model the process is interface monitoring: a check if the behavior of a component that provides the interface conforms to the interface definition. In case of component model the process is component monitoring: a check if the behavior of a component conforms to the functional, time and data constraints in the model.

To perform monitoring with the CommaSuite framework, the following ingredients are required:

- Behavioral model (.interface or .component file). The model in turn imports interface and signature definitions.

- One or more trace files with events (calls to commands and signals, notifications and replies to commands) observed during a component execution. Traces are stored in files with extension .events. It is not recommended to create such files manually. They are usually obtained via network sniffing or from execution logs. Trace file names have to start with a letter and have to contain only letters, digits and underscore ("_").

In order to perform monitoring, follow the steps:

1. Include one or more tasks for monitor generation in the CommaSuite project file. A monitoring task has to specify how the trace files are obtained. Trace files can be listed, a folder with trace files can be given, or they can be produced by another generation task. Furthermore, the task has to indicate the behavioral model (interface or component). In the example below a single file with traces is used (TraceITest.events) for interface monitoring:

       Generate Monitors {
           iTestMonitor for interface ITest {
               trace files
               "TraceITest.events"
           }
       }

   The following example shows a task that uses a directory (named trace_files) with traces for component monitoring:

       Generate Monitors {
           monitorTask for component Imaging {
               trace directories
               "trace_files"
           }
       }

   If the trace files are produced by another task then refer to the task:

       Generate Monitors {
           iTestMonitor for interface ITest {
               trace files
               task traceProducer
           }
       }

   Trace directories have to contain trace files with unique names. If two directories contain distinct files with the same name an error will be indicated.

   During monitoring all time and data constraints defined in the interface or component are checked over the traces. In some cases users may need to skip the check of time and data constraints due to various reasons, for example when the real hardware is not present yet thus making the execution environment different than the expected one. Constraint checks can be skipped by including one of the instructions skip time constraints, skip data constraints, skip constraints. The following example monitoring task skips both time and data constraints:

       Generate Monitors {
           iTestMonitor for interface ITest {
               skip constraints

               trace files
               task traceProducer
           }
       }

   There are certain situations when the order of the events in the trace files is different than the order they are observed at the client or the server side. This usually occurs when network sniffing is used to collect the traces. It may happen that notifications are observed between a call and its reply whereas at the client side, the notifications are expected after the reply. In these situations, an instruction for reordering of trace events can be given in the monitoring task:

       Generate Monitors {
           iTestMonitor for interface ITest {
               reorder

               trace files
               task traceProducer
           }
       }

   For a given pair of client and server, the 'reorder' instruction moves all notifications observed between a command and its reply after the reply and modifies the notifications' timestamps to be the same as the one of the reply.

In case of component monitoring, interface monitoring is performed as well for the interfaces on each component port. Every event used in a component instance is first checked by the corresponding interface monitor and after that passed to the component monitor. The component monitor checks if functional, data and time constraints are satisfied. . Generate the monitoring code and execute it. After a right mouse button click on the project file, select 'Run As'. Select: "Generate and execute monitoring". As a result, all generator tasks will be executed followed by execution of all the monitoring tasks. Results are located in the folder "comma-gen" (explained further on this page).

## Generation of Monitoring Code and Namespaces

When monitoring code is generated for interfaces and components all the used constructs like types, signatures, interfaces, etc. have to be accessible from the project in which the monitoring is performed. Some of these constructs can be located in files in other Eclipse projects.

When explicit file imports are used, the transitive import mechanism ensures that all these elements are accessible and can be loaded. This may not be the case when namespace imports are used. If the models used for monitoring use namespace imports, the Eclipse project has to refer to all the projects that contain the models contributing to the used namespaces. If these dependencies are not properly set, the generated code may miss some classes. This will be manifested by a compilation error.

When monitoring based on namespace imports is performed from the CommaSuite command line tool, then the argument modelpath has to be used.

## Inspecting the Results of Monitoring

Monitoring results unite the results from all traces in all monitoring tasks in the given project file.

For a given trace:

- in case of interface monitoring, all connections in the trace over the given interface are monitored

- in case of component monitoring, all instances of the given component model are monitored. Component monitoring process checks if all functional constraints hold for a set of traces. The process of checking a functional constraint is done in parallel with interface monitoring, that is, a check if the given trace satisfies all the interfaces at the component ports. A port associated to multiton interface is monitored by creating a monitor for each connection. A port of singleton interface uses a single monitor for all the connections

- if trace error occurs (this is a syntax error in the events file) all monitoring stops and the results obtained so far are reported along with the trace error

For interface monitoring, we may observe:

- interface error: a violation of the behavior given in interface state machines. Monitoring of the corresponding connection stops

- interface warning: a violation of interface time/data/generic constraints. Monitoring of the connection continues

For component monitoring, we may observe:

- component error: a violation of a functional component constraint. Monitoring of this constraint stops

- component warning: a violation of a component time/data/generic constraint. Monitoring of the component instance continues

- port error: interface error in a connection to a component port. Monitoring of functional and time/data constraints for the corresponding instance stops. Interface monitoring of the remaining port connections continues

- port warning: interface warning in a connection to a component port. Monitoring of the component instance continues

In order to inspect the monitoring results:

1. Navigate to folder comma-gen. For every monitoring task, a folder with the name of the task is created. A summary of the results from all tasks is stored in JSON format in file tasksResults.json. Furthermore, a web page dashboard.html is automatically created. It contains an overview of the results per trace. If you want more detailed information about the results continue to step 2. Every task results folder has the following structure:

   - One folder for each trace file, named after the file name

   - Folder statistics that contains information about the checking of the time and data constraints (if any)

2. Inspect the results for a particular trace file by opening the corresponding folder. Results are structured differently for interface and component monitoring.

   - Interface monitoring. For every connection (usage of an interface by a unique pair of client and server) there will be a folder with name $ServerName$_$ClientName$_$Interface$. The folder has at least two files: $ServerName$_$ClientName$_$Interface$_trace.txt that contains the processed trace and $ServerName$_$ClientName$_$Interface$_results.txt that contains a summary of the monitoring results for the connection. If errors and warnings are detected they will result in additional files.

     In case of detected issues, they are described by giving the context and their exact nature. Errors and warnings are also described in separate files that can be visualized in the PlantUML view.

     Visualize the content of .plantuml files (if any) in the PlantUML view to check the description of the error/warning. The issue is reported in a UML sequence diagram that gives the path to the error and a description.

   - Component monitoring. For every component instance in the given trace file there will be a folder named after the component instance. Each such folder further contains one folder per component port (named after the port), one folder per functional constraint (named after the constraint), and one folder named time_data that contains the results of checking the time and data constraints in the component model (if any). The folders derived from the ports contain the results of the interface monitoring on the given port. The results in all the folders are structured similarly to the case of interface monitoring: one text file with a summary of the results, one text file with the processed trace (up to the occurrence of an error if any), and possibly many files with error/warning descriptions.

## Interface Monitoring and Non-Determinism

Interface specifications support non-determinism by allowing multiple ways to react to a given trigger (check the interface language and the "OR" clause in particular). The presence of non-determinism affects how monitoring is performed and how monitoring results are reported.

In general, when a non-determinism is detected during monitoring, the monitor will maintain and explore all execution paths. As more events from the trace are checked by the monitor, some execution paths may be identified as erroneous and therefore discarded. If all paths at a given moment do not allow the occurrence of an event then a monitoring error is reported and contextual information for all the paths will be shown.

We give an example of monitoring error in case of non-determinism and explain the structure of the reported results.

Imagine the interface of a simple vending machine that accepts coins and delivers products on user request. When a coin is inserted, the machine checks first if this is a valid coin and then reports the outcome to the user together with the value of possibly updated credit. If the machine is in error state then the coin is simply returned. The code snippet shows only the transition when a coin is inserted:

    initial state NoCredit {
        transition trigger: InsertCoin(int val)
                do: credit := credit + 1
                reply(credit,CoinResult::ACCEPTED)
                next state: CreditAvailable
            OR
                do:
                reply(credit,CoinResult::NOT_ACCEPTED)
                next state: NoCredit
            OR
                do:
                reply(0,CoinResult::NOT_OPERATIONAL)
                next state: NoCredit
        ....
    }

After inserting the coin, the transition gives three possible paths. This is an example of non-determinism. By reading the observed reply in the trace, the monitoring algorithm can determine which path to take or possibly an error is detected. Consider the following trace:

    Command InsertCoin
    Reply to command InsertCoin. Parameters: -1 CoinResult::NOT_OPERATIONAL

The trace will cause an error because the reply parameters do not match any of the expected values given in the three possible transition bodies. All the three possible executions at the moment of observing the reply will be reported in the monitoring results.

The text file with the results will contain the following:

    Interface monitoring error
    The event does not conform to the model

    Possible events in the current state of the model:

    Reply to command InsertCoin. Parameters: 1 CoinResult::ACCEPTED
    Reply to command InsertCoin. Parameters: 0 CoinResult::NOT_ACCEPTED
    Reply to command InsertCoin. Parameters: 0 CoinResult::NOT_OPERATIONAL

    Execution states of the model:

    Active state: NoCredit
    Values of global variables and current machine states:

    credit = 0
    Machine userMachine in state NoCredit

    Active state: NoCredit
    Values of global variables and current machine states:

    credit = 0
    Machine userMachine in state NoCredit

    Active state: NoCredit
    Values of global variables and current machine states:

    credit = 0
    Machine userMachine in state NoCredit

As can be seen there is a description for each possible execution. It contains: the state in which the event is observed (active state), the values of interface variables, the current states of the interface state machines (recall that multiple machines are possible). In our example, it happens that all executions have the same information.

The visualization of the error as a sequence diagram will contain similar information:

The three sections in the note indicate the three possible executions at the moment of observing the reply. In this example they are identical.

## Singleton and Multiton Interface Monitoring

The singleton/multiton distinction of an interface determines how the interface is monitored. Generally, a trace file contains information about servers that provide interfaces and clients that use them. A single server may have multiple clients of a given interface. A triplet of a client that uses an interface provided by a server is called connection.

A multiton interface is monitored by creating a separate monitor for every connection over that interface.

A singleton interface is monitored by creating a single monitor for all the connections that have the same server at their end. This reflects the fact that the clients of a singleton interface share one instance. In other words, the monitor of a singleton interface does not distinguish its clients, it treats them as a single client.