User's Guide

\textbf{Defining and Testing Operational State Conditions in the Data Monitor Tool}

\textbf{Abstract}

This document is a user's guide to defining and testing Operational State Conditions (OSCs) in the Data Monitor Tool (DMT) background environment. It is assumed the reader knows the rudiments of creating a background monitor in the DMT using the \texttt{DatEnv} class.

\textbf{Features}

\textbf{Introduction}

Before undertaking an analysis, one often needs to specify required interferometer or environment conditions. For example, before looking for instability in the Recycling Mirror (RM) servo, one would likely require both that servo and the Beam Splitter (BS) servo to be locked. One might also want Wave Front Sensing (WFS) to be engaged. In another analysis, one might require the laser intensity to exceed a threshold or require seismic motion to fall below a ceiling. To allow for flexible and comprehensive setting of conditions, one would also like to specify Boolean combinations of conditions. For example

    Cond_1 = RM servo locked
    Cond_2 = BS servo locked
    Cond_3 = Cond_1 & Cond_2

The OSC tool has been written to support convenient defining and run-time checking of such conditions. Definitions are made via a text configuration file. A handful of preliminary ``standard'' conditions (e.g., ``\texttt{Arm_locked}'' and ``\texttt{LVEA_quiet}'') have been provided in a sample configuration file. It is expected that many more such standard conditions will be defined and refined as physicists gain more experience with his and future datasets. Definitions for useful conditions should be sent to the author (\texttt{dwchin@umich.edu}) for incorporation into a public repository.

\textbfQuick Sample of Code

Before detailing how to use the OSC tool, let's first get a flavor of it with a stripped-down sample of code. The following code defines conditions and checks them in real time:

    // During initialization:
    osclist.readConfig("osc_sample.config");
  
    // As each time interval of data (e.g., 1-second frame) is read: 
    if ( osclist.satisfied("LVEA_quiet") ) {
      // Do some analysis....
    }

One can also loop through \textit{all} OSCs defined in a configuration file using an iterator: 

    // During initialization:
    osclist.readConfig("osc_sample.config");

    // As each time interval of data is read:
    OperStateCondList::iterator iter = osclist.begin();
    for (; iter != osclist.end(); ++iter) {

        // print out value being monitored. (does NOT work for meta-OSCs)
        cout << "'" << (*iter).first << "': watched value = "
             << (*iter).second->watchedQuantity() << endl;

        if ((*iter).second->satisfied() == true)
            cout << "'" << (*iter).first << "'\tsatisfied" << endl;
        else 
            cout << "'" << (*iter).first << "'\t\tNOT satisfied" 
                 << endl;
    }
This scrap of code assumes that the monitor class has been written with \texttt{DatEnv} as a base class, and \texttt{osclist} is a member \texttt{OperStateCondList} object.

The \texttt{readConfig()} call is made in the monitor's constructor. The \texttt{satisfied()} call is made from the monitor's \texttt{ProcessData} method. The configuration file might look like:

    # A comment
    x_quiet    rmsrange "H0:PEM-LVEA_SEISX" lo=0. hi=2000.
    y_quiet    rmsrange "H0:PEM-LVEA_SEISY" lo=0. hi=2000.
    z_quiet    rmsrange "H0:PEM-LVEA_SEISZ" lo=0. hi=2000.
    LVEA_quiet boolean  "x_quiet & y_quiet & z_quiet"   # a comment

    # in general, lines look like the following, where
    # "OscName", osc_type, "paramname" are case-insensitive,
    # i.e. FooBar, fOObAR and foobar are all equivalent names
    OscName  osc_type "channel_name" paramname=paramvalue  ...

The first three conditions have the names ``\texttt{x_quiet}'', ``\texttt{y_quiet}'' and ``\texttt{z_quiet}''. Each is of type \texttt{rmsrange} which means the data channel with the specified name must have an RMS value between the minimum and maximum values specified (0. and 2000. in each case). The fourth condition is Boolean, in this case the logical AND of the first three conditions.

Fields are separated by spaces or tabs. Valid condition names must begin with an alphabetic character and contain only alphanumeric characters, ``_'' (underscore), or ``:'' (colon). Condition names are case-insensitive, \textit{i.e.} \texttt{FooBar}, \texttt{fOObAR}, and \texttt{foobar} are all equivalent. For \texttt{boolean} conditions, spaces are not required in the Boolean expression. The bitmasks can be specified in decimal, octal, and hexadecimal format: octal numbers are denoted by a leading ``0'' (zero) and hexadecimal numbers are denoted by a leading ``0x'' (zero x).

\textbf{Other Directives}

There are other directives that can be specified in the configuration file which are not directly involved in defining OSCs.

\texttt{\textbf{include}}

\texttt{\textbf{ignore}}

\texttt{\textbf{ignoreallexcept}}

\texttt{\textbf{debuglevel}}

\texttt{\textbf{stride}}

\textbf{Conditions}

The following list of OSC types are expected to expand with future releases of the OSC tool.

There are two types of OSCs: atomic OSCs, and meta-OSCs. The atomic OSCs specify conditions on the data stream, whereas the meta-OSCs specify conditions on other OSCs. Each OSC has some number of parameters which specify the details of the condition, \textit{e.g.} threshold, frequency band, etc.

The atomic OSCs require a channel name in the configuration file definition, while the meta-OSCs require another OSC name in the definition. The OSC upon which a meta-OSC acts must have been defined before the meta-OSC.

Some classes of OSCs have parameters which others do not, \textit{e.g.} OSCs which place conditions on each sample in the time-series data have a \texttt{fraction} parameter which specifies the minimum fraction of data that must satisfy the condition for the OSC to be satisfied. The \texttt{fraction} parameter would make sense for something like \texttt{valueabove} but not for something like \texttt{meanabove}.

Some parameters are \textit{optional}, meaning that they need not be specified explicitly in the configuration file. If a parameter is optional it always has some default value.

If a line defining an OSC in the configuration file contains parameters that are not part of that OSC or meta-OSC, an undefined error may result.

\textbf{Condition Names}

All conditions must be named. Legal names must begin with an alphabetic character and may consist of letters, numbers, underscores and colons. Names are \textit{case-insensitive}.

\textbf{Common Parameters}

This is a list of names of parameters which are used for more than on OSC type. Note that is to be interpreted as the limit of the computer representation of the appropriate data type. A \textit{stride} corresponds to one Frame of data. (FIXME)

\texttt{\textbf{fraction}}

\texttt{\textbf{hold}}

\texttt{\textbf{dead}}

\texttt{\textbf{threshold}}

\texttt{\textbf{nstrides}}

\texttt{\textbf{lo}}

\texttt{\textbf{hi}}

\texttt{\textbf{freqlo}}

\texttt{\textbf{freqhi}}

\texttt{\textbf{mask}}

\textbf{Atomic OSCs}

These OSCs act on the time-series data in channels, and hence require a channel name in their definition, \textit{e.g.}:

    X_arm_locked  meanabove "H2:LSC-AS_I" threshold=17.3

\textit{NOTE:} The power in the \texttt{power*} OSCs is really band-limited RMS.

\texttt{\textbf{valueabove}}

\texttt{\textbf{valuebelow}}

\texttt{\textbf{valuerange}}

\texttt{\textbf{meanabove}}

\texttt{\textbf{meanbelow}}

\texttt{\textbf{meanrange}}

\texttt{\textbf{meanrise}}

\texttt{\textbf{meanfall}}

\texttt{\textbf{rmsabove}}

\texttt{\textbf{rmsbelow}}

\texttt{\textbf{rmsrange}}

\texttt{\textbf{bitand}}

\texttt{\textbf{bitnand}}

\texttt{\textbf{bitor}}

\texttt{\textbf{bitnor}}

\texttt{\textbf{abspowerabove}}

\texttt{\textbf{abspowerbelow}}

\texttt{\textbf{abspowerrange}}

\texttt{\textbf{abspowerrise}}

\texttt{\textbf{abspowerfall}}

\texttt{\textbf{abspowergain}}

\texttt{\textbf{fractpowerabove}}

\texttt{\textbf{fractpowerbelow}}

\texttt{\textbf{fractpowerrange}}

\texttt{\textbf{fractpowerrise}}

\texttt{\textbf{fractpowerfall}}

\textbf{Meta-OSCs}

\texttt{\textbf{boolean}}

\texttt{\textbf{transitup}}

\texttt{\textbf{transitdown}}

The bitwise conditions merit further explanation. The \texttt{bitand} and \texttt{bitnand} conditions refer to whether \textit{every} bit in the bitmask has corresponds to a bit in the data channel value. The \texttt{ bitor} and \texttt{bitnor} conditions refer to whether \textit{at least one} of the bits in the bitmask has a corresponding bit in the data channel value. Hence for bit masks with only one bit turned on, the \texttt{and} and \texttt{or} conditions are identical.

\texttt{transitup} and \texttt{transitdown} are conditions on other OSCs. \texttt{transitup} becomes True and is held True for strides whenever the named OSC changes state from False to True, where is the hold duration parameter. \texttt{transitdown} works in a similar way, except that it becomes True when the named OSC goes from True to False. The \texttt{dead} parameter prevents \texttt{transitup} and \texttt{ transitdown} conditions from becoming True again for an interval of \texttt{dead} strides after the end of the \texttt{hold} period.

One cannot yet define OSCs in code, only via a configuration file.

\textbf{Other Features}

The \texttt{OperStateCond} objects have some features worth noting:

\textbf{FIXME: Old Stuff}

The following spectral power conditions (meant for detecting transients, including servo instabilities) require specifying a minimum and maximum frequency for the range of computed power. The power is normalized so that its sum from zero to the Nyquist frequency equals the mean square value of the corresponding time series. Conditions on fractional power refer to the fraction of total power in the specified frequency range. Only parameters additional to the min/max frequencies are listed in the following table.

The ``mag'' and ``demag'' conditions refer to fractional changes in power over a given frequency range, independent of the absolute power level and of power in other frequency ranges. Normally, the scale factor used for ``mag'' should be greater than one and that used for ``demag'' less than one. The ``any'' and ``ave'' modifiers have similar meanings as for power rises and falls, except that for ``any'' the instantaneous rate threshold is taken as (scale factor)**(1 / number of time intervals). The number of time intervals N must be a positive integer. For ``ave'' conditions, at least N+1 time consecutive intervals must be examined before the condition can be satisfied.

Provision for these various ways of defining power changes has been made for flexibility. Very stable channels with nearly constant total power may be well suited to conditions based on absolute power. Channels with time-dependent total power but stable spectral shape may be well suited to conditions on fractional power. Channels with variations in total power and spectral shape may be best suited to conditions on power magnification scale factors.

\textbf{Checking conditions}

If an OSC has been defined during initialization, then one can check whether it is satisfied during a given time interval with a call to the \texttt{satisfied()} method. \textit{e.g.}:

    if (osclist.satisfied("oscname"))
        // send a trigger

The \texttt{OperStateCondList} class inherits from a \texttt{hash_map<const string, osc::OperStateCond*, osc::hash<const string> >}, so all the methods that a \texttt{hash_map} has are available. (See SGI's documentation for the Standard Template Library at \texttt{http://www.sgi.com/tech/stl/}.) So, for instance, one can loop through all defined conditions:

    // As each time interval of data is read:
    OperStateCondList::iterator iter = osclist.begin();
    for (; iter != osclist.end(); ++iter) {
        if ((*iter).second->satisfied() == true)
            cout << "'" << (*iter).first << "'\tsatisfied" << endl;
        else
            cout << "'" << (*iter).first << "'\t\tNOT satisfied" 
                 << endl;
    }

\textbf{Other Facilities}

In addition to the \texttt{OperStateCondList} class, a helper class called \texttt{TSWindow} is also available. \texttt{TSWindow} represents a time series with at most elements. It is typically used as a ``window'' on a time series that is to be presented to the DMT Viewer. For a relatively simple example, see the source code for the \texttt{LockLoss} monitor: \texttt{~dmt/cvs/dmt/src/monitors/LockLoss}. To add a datapoint to a \texttt{TSWindow} object, use the \texttt{append()} method.

\newpage

\textbf{Hacker's Guide}

\textbf{Big picture from the end-user standpoint}

We want to have various Operational State Conditions, encapsulated by the class \texttt{OperStateCond} (and abbreviated OSC for the rest of this document), which specify conditions which may or may not be satisfied by the data stream in \textit{one (1)} channel.

Then, for the user, there is a \texttt{hash_map} of standard OSCs, \textit{e.g.} \texttt{valueabove} and \texttt{meanbelow}, indexed by the user-specified names defined in a configuration file. This is so that the user can refer to specific OSCs.

Each OSC has none or some parameters associated with it, \textit{e.g.} \texttt{valueabove} has two parameters, one of which is its \texttt{threshold}: if the data in the channel ever goes above that threshold value, the OSC is said to be ``satisfied'' (the \texttt{satisfied()} member function).

Each parameter has a name, \textit{e.g.} \texttt{threshold}, and a datatype, \textit{e.g.} \texttt{double}.

This code will read a configuration file specifying the various OSCs and the corresponding parameter values. Each OSC must be given a unique name, a string of alphanumeric characters beginning with an alphabetic character.

The reason for naming each OSC is so that each OSC maybe referred to later in a configuration file and used in meta-OSCs. Since each OSC.\texttt{satisfied()} returns a \texttt{boolean}, one may now define a Boolean OSC to say something like ``True if CHAN1 has valueabove 2.3 AND CHAN2 has meanbelow 4.2''.

Rather than having to manually type in the names of each OSC into the monitor code, we define the \texttt{OperStateCondList} class, which, despite its name, is not a linked-list but a \texttt{hash_map}, \textit{ i.e.} a dictionary that associates a pointer to an \texttt{OperStateCond} object with a string (the user-defined name of the OSC, specified in the configuration file). Making \texttt{OperStateCondList} a \texttt{ hash_map} means that we may iterate over all defined OSCs:


  OperStateCondList mOSClist;
  OperStateCondList::iterator iter = mOSClist.begin();
  for (; iter != mOSClist.end(); ++iter)
         if ((*iter).second->satisfied())
             // generate and send a trigger to the MetaDB

Notice that this still does not obviate the need for \texttt{ OperStateCondList::satisfied(const char *oscname)} since the user might want to test for a particular OSC, and the method was defined in a previous incarnation of this code. Another way of accessing particular OSCs is to use the fact that it's a \texttt{hash_map} (which is actually what \texttt{OperStateCondList::satisfied()} does):

  OperStateCondList mOSClist;
  mOSClist.readConfig("configfile.conf");
  
  if (mOSClist["my_condition"]->satisfied())
     cout << "my_condition is satisfied" << endl;
  else
     cout << "my_condition is NOT satisfied" << endl;

\textbf{Big picture from the programmer's standpoint}

This is more info for one who wishes to modify the OperStateCondList code. If you read the above, you will see a very striking resemblance between OSCs and the basic datatypes of a programming language. In a programming language, we may define variables of different datatypes; in the OperStateCondList system, we may define OSCs of different OSC types (like \texttt{valueabove}).

And then, the Boolean OSCs are just a Boolean expression of previously defined OSCs. So, this really becomes a small language, and hence the scanning, tokenizing and parsing objects associated with the Boolean OSC. (See \texttt{osc/boolean/*.hh}.)

\texttt{boolean}, \texttt{transitup}, and \texttt{transitdown} OSCs are really meta-OSCs, \textit{i.e.} they define conditions on other OSCs. So, to know whether the OSCs they refer to are satisfied, they need to get a pointer to the \texttt{OperStateCondList} that contains them.

To make the evaluation of the Boolean expression a little more efficient, we convert the infix form of the Boolean expression, \textit{ e.g.} \texttt{Locklost \& PSLglitch}, to Reverse Polish Notation (RPN), and in the process verify the correctness of the expression. RPN is quicker to evaluate since it simplifies putting the running value in a stack.

To make evaluations for the other conditions a little more efficient, we store a short history of two \texttt{satisfied()} values: the value for the current stride, and the value for the previous stride. Some OSCs depend on more than just the previous stride, so they have their own private data members defined for storing this data.

Now, for a larger overview. And let's do this in a top-down way, with a very large grain.

We've already seen that \texttt{OperStateCondList} is a \texttt{ hash_map<const string, OperStateCond*>}.

Any new types of OSCs that are to be written will have to inherit from \texttt{OperStateCond}: it is a base class that provides data and functions that all OSCs need.

\texttt{OperStateCondList} also contains (`has a' relationship) information about the various basic OSC types. This information is stored in \texttt{osc::TypeInfoMap mTypeInfoMap}. (See \texttt{ OSCListMisc.hh}.)

\texttt{osc::typeInfoMap} is a \texttt{hash_map<const string, TypeInfo>}, \textit{i.e.} it's a dictionary that relates a \texttt{TypeInfo} object to a \texttt{const string} (which is the name of that type, \textit{e.g.} \texttt{ valueabove}).

The \texttt{TypeInfo} class contains information about the OSC type. It does this by containing a sorted map of parameter types: \texttt{typedef map<const string, tseriesType_t> ParamTypeMap}. \texttt{tseriesType_t} is defined in \texttt{namespace osc} and is an enumeration of the various ``atomic'' data types that may be contained in a \texttt{TSeries}. The indexing key for \texttt{ParamTypeMap} is the name of the parameter, \textit{e.g.} \texttt{mParamTypeMap["threshold"]} would be a \texttt{DOUBLE}.

In summary, let ``*'' mean ``has a'', and ``>'' mean ``is a''.

OperStateCondList
 > hash_map of OperStateCond*, indexed by user-defined names of OSCs
 * TypeInfoMap, containing information about the basic types

OperStateCond
 > a base class from which actual OSCs will inherit
 * a bunch of stuff.  See OperStateCond.hh and misc.hh

TypeInfoMap 
 > hash_map of TypeInfo, indexed by names of types
 * ParamInfoMap, containing information about the parameters
                 associated with this type of OSC

ParamInfoMap 
 > hash_map of ParamInfo, indexed by names of parameters

The \texttt{ParamInfoMap} and \texttt{TypeInfoMap} are initialized in \texttt{OperStateCondList}'s constructor. Since this data should be common to all instances of \texttt{OperStateCondList}, one would like this to be a static data member, but unfortunately it is almost impossible to initialize a complex data structure such as this at compile time.

Now, how do the OSCs store the values of their parameters? Since these parameters may be of different types, and they may be read from an Epics channel, we can't just store them as simple values. We use a template class (just because I don't know how to do something like the \texttt{Param} class in DMT: see \texttt{osc/Param.hh}). For now, we only have \texttt{int} and \texttt{double} parameters, so each \texttt{OperStateCond} object has two \texttt{hash_map}s: one for integer parameters (\texttt{mIntParams}) and one for doubles (\texttt{mDblParams}). These two \texttt{hash_map}s are indexed by the names of the parameters. The value of a parameter may be retrieved by, \textit{e.g.}:

    mDblParams["threshold"].value()

\textbf{How to Add New OSC Types}

Look in the \texttt{osc} subdirectory. You'll create a new class that inherits from \texttt{OperStateCond}. Look at the others for examples. You'll have to initialize the object's \texttt{ParamInfoMap} in the \texttt{OperStateCondList} constructor, and then enter that into \texttt{OperStateCondList}'s \texttt{TypeInfoMap}. You will need to add a section that actually adds a new OSC to the \texttt{hash_map} (around line no. 612 and onwards). You may need to write sections to parse the config file line for your new OSC, though it's quite unlikely unless you have a new type of parameter (\textit{e.g} a complex number) that behaves very differently from the types that already exist. \texttt{OperStateCondList::parseAtomicParams()} is the method that parses config file lines for OSCs. The \texttt{transit*} conditions have config line parsing written inline in the constructor of \texttt{OperStateCondList}. Only the \texttt{boolean} OSC type has its own parser since it's a bit of a trick to parse a Boolean expression.

alphabetic index hierarchy of classes

Please send questions and comments to zweizig_j@ligo.caltech.edu

generated by doc++