Data Browser

- Overview - Getting Started - Common Tasks - Perspective - Toolbar - Properties Panel - Annotations - Search - Export - Import - Inspect Samples - Waveforms - Preferences - Older Data Browser - History -

Overview

The Data Browser is a trending tool. It can display the values of 'live' Process Variables (PVs) as well as historic data in a Stripchart-type plot.

Fig. 1: Data Browser Screenshot

Live Data

For each PV, the Data Browser collects 'live' samples from the control system and plots them over time. The size of the live sample buffer is configurable, as is the sample period.

Historic Data

The Data Browser queries archive data sources for historic samples to obtain data for time ranges 'before' the live sample buffer. You can configure one or more archive data source for each PV.

Getting Started

To create a new plot:

  1. Click on the "Open new Data Browser plot" icon in the CSS tool bar (top row of the CSS window).
  2. Open the plot's context menu by right-clicking into the plot. Select "Add PV".
  3. Enter the desired PV name, press "OK"
  4. Use the Toolbar or Properties Panel to change the displayed time range, zoom in/out etc.

Common Tasks

Data Browser Perspective

In addition to the Browser plot window itself, you typically use the Properties Panel to change colors etc., or the Search Panel to locate the names of PVs with historic data.

The Data Browser "Perspective" provides a suggested arrangement of these additional panels around the plot window, similar to the screenshot in Fig. 1 where the Properties and Export panels are below the plot, and a Search panel is to its left.

To switch to the Data Browser Perspective in case it is not currently selected, right-click on the Data Browser plot and select "Data Browser Perspective".

While in the perspective, you can then close the search panel, move the properties panel around as desired. If you later want to restore the default arrangement, again right-click on the Data Browser plot and select "Data Browser Perspective". Since it's already active, it will ask if you want to reset the perspective to its original state.

Plot Toolbar

The toolbar is displayed on top of a Data Browser plot. If it is not visible, right-click on the plot to open the context menu, then select "Show Toolbar".

Fig. 2: Data Browser Plot Toolbar

Tool tips with brief explanations will appear if you hover the mouse pointer over a toolbar button for a few seconds. Explanation of toolbar buttons:

Properties Panel

The Properties panel allows configuration of the plot, its axes and traces.

The same Properties panel is shared with other Data Browser plots or even other CSS tools. When you have multiple Data Browser plots open, the Properties panel will show the configuration of the plot that was last selected via a mouse click. When no Data Browser plot is the active window, the Properties panel will be empty or even show the configuration of a different CSS tool.

You can force the Properties panel to stay attached to a specific Data Browser plot by selecting the " pin" icon, or open more than one Properties panel (which you can then 'pin' to specific Data Browser plots) via the " drop-down" menu of the panel.

Subsection 'Traces'

Fig. 3: Properties panel, subseciton 'Traces'

Configure the individual traces in the plot via the table of traces. Right-click in the table to open a context menu for adding PVs or Formulas, deleting selected items, or adding archive data sources to PVs.

Fig. 4a: Traces with 'Optimized' Requests.

With 'Optimized' requests, the Data Browser requests historic data as a reduced (optimized) set of min/max/average samples. When using 'Area' traces as shown in Fig. 4a, the average values are plotted as a line, and the min/max outline is displayed as a light-colored area. This not only reduces the amount of data compared to plotting every single raw sample from the archive. In addition it makes it obvious if a value was fairly stable over time, or if there was a lot of movement in the data as around 06:00:00 in Fig. 4a.

The number of requested min/max/average samples is configurable in the Preferences. When there are less 'raw' data points than requested min/max/average samples, the 'Optimized' algorithm falls back to returning the raw data, meaning: When you zoom into the data far enough, you will eventually get raw data.

Fig. 4b: Blue traces switched to 'Raw' Request.

When switching to 'Raw Data' requests as in Fig. 4b, each original sample is requested from the history. Not only is this usually more data, taking longer to receive and then plot to. It also uses more memory, and when you try to look at raw data for a long time range your computer will eventually ran out of memory. The 'Raw Data' requests should therefore only be used when necessary because of shortcomings in the 'Optimized' algorithm.

Subsection 'Time Axis'

Configure the start and end time of the time (horizontal) axis.

Subsection 'Value Axes'

Add/remove value ('Y', vertical) axes, change their label or color.

Subsection 'Misc.'

General plot settings: Background color, refresh rate.

Annotations

The Toolbar contains buttons to add and remove Annotations.

When you add an Annotation, you select if and to which trace it should "snap". You can enter an arbitrary annotation text ("Name") and show that with the Annotation, and/or have it display the plotposition or Sample Information.

Move Annotation around the plot in the Panning or inactive zoom modes.

Change the Name, color, "snap" behavior of existing Annotations via the Configure Settings button in the toolbar.

Archive Search Panel

The Archive Search Panel is part of the default Perspective. If you cannot find it, try to reset the perspective, or open a new one by right-clicking into the plot and selecting "Open Archive Search Panel".

To locate Process Variable names (PVs) in an archive:

  1. Optional: Select an archive URL other than the default.
  2. Optional: If the archive has more than one sub-archive, select one or more sub-archive to search.
  3. Enter PV name pattern, see details below.
  4. Press "Search".

To show data for the resulting PVs, either right-click on PVs of interest and select "CSS/Data Browser" to open a new Data Browser Plot for them, or drag/drop the PVs from the search result table into an existing Data Browser Plot.

Simple Pattern

By default, PV name patterns can contain a question mark '?' as a place-holder for any single character or an asterisk '*' in place of any sequence of characters.

Regular Expression Pattern

When enabling "Reg.Exp." pattern search via the corresponding check box, a more powerful but also slower regular expression can be used:

Reg.Exp..Meaning
.Any Single Character
.*Zero or More Characters
a?Single 'a' or no 'a'
a*Zero or more 'a'
a+One or more 'a'
.+One or more arb. characters
a|bEither single 'a' or 'b'
[0-9]'0' or '1' or ... or '9'
()grouping of subexpressions
TempAny Channel containing 'Temp'
Temp$Any Channel ending in 'Temp'
^RFQHRAny Channel starting with 'RFQHR'
Tin_[0-9]+_sig$Channels ending in 'Tin_<NUMBER>_sig'
^XYZSys_Temp[2-3]$'XYZSys_Temp2' or 'XYZSys_Temp3'
Pwr_(in)|(out)Channels containing 'Pwr_in' or 'Pwr_out'

Data Export

Use the "Export Samples" panel to write data into files suitable for spreadsheet programs or Matlab.

Time Range

By default, the export will use the time range from the plot, but you can export data for a different time range.

Source Options

By default, the export will use the time range from the plot but fetch raw data from the archive, not the optimized, reduced min/max/average data that is displayed in the plot. You can modify the time range of the export to differ from the range of the plot, and also select to export either the exact data that is displayed in the plot, or request optimized data with a different number of "bins" from the archive.

Format Options

The format of the exported file defaults to a spreadsheet suitable for import into most spreadsheet programs. You can modify these settings:

Excel Spreadsheets

The exported data files are in a text format with TAB-delimited columns suitable for import into spreadsheet programs like Microsoft Excel.

Assume you chose a filename of "test.dat" on your Desktop, follow these steps for import into Excel:

  1. In Excel, use File/Open to open the file "test.dat".
    You might have to select "Files of type: All Files (*.*)" in the file "Open" dialog to do this.
  2. A "Text Import Wizard" should appear, and the default settings should already be set to
  3. Excel will default to a less useful format for the first column, the "Time" column, and only show time down to minutes, omitting the seconds or microseconds.
    Fix this by clicking on the "A" table header, i.e. selecting the whole first column; right-click to get the "Format Cells..." dialog, and enter a "Custom" format:
      m/d/yy h:mm:ss.000

When performing computations on the data, values marked "#N/A" which have non numerical value because they represent a status or error should be ignored by Excel.

For plots, the "X/Y scatter" plot type using time as the X axis and the value column for the Y axis tends to work best.

Note that Excel is limited to about 65000 lines. If your data file includes more lines, those will be lost in the import. You can work around this by exporting a smaller time range into separate files, or by exporting averaged data.

The value columns for averaged data will contain text like "5 [0 ... 10]" to indicate that the average value for that time was 5, with a minimum/maximum range of 0 to 10. To perform computations in excel, it might be useful to select the column and perform a text replacement of "[*]" with "" (nothing) to delete the min/max info.

OpenOffice.org/LibreOffice Calc (Spreadsheets)

As long as the exported data file has a ".csv" suffix, OpenOffice.org/LibreOffice should open the file with an "import" dialog similar to MS Excel, where you select "tab" delimited columns as explained above.

File names ending in ".dat", ".txt" etc. might only open in the OpenOffice.org/LibreOffice Writer (word processor), so you have to use the correct file suffix.

Matlab

The Matlab export can use two different file formats:

Binary Matlab .mat Files

The binary data file like "example.mat" is suitable for loading into Matlab 5 and newer like this:

load /path/to/the/example.mat

In your Matlab workspace you should then find structures named "channel0", "channel1", ..., one for each channel that was loaded from the file. Each structure has the following elements:

The time stamps are saved as text. In reasonable modern versions of Matlab you best convert them to Matlab serial date numbers, which you can then for example plot over time like this:

channel0.date = datenum(channel0.time, 'yyyy/mm/dd HH:MM:SS.FFF');
plot(channel0.date, channel0.value);
datetick('x',0);
title(channel0.name);

Matlab .m Text Files

The text file like "example.m" is suitable for loading (executing) in Matlab R2006b or newer, creating one 'timeseries' object per trace. To load it into Matlab, you execute the file like this:

cd /path/to/the/file
example

Matlab will then execute the commands in the file which define the data in the Matlab workspace and also plot it. With Matlab releases older than R2006b you will have to edit the generated file to suit your needs, for example use simply "plot(v);" to show the values.

Data Import

There is basic support for importing data from data files. The data file is fundamentally treated like as archive data source. The data file must reside in the workspace, i.e. it must be visible in the file Navigator.

A channel based on a data file can be added in two ways:

The data file must only contain data for a single channel. The lines of this data file must match the following format:

# YYYY-MM-DD HH:MM:SS.SSS   value
2011-08-09 15:51:16.734     299.634000

# or
# YYYY/MM/DD HH:MM:SS.SSSSSSSSS   value
2012/12/06 07:38:05.764764346     26.6504 OK      OK

The time stamp and value may be separated by spaces or tab characters. Lines that do not match this format are ignored.

One example of a suitable data file is the format that results from 'exporting' data for a single channel.

When saving a *.plt file that includes channels with imported data, the name of the data file is stored, not the actual sample data. The next time such a plot is loaded, data will again be imported from the file.

Inspect Samples

The "Inspect Samples" panel, accessible from the context menu of the plot, provides a tabular view of all samples in the Data Browser memory. It is mostly used for debugging.

For performance reasons, the table does not update with each received sample. Instead, a manual update via the "Refresh" button is required to show newly received "live" samples or to force an update of a table that appears inconsitent.

Waveforms (Arrays)

Support for waveform (array) type channels is limited. The Data Browser will only display one array element of a waveform channel over time. By default this is the first array element of each waveform sample, array element zero. The array index can be changed via the "Index" property of the channel in the Properties Panel.

The context menu of the plot has an entry to open the "Inspect Waveform" view. It can display each waveform sample individually.

The fundamental problem with waveform channels is that the channel data itself does not provide sufficient information: Is the array sample

Preference Settings

Use the menu Edit/Preferences/CSS Applications/Trends/Data Browser to change various settings including the default time range of a new plot, the number of 'Plot Bins' used by the Optimized request type.

Transition from older Data Browser

Even though this Data Browser code is quite different from the original code, you will notice a very similar look and overall behavior. The data browser also loads old "*.css-plt" configuration files.

When you first open the new Data Browser, you should reset the perspective.

The main differences:

Product History

At LANL, Craig McChesney and Sergei Chevtsov with contributions by Peregrine McGehee implemented the Java Archive Viewer for displaying Channel Archiver data (2004).

The first CSS Data Browser was based on Archive Viewer ideas but added live data handling as well as CSS integration and pluggable support for more than one type of historic data source (2007)

After many changes to the code, a rework was required (2010):

Kay Kasemir, kasemirk@ornl.gov