In addition to standard types, the datastore supports several custom classes of types.

Abstract Types

An abstract type represents a standard interface for a set of other types. This lets you treat separate types that have a common set of attributes interchangeably. For example, different monitoring systems will have different ways of collecting and labeling properties of the system like CPU or memory usage, but usually have a core set of common attributes. An abstract type would be a type with those common attributes defined on it. Records cannot be stored to the abstract type–it is used only to define the common attributes. There are then one or more actual types that extend an abstract type and supply metadata indicating how their attributes map to those of the abstract type. Actual types can be added dynamically and will appear as that abstract type immediately.

Example

A process in Windows reports WorkingSetSize, which is effectively the same as RSS. An abstract Process type could represent processes on different systems. The abstract type might be defined as:

AdType = "Process"
KeyAttributes = "ActualType,PID"
Abstract = true

Note

The first key attribute of an abstract type must be ActualType. The remaining key attributes must be the key attributes for the actual types.

The following attribute might represent the amount of RAM currently occupied by the process:

AdType = "Attribute"
ForType = "Process"
ForName = "ActiveMemory"

Then the actual Unix type would be defined as:

AdType = "UnixProcess"
KeyAttributes = "PID"
Extends = "Process"

Its ActiveMemory attribute would in turn be defined as:

AdType = "Attribute"
ForType = "UnixProcess"
ForName = "ActiveMemory"
Source = RSS

The Source attribute specifies how to compute the values of ActiveMemory for UnixProcess types. In this case, it is simply evaluating the RSS attribute. If not specified, it defaults to the value of the same attribute (in this case, ActiveMemory). Thus when you are creating a type, you do not need to predefine attributes with the same names as the abstract attributes, since the attributes will be created automatically when records are stored, and the default expression for Source will be correct.

Note

The value of the Source attribute is an expression in general, not a string. It is evaluated in the context of the actual type. If you supply a string, the value will always be that string literal, which is a constant.

When you then run a query on Process, you will get back results from UnixProcess records, but they will appear to be Process records. For example, suppose you have the following UnixProcess records stored:

AdType PID Name RSS
UnixProcess 100 emacs 8192
UnixProcess 101 python 442

Furthermore, suppose you have a similar WinProcess type that specifies WorkingSetSize as the source for the ActiveMemory attribute.

AdType PID Name WorkingSetSize
WinProcess 90 python 732

Then a query of select * from Process will return the following:

AdType ActualType PID Name ActiveMemory
Process UnixProcess 100 emacs 8192
Process UnixProcess 101 python 442
Process WinProcess 90 python 732

Note that ActualType is added automatically. In addition, RSS is only defined on UnixProcess so it is not returned. However, its value is used for the ActiveMemory attribute.

Abstract records can be aggregated as well, even if the records come from separate actual types:

select Name, sum(ActiveMemory) as TotalMemory from Process group by Name
Name TotalMemory
emacs 8192
python 1174

This lets you work interchangeably with records from different systems, as long as they have a common core set of attributes.

Virtual Types

A virtual type is a type whose records come solely from running a plugin. A virtual type is one which has Virtual set to true. The name of the plugin to call must be specified in the VirtualPlugin attribute, and it must implement a find(view, policy) function that returns the current records for this type.

Example

As an example, to make a type whose records are the current files in a directory, first define a type:

AdType = "File"
Virtual = true
VirtualPlugin = "file.list"

Then define a plugin called file.list that uses a function parse_ls_output (not specified here) that could parse the results of ls and convert them into records:

import subprocess

def find(view, policy):
    text = subprocess.check_output(["ls", "-l"])
    return parse_ls_output(text)

It is not possible to store records directly as a virtual type. Instead, the plugin is called each time the type is queried. Aggregating virtual types is not currently supported, nor are timeline queries.