# Filter Grammar

The grammar components of filter expressions within VisionPro Deep Learning are comprised of the following:

- Literal values
- Built-in variables
- Operators
- Functions
- Regular Expressions

## Filter Literal Values

The VisionPro Deep Learning filter grammar supports string and floating point value types. These literal values must be entered, as shown below:

Type | Example | Notes
--- | --- | ---
string |
    filename = 'RUN00031.bmp'
    sets ~ 'PlantA'
| Strings must be enclosed in single quotation marks, and escaping is not supported.
floating point |
    feature [score > 0.750]
    count(feature) = 10
| A decimal point is not required.

## Filter Variables

The filter grammar includes variables that are defined for views. You use these simply by including them in an expression that evaluates to a Boolean result, such as the following:

width > 1500// This filter returns views with a width greater than 1500.

View-level variables can be one of the following:

- Generic, in which case they are available in all views.
- Tool-specific, in which case they are only available for specific tool types.

For example, the trained variable is available for any view, while the precision variable is only available for Red Analyze or Blue Locate tool views.

In addition to view-level variables, the grammar also supports tool-specific, axis-level variables that you can use to filter views based on the presence, absence, or values of individual markings, labels, and other view elements. You use axis variables by including them in square brackets, following the name of the axis.

For example, the Blue Locate tool has two axes: feature (for found features) and match (for matched models). If you want to filter views that contain specific features, you would include feature, followed by square brackets containing the axis filter:

feature[x > 10 and x < 900]// This filter returns views containing a feature with an x-position between 10 and 900.

In this case, feature is the axis, and x and y are the axis-level variables.

Note:

For the region and filter axis in the Blue Locate tool, append an exclamation mark ('!') to the axis name to match labels instead of features:

feature![x > 10 and x < 900]// This filter returns views containing a label with an x-position between 10 and 900.

View-level and axis-level variables can be combined in a single expression:

trained and feature[x > 10 and x < 900]// This filter returns trained views containing a feature with an x-position between 10 and 900.

You can also use a path-delimiter ('/') to refer to axis variables in an expression.

feature/x > 10 and feature/y < 900// This filter returns trained views containing at least one feature with an x-position greater than 10, and at least one feature with a y-position less than 900.

### Generic View-Level Variables

These variables are used with any view, regardless of the Deep Learning tool.

View-level Variable | Data Type | Description
--- | --- | ---
new | Boolean | Returns all views that were added since the last training.
anomalous | Boolean | Returns samples where the image plane characteristics do not match the tool configuration.
all | Boolean | Returns all the views in the database.
processed | Boolean | Returns views with markings that were generated since the last training.
masked | Boolean | Returns all views that contain a mask.
invalid | Boolean | Returns all views containing invalid labels, such as empty Blue Locate tool features, or overlapping regions in a Red Analyze tool.
trained | Boolean | Returns all views that were included in the training set.
labeled | Boolean | Returns all views that contain a label.
sample_name | String | The name of sample, which may be different if multi-channel samples are being used.
filename | String | Returns the view with that filename.
added | String | Returns the date when the image was added to the workspace.
duration | Floating Point | Returns the last processing time for the view.
index | Floating Point | Returns an index of views in the enumeration of all views, from 0 to N-1. This is useful when you want to select a specific range of views.
view_index | Floating Point |
random | Floating Point | Returns a random value. This is useful to shuffle views (when used to sort views) or to select a random set of views. For example, 'random < 0.5' would select half of the views.
width | Floating Point | Returns views that match the specified width value.
height | Floating Point | Returns views that match the specified height value.
difficulty | Floating Point | Returns views that match a unit-less measure of the degree of difficulty to train a view.
sets | String [ ] | Returns all of the views that are included in the specified set. This variable is unique, in that a vector of strings are specified in order to return a result. The following grammar summarizes the semantics of the equality, inequality, and contains operators ('~') when applied to the sets variable: sets = '<set_name>'// This returns true and the associated views if at least one element of a sets is identical to set_name. sets != '<set_name>'// This returns true and the associated views if none of the elements of sets is identical to set_name. sets ~ '<set_name>'// Returns true and the associated views if at least one element of a sets contains the string set_name. Note: You can use the contains operator ('~') to perform partial matching. For example, this syntax can be used: sets ~ "Plant17"// This returns true for views that are members of Plant17, such as "Plant17-LineA" or "Plant17-LineB", but not "Plant18-LineA".

### Tool-Specific View-Level Variables

These variables are tied to specific VisionPro Deep Learning tools, along with specific components of those tools. The variables below can be used to filter the result of processing.

Blue Locate and Blue Read Tool View Variables | Data Type | Notes
--- | --- | ---
mismatch | Boolean |
recall | Floating Point |
precision |
f_score |
best_match_string | String |
Blue Locate and Blue Read Tool - Feature Axis Variables | Data Type | Notes
covered | Boolean |
matched |
score | Floating Point |
angle |
size |
aspect_ratio |
width |
height |
x |
y |
angle_dist |
size_dist |
aspect_ratio_dist |
dist |
x_dist |
y_dist |
id | String |
covered_id |
Blue Locate and Blue Read Tool - Match Axis Variables | Data Type | Notes
covered | Boolean |
score | Floating Point |
angle |
scale |
aspect_ratio |
shear |
x |
y |
features |
name | String |
feature_string |

Green Classify Tool View Variables | Data Type | Notes
--- | --- | ---
threshold | Floating Point |
score |  |
unknown |  | Denotes an image that is labeled as an unknown (in other words, untrained) class.
mismatch |  |
best_tag | String |
validated |  | Returns only the views that are included in the validation set of Green Classify High Detail.
not validated |  | Returns only the views that are not included in the validation set of Green Classify High Detail.
Green Classify Tool - Tag Axis Variables | Data Type | Notes
score | Floating Point |
name | String |

Red Analyze Tool View Variables | Data Type | Notes
--- | --- | ---
good | Boolean | Returns views that are labeled as good. Example: "good and score<=0.2"
negative | Returns the same results as good.
bad | Returns views that are labeled as bad. Example: "bad and score>=0.8"
positive | Returns the same results as bad.
mismatch |
recall | Floating Point | Returns the recall as the processing result. Example: "recall >= 0.7"
precision | Returns the precision as the processing result. Example: "precision>= 0.7"
f_score | Returns the f-score as the processing result. Example: "f-score>= 0.75"
T1 | Returns the lower threshold value. Example: "T1>= 0.4"
T2 | Returns the upper threshold value. Example: "T2>= 0.6"
score | Returns the highest defect probability. Example: "score>= 0.5"
deviation | Returns the same results as score.
validated |  | Returns only the views that are included in the validation set of Red Analyze High Detail.
not validated |  | Returns only the views that are not included in the validation set of Red Analyze High Detail.

## Filter Grammar Operators

The grammar operators are used to construct expressions that can be evaluated to true or false (for filtering) or to an integer or floating point value (for sorting) or to a vector (for counting).

Note: There are some specific constraints and behaviors associated with the sets variable, which is unique because it is a vector of strings (in other words, all of the sets of which the view is a member).

Operator | Notes | When used with sets
--- | --- | ---
~ | String comparison: True if LHS contains RHS. Use 'string' as shorthand for filename ~ 'string' | Returns a second vector of strings in LHS that contain RHS. The returned vector can be evaluated as a Boolean (true if at least one string) or used as a new vector of strings.
in | Floating point within range: True if LHS is within the range specified by RHS. | N/A
and |  |
or |  |
not |  |
< | Natural sort for strings. | Natural sort for strings.
<= |  |
> |  |
>= |  |
= |  | True if the vector of strings contains at least one element that exactly matches the supplied string. Cannot be used to compare two vectors for equality.
!= |  | True if at least one element of the vector does not exactly match the supplied string. This is not the same as not(sets = 'string')
( ) | Precedence |

## Filter Grammar Functions

When used without qualification, filter axes (feature, match, region, tag) return lists of items. These lists cannot be tested directly, but they can be supplied to the count() function, the result of which can be tested.

In addition, vectors of string (for example sets) can also be supplied to count(), but not to the other functions.

Additionally, when any axis floating point variable is provided using a path separator instead of a part of an axis filter, it will return a list of values which cannot be tested directly, but can be supplied directly to the min() and max() functions, and those results can be tested.

For example:

- count (feature) > 10// Returns views that have more than ten features.
- min (feature/score) < 0.95// Returns views where the minimum score for any feature is less than 0.95.
- count (sets) > 1// Returns views that are members of more than one set.

Operator | Notes
--- | ---
count() |
min() |
max() |
sum() |
avg() |
std() |

## Regular Expressions in Filters

You can apply a regular expression to any filter variable that is a String data type.

You can return whether a regex matches: /<regex>/.test(<variable>)

- /DF\d\d\d\d/.test(sample_name)// Returns true if the view's sample name matches the regex DF\d\d\d\d.

You can also use a capture string and compare the captured string to another string: /<regex(capture)>/.extract(<variable>), which returns the captured string.

- /DF(\d\d\d\d)/.extract(sample_name) = '1234'// Captures the four digits from the regex and tests against '1234'.

Note:

- The examples above use the view-level variable sample_name, which evaluates to a String. If you attempt to use a String data type axis variable, those variables may evaluate to a vector of strings, since there may be multiple instances of the axis variable within a single view. In this case, the regex must be applied within the axis filter, as shown below:

  match[/FH\d\d\d\d/.test(feature_string)]// For this expression, the regex is applied for each match's feature_string. If any of the matches' feature_string match the regex, the view will be included.

  The following is an example of an invalid regex filter expression:

  /FH\d\d\d\d/.test (match/feature_string)// The expression should return true if the view includes a match with a feature_string matching the regex. However, this is invalid, because each view may have multiple matches.

- You cannot use the sets variable in a regular expression.

## How to Write Filter Expressions

Filter expressions can include view-level filters where the variable refers to the entire view.

- filename ~ 'REJECT'// Returns views where filename contains 'REJECT'.
- count(feature) > 3// Returns views with more than three features.

Filter expressions can also include axis filters, where the individual marking or labeling for each view is evaluated.

- feature [score > 0.5 ]// Returns views with a feature with score greater than 0.5.

You can combine view-level and axis-level filters.

- filename ~ 'REJECT' and feature [score > 0.5 ]// Returns views where filename contains 'REJECT' and a feature scoring greater than 0.5.

You can combine filters for multiple axes.

- feature [score > 0.5 ] and match [ score > 0.5 ]// Returns views with a feature with score greater than 0.5 and a match with a score greater than 0.5.

You can specify multiple filter conditions for an axis.

- feature[score > 0.5 and matched ]// Returns views with a feature with a score greater than 0.5 and the feature is in a matched model.