OpenSearch Query DSL
The OpenSearch Query DSL (domain-specific language) is based on JSON and is structured as an Abstract Syntax Tree. The DSL consists of two types of clauses:
- Leaf query clauses - they look for a particular value in a particular field
- Compound query clauses - they wrap other leaf or compound queries and are used to combine multiple queries in a logical fashion
Furthermore, a query clause can be executed in two different contexts, the Filter context and the Query context.
In the query context, a query clause both determines whether a clause is matched on a document or not, and answers the question “How well does this document match this query clause?”, which is used to calculate a relevance score of the document.
The filter context only determines whether a clause is matched on a document or not, it does not calculate a score. The important part is that clauses executed in the filter context will be cached and improve the performance.
Query Clauses
Leaf clauses#
The following section presents the leaf query clauses. It briefly describes what the results of the clause are, and how they are structured.
Regexp#
Returns documents that contain terms matching a regular expression. For a list of operators supported by the regexp query, see Regular expression syntax.
Clause structure#
The case insensitive option allows case insensitive matching of the regular expression value with the indexed field values when it is set to true. Default value is false.
{ "regexp": { "{path-to-field}": { "value": regular-expression, "case_insensitive": boolean } }}Example#
The following mapping for an index exists:
{ "mappings": { "properties": { "risk": { "properties": { "description": { "type": "text" } } } } }}A simple query to get all documents, where description contains "ing", would look like:
{ "query": { "regexp": { "risk.description": { "value": ".*ing.*", "case_insensitive": true } } }}Terms#
Returns documents that contain an any of terms in a provided field, including whitespace and capitalization.
Clause structure#
Below is the clause structure.
{ "terms": { "{path-to-field}": [ value1, value2 ], }}Example#
The following mapping for an index exists:
{ "mappings": { "properties": { "design": { "properties": { "status": { "type": "keyword" } } } } }}A simple query to get all documents, where status is equal to "Deprecated" would look like:
{ "query": { "terms": { "design.status": [ "Deprecated" ] } }}Range#
Returns documents that contain values in the provided range
Supported operators
- gt - Greater than.
- gte - Greater than or equal to.
- lt - Less than.
- lte - Less than or equal to.
Clause structure#
One or more of the supported operators can be specified.
{ "range": { "{path-to-field}": { "operator1": value, "operator1": value } }}Example#
The following mapping for an index exists:
{ "mappings": { "properties": { "design": { "properties": { "version": { "type": "integer" } } } } }}A simple query to get all documents, where version is greater than 5 would look like:
{ "query": { "range": { "design.version": { "gt": 5 } } }}Exists#
Returns documents that contain an indexed value for a field.
An indexed value may not exist for a document’s field due to a variety of reasons, most importantly:
- The field in the source JSON is null or []
- The field has "index" : false set in the mapping
Clause structure#
{ "exists": { "field": "path to field" }}Example#
The following mapping for an index exists:
{ "mappings": { "properties": { "mainSystem": { "properties": { "dateOfManufacture": { "type": "date" } } } } }}A simple query to get all documents, where dateOfManufacture has been assigned a value would look like:
{ "query": { "exists": { "field": "mainSystem.dateOfManufacture" } }}Compound clauses#
Boolean query#
The bool query lets you combine multiple search queries with boolean logic. You can use boolean logic between queries to either narrow or broaden your search results.
| Clause | Behaviour |
|---|---|
| must | The results must match the queries in this clause. If you have multiple queries, every single one must match. Acts as an and operator. |
| must_not | This is the anti-must clause. All matches are excluded from the results. Acts as a not operator. |
| should | The results should, but don’t have to, match the queries. Each matching should clause increases the relevancy score. As an option, you can require one or more queries to match the value of the minimum_number_should_match parameter (default is 1). |
| filter | Filters reduce your dataset before applying the queries. A query within a filter clause is a yes-no option, where if a document matches the query it’s included in the results. Otherwise, it’s not. Filter queries do not affect the relevancy score that the results are sorted by. The results of a filter query are generally cached so they tend to run faster. The filter query is used to filter the results based on exact matches, ranges, dates, numbers, and so on. |
Clause structure#
{ "query": { "bool": { "must": [ {query-clause} ], "must_not": [ {query-clause} ], "should": [ {query-clause} ], "filter": [ {query-clause} ] } }}Example#
The following mapping for an index exists:
{ "mappings": { "properties": { "design": { "properties": { "version": { "type": "integer" }, "status": { "type": "keyword" } } } } }}A simple query to get all documents, where version > 5 and status = "Active".
{ "query": { "bool": { "filter": [ { "terms": { "design.status": [ "Active" ] } }, { "range": { "design.version": { "gt": 5 } } } ] } }}Note: Since the query clauses were in the filter section, no score is assigned to the returned documents, but the result of the clauses can be cached.
Nested query#
Wraps another query to search nested fields.
The nested query searches nested field objects as if they were indexed as separate documents. If an object matches the search, the nested query returns the root parent document.
Clause structure#
A nested query starts with the nested keyword. It contains the path to the nested object and a new query object beneath. All field references must contain the full path from the root object.
{ "query": { "bool": { "{bool-operator}": [ { "nested": { "path": "{nested.path.to.object}", "query": { "bool": { "{bool-operator}": [ { "query-clause": { } } ] } } } } ] } }}With different combinations of boolean operators the query will return different results.
Examples#
The following examples are based on an index with this mapping. Note the employees field has the type "nested".
{ "mappings": { "properties": { "workItem": { "properties": { "status": { "type": "keyword" }, "activities": { "type": "nested", "properties": { "instruction": { "type": "keyword" }, "dateOfActivity": { "type": "keyword" } } } } } } }}It corresponds to this JSON-object:
{ "workItem": { "status": "", "activities": [ { "instruction": "", "dateOfActivity": "" } ] }}Example - Combined normal and nested query#
User wishes to find all work items which have activities done after 2022-03-13 and status is not removed. The query below will achieve that, using a bool query with the nested query inside. The top level "must_not" clause ensures that even though the nested documents might match, their root document is still filtered away.
{ "query": { "bool": { "must_not": [ { "terms": { "workItem.status": [ "removed" ] } } ], "filter": [ { "nested": { "path": "workItem.activities", "query": { "bool": { "filter": [ { "range": { "workItem.activities.dateOfActivity": { "gt": 2022-03-13 } } } ] } } } } ] } }}Example - Bool operators#
User wishes to find all risks where all mitigations statuses are not other than "Active". The top level "must_not" clause ensures that there is no clause which matches the nested query.
{ "query": { "bool": { "must_not": [ { "nested": { "path": "mitigations", "query": { "bool": { "must_not": [ { "terms": { "mitigations.status": [ "Active" ] } } ] } } } } ] } }}Note The below query will return all risks where at least one mitigation status is equal to "Active".
{ "query": { "bool": { "filter": [ { "nested": { "path": "mitigations", "query": { "bool": { "filter": [ { "terms": { "mitigations.status": [ "Active" ] } } ] } } } } ] } }}