Search API

Overview

The TrueVault Document Search Engine is a HIPAA compliant search engine that gives you full programatic control over the Documents in your Vaults

With the Search API you define the fields you want to index using a Schema. Any Document associated with a particular Schema will be indexed according to the Schema definition in JSON. You manage vault-specific Schemas via the Schema API.

The Search API has one required query string parameter, search_option. It is a base64 encoded Document in JSON that specifies various search options such as: filters, pagination parameters and the sort order.

Warning

By default, the search will perform an or query based on the search_option.filter fields. For and queries, please use the search_option.filter_type option.

Warning

At this time the maximum search result size is 500 entries.

Search Permissions

In order to search documents in a specific vault, the user performing the search must have the R Vault::<vault id>::Document::.* and R Vault::<vault id>::Search:: permissions. A user can also be allowed to search documents in any vault by providing the R Vault::.*::Document::.* and R Vault::.*::Search:: permissions. To search users, the user performing the search must have the R User::.* permission.

Defining Search Options

  • schema_id: string(req’d) - ID of the Schema definition which the Documents you want to search is associated with

  • page: integer(optional, default: 1) - specify page number for paginated response

  • per_page: integer(optional, default: 100, max: 500) - specify number of results per page in paginated response

  • filter_type: string(optional, default: or) - specify the query’s boolean operation when multiple filters are given. Accepts and or or

  • full_document: boolean(optional, default: false) - specify whether to return the full Documents in the result or just document_id s

  • sort: object(optional, default: None) - list object of field name and sort directions.

    Example sort object:

    In this example, the results will be sorted in ascending order on first_name, and then descending order on last_name.

    {
        "sort": [
            {
                "first_name": "asc"
            },
            {
                "last_name": "desc"
            }
        ]
    }
    
  • filter: object(req’d) - an object of field filters.

    • The case_sensitive option can also be used in conjunction with in and not_in filters, where case (in)sensitivity will be applied to each item in the given list.

    • case_sensitive has no effect on numerical values.

    • The range filter can be applied to both numerical and string values, the latter of which is determined by lexicographical order.

      Operation Description Example filter (JSON object)
      eq exact match
      {
        "filter":{
           "last_name":{
               "type":"eq",
               "value":"John Smith"
           }
        }
      }
      
      not negation
      {
        "filter":{
           "last_name":{
               "type":"not",
               "value":"Smith"
           }
        }
      }
      
      in, not_in inclusion/exclusion
      {
        "filter":{
           "last_name":{
               "type":"in",
               "value":["Yang","Doe"]
           }
        }
      }
      
      wildcard standard *, ?
      {
        "filter":{
           "last_name":{
               "type":"wildcard",
               "value":"John*"
           }
        }
      }
      
      case_sensitive specify case sensitivity
      {
        "filter":{
           "last_name":{
               "type":"eq",
               "value":"john smith",
               "case_sensitive":false
           }
        }
      }
      
      range
      gt, gte, lt,
      and lte operations
      {
        "filter":{
          "age":{
            "type":"range",
              "value":{
                "gte":18,
                "lt":100
            }
          }
        }
      }
      
      geo_distance distance from specified geospatial location
      {
        "filter": {
          "clinic_location": {
            "type": "geo_distance",
            "value": {
              "lat": 40,
              "long": -70,
              "distance": 96.7,
              "unit": "mi"
            }
          }
        }
      }
      

Sample search_option

{
    "filter": {
        "first_name": {
            "type": "eq",
            "value": "John",
            "case_sensitive": false
        },
        "last_name": {
            "type": "not",
            "value": "Smith"
        },
        "provider": {
            "type": "in",
            "value": ["insurer_a", "insurer_b"]
        },
        "created_date": {
            "type": "range",
            "value": {
                "gte": "2013-09-23",
                "lt": "2014-01-01"
            }
        },
    },
    "full_document": true,
    "filter_type": "and",
    "page": 1,
    "per_page": 3,
    "sort": [
        {
            "first_name":"asc"
        }
    ],
    "schema_id": "00000000-0000-0000-0000-000000000000"
}

Sample Search Response

{
    "data": {
        "info": {
            "per_page": 10,
            "current_page": 1,
            "num_pages": 1,
            "total_result_count": 2
        },
        "documents": [
            {
                "document": "e30=",
                "document_id": "00000000-0000-0000-0000-000000000000",
                "owner_id": "00000000-0000-0000-0000-000000000000"
            },
            {
                "document": "e30=",
                "document_id": "00000000-0000-0000-0000-000000000000",
                "owner_id": "00000000-0000-0000-0000-000000000000"
            }
        ]
    },
    "result": "success",
    "transaction_id": "00000000-0000-0000-0000-000000000000"
}

Searching TrueVault System Fields

TrueVault System Fields are metadata that are automatically generated for every User created in TrueVault. Each system field is prefixed with $tv..

Each User must have attributes in order be searchable by system fields. In the case where there are no user attributes, you must provide an empty JSON object, {}, in order for the User to be searchable by system fields.

System Field Description Example filter (JSON object)
$tv.created_at Date user was created
{
  "filter":{
    "$tv.created_at":{
      "type":"range",
      "value": {
        "gte": "2014-01-01",
        "lt": "2015-01-01"
      }
    }
  }
}
$tv.id ID of a user
{
  "filter":{
     "$tv.id":{
         "type":"eq",
         "value":"11111111-1111-1111-1111-111111111111"
     }
  }
}
$tv.status

The status of a user

Accepted values:
  • DEACTIVATED
  • ACTIVATED
  • LOCKED
{
  "filter":{
     "$tv.status":{
         "type":"eq",
         "value":"LOCKED"
     }
  }
}
$tv.username Username of a user
{
  "filter":{
     "$tv.username":{
         "type":"eq",
         "value":"john.smith@test.com",
         "case_sensitive":false
     }
  }
}

Searching and Sorting By Geospatial Distance

Oftentimes applications want to store and retrieve location data based on distance away from a certain point. TrueVault provides the geo_point data type to store coordinates. You can subsequently filter these data points by geo_distance, the geospatial distance away from the geo_point field.

Example Geo Point Document Field:

{
  "clinic_location": {
    "lat": 37.3382,
    "long": 121.8863
  }
}

Warning

geo_point objects must include lat and long keys.

Example Geo Point Schema:

{
  "name": "clinic",
  "fields": [
    {
      "name": "clinic_location",
      "index": true,
      "type": "geo_point"
    }
  ]
}

Example Geo Point Search Option:

{
  "full_doc": true,
  "filter": {
    "clinic_location": {
      "type": "geo_distance",
      "value": {
        "lat": 37.0,
        "long": 121.0,
        "distance": 10.0,
        "unit": "mi"
      }
    }
  },
  "sort": [
    {
      "clinic_location": "asc"
    }
  ],
  "schema_id": "00000000-0000-0000-0000-000000000000"
}

Warning

  • The value of unit must be either "mi" or "km".
  • The maximum filter distance is 100 miles (~160.934 kilometers).
  • lat, long, distance, and unit are all required fields.
  • lat must be between -90 and 90 degress, and long must be between -180 and 180 degrees.
  • Any geo_distance sort must be accompanied by a geo_distance filter.
  • If full_doc is true and a sort on a geo_distance field is provided, each search result will include a distance_from, the geospatial distance away (float) from the corresponding filter’s lat and long in terms of the unit specified in the search filter.

Example Geo Point Search Result:

{
    "data": {
        "info": {
            "per_page": 10,
            "current_page": 1,
            "num_pages": 1,
            "total_result_count": 2
        },
        "documents": [
            {
                "distance_from": 0.08827010485323095,
                "document": "eyJjbGluaWNfbG9jYXRpb24iOiB7ImxhdCI6IDM3LjMzOTIsICJsb25nIjogMTIxLjg4NzMwMDAwMDAwMDAxfSwgImNsaW5pY19uYW1lIjogInRlc3RfY2xpbmljIn0=",
                "document_id": "11111111-1111-1111-1111-111111111111",
                "owner_id": "00000000-0000-0000-0000-000000000000"
            },
            {
                "distance_from": 0.17653975451800838,
                "document": "eyJjbGluaWNfbG9jYXRpb24iOiB7ImxhdCI6IDM3LjM0MDIsICJsb25nIjogMTIxLjg4ODN9LCAiY2xpbmljX25hbWUiOiAidGVzdF9jbGluaWMifQ==",
                "document_id": "22222222-2222-2222-2222-222222222222",
                "owner_id": "00000000-0000-0000-0000-000000000000"
            },
        ]
    },
    "result": "success",
    "transaction_id": "00000000-0000-0000-0000-000000000000"
}

Search Documents

https://d33wubrfki0l68.cloudfront.net/8513b318bb356a4044b0b241683868db1da61329/895a1/_images/plans-available-standard-plus.png
POST /v1/vaults/(string: vault_id)/search

Searches for Documents in the specified Vault.

Note

Each request consumes one Operation for the search function, and an additional Operation for each document record returned. So, a search returning 20 documents will consume 21 Operations.

Form Parameters:
 
  • search_option – string(req’d) - search_option as a base64 encoded JSON Document
Request Headers:
 
Status Codes:
  • 200 OK – success
  • 400 Bad Request – invalid vault_id, invalid base64 encoded JSON Document, invalid options in search_option

Policy Requirements

{
    "Resource": "Vault::{vault_id}::Search::",
    "Activities": "R"
}

Search Users

https://d33wubrfki0l68.cloudfront.net/46256db48654ea43d1fd9377023f4d876582d839/ab631/_images/plans-available-all.png
POST /v1/users/search

Searches through User attributes Documents for the specified Account.

Note

The initial User Schema for a new account only enables searching on system fields. You can update the User Schema if you want additional fields. See the Schemas API for details.

Note

Each request consumes one Operation for the search function, and an additional Operation for each user record returned. So, a search returning 20 users will consume 21 Operations.

Form Parameters:
 
  • search_option – string(req’d) - search_option as a base64 encoded JSON Document
Request Headers:
 
Status Codes:
  • 200 OK – success
  • 400 Bad Request – invalid vault_id, invalid base64 encoded JSON Document, invalid options in search_option