API Reference

Data Repositories

This page covers configuration and management of View data repository objects.

Object Overview

Data repositories define where source data resides that should be crawled for further processing by View.

Endpoint, URL, and Supported Methods

Objects are managed via the crawler server API at [http|https]://[hostname]:[port]/v1.0/tenants/[tenant-guid]/metadatarules

By default, the crawler server is accessible on port 8101.

Supported methods include: GET HEAD PUT DELETE

Structure

Data repository objects can have one of many structures depending on the type of repository. An array of fully populated examples are shown below.:

[
    {
        "GUID": "4ae4294d-d135-4b21-a75d-3df5e1c84d2b",
        "TenantGUID": "default",
        "OwnerGUID": "default",
        "Name": "Local filesystem",
        "RepositoryType": "File",
        "IncludeSubdirectories": true,
        "DiskDirectory": "./files/",
        "CreatedUtc": "2024-10-22T13:57:54.000000Z"
    },
    {
        "GUID": "876c139e-e57f-44ed-b2e6-4dcb5d3677e6",
        "TenantGUID": "default",
        "OwnerGUID": "default",
        "Name": "NFS file server",
        "RepositoryType": "NFS",
        "IncludeSubdirectories": true,
        "NfsHostname": "nfsserver",
        "NfsUserId": 0,
        "NfsGroupId": 0,
        "NfsShareName": "export1",
        "NfsVersion": "V3",
        "CreatedUtc": "2024-10-22T13:58:18.000000Z"
    },
    {
        "GUID": "8fface0d-9514-4cf6-b260-827dc1c180f4",
        "TenantGUID": "default",
        "OwnerGUID": "default",
        "Name": "CIFS file server",
        "RepositoryType": "CIFS",
        "IncludeSubdirectories": true,
        "CifsHostname": "windowshost",
        "CifsUsername": "[email protected]",
        "CifsPassword": "password",
        "CifsShareName": "share3",
        "CreatedUtc": "2024-10-22T13:58:43.000000Z"
    },
    {
        "GUID": "e37d0a94-e7e3-447c-9eab-489d1baaad49",
        "TenantGUID": "default",
        "OwnerGUID": "default",
        "Name": "S3 compatible object store",
        "RepositoryType": "AmazonS3",
        "IncludeSubdirectories": true,
        "S3EndpointUrl": "http://s3storage.company.com/",
        "S3BaseUrl": "http://s3storage.company.com/{bucket}/{key}/",
        "S3AccessKey": "myaccesskey",
        "S3SecretKey": "mysecretkey",
        "S3BucketName": "bucket1",
        "S3Region": "us-west-1",
        "CreatedUtc": "2024-10-22T14:02:14.000000Z"
    },
    {
        "GUID": "c28df7e3-28c2-40a6-8203-c3ac433992c1",
        "TenantGUID": "default",
        "OwnerGUID": "default",
        "Name": "S3 bucket",
        "RepositoryType": "AmazonS3",
        "IncludeSubdirectories": true,
        "S3EndpointUrl": "https://mybucket.us-west-1.s3.amazonaws.com/",
        "S3BaseUrl": "https://{bucket}.us-west-1.s3.amazonaws.com/{key}/",
        "S3AccessKey": "myaccesskey",
        "S3SecretKey": "mysecretkey",
        "S3BucketName": "mybucket",
        "S3Region": "us-west-1",
        "CreatedUtc": "2024-10-22T14:03:13.000000Z"
    },
    {
        "GUID": "21d149e2-f405-41fe-a20a-e9a3d6073783",
        "TenantGUID": "default",
        "OwnerGUID": "default",
        "Name": "Azure BLOB storage",
        "RepositoryType": "AzureBlob",
        "IncludeSubdirectories": true,
        "AzureEndpointUrl": "https://myblobcontainer.blob.core.windows.net/",
        "AzureAccountName": "myazureaccount",
        "AzureContainerName": "myblobcontainer",
        "AzureAccessKey": "myaccesskey",
        "CreatedUtc": "2024-10-22T14:04:08.000000Z"
    }
]

Properties (all repository types):

  • GUID string globally unique identifier for the object
  • TenantGUID string globally unique identifier for the tenant
  • OwnerGUID string GUID of the owner that created the object
  • RepositoryType enum the type of repository, valid values are File NFS CIFS AmazonS3 AzureBLOB
  • Name string name of the object
  • IncludeSubdirectories bool indicates whether or not subdirectories should be crawled
  • CreatedUtc datetime timestamp from creation, in UTC time

Additional properties (Local filesystem repositories)

  • DiskDirectory string for file repositories, the directory to crawl

Additional properties (NFS repositories)

  • NfsHostname string for NFS repositories, the hostname of the server
  • NfsUserId int the NFS user ID
  • NfsGroupId int the NFS group ID
  • NfsShareName string the full name of the export on the NFS server
  • NfsVersion enum the NFS version, valid values are V2 V3 V4

Additional properties (CIFS repositories)

  • CifsHostname string for CIFS repositories, the hostname of the server
  • CifsUsername string for CIFS repositories, the username
  • CifsPassword string for CIFS repositories, the password
  • CifsShareName string for CIFS repositories, the share name

Additional properties (S3 repositories)

  • S3EndpointUrl string for S3 repositories, the endpoint URL
  • S3BaseUrl string for S3 repositories, the base URL format
    • For virtual-hosted deployments (including S3 itself), base URL should be of the form [http||https]://{bucket}.[hostname]:[port]/{key}
    • For path-style deployments, the base URL should be of the form [http||https]://[hostname]:[port]/{bucket}/{key}
  • S3AccessKey string for S3 repositories, the access key
  • S3SecretKey string for S3 repositories, the secret key
  • S3BucketName string for S3 repositories, the bucket name
  • S3Region string for S3 repositories, the region string, i.e. us-west-1

Additional properties (Azure BLOB repositories)

  • AzureEndpointUrl string for Azure BLOB repositories, the endpoint URL
  • AzureAccountName string for Azure BLOB repositories, the account name
  • AzureContainerName string for Azure BLOB repositories, the container name
  • AzureAccessKey string for Azure BLOB repositories, the access key

Create

To create, call PUT /v1.0/tenants/[tenant-guid]/datarepositories with the properties of the data repository as defined above, using the crawler server.

curl -X PUT http://localhost:8101/v1.0/tenants/[tenant-guid]/datarepositories \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer [accesskey]" \
     -d '
{
    "Name": "CIFS file server",
    "RepositoryType": "CIFS",
    "IncludeSubdirectories": true,
    "CifsHostname": "windowshost",
    "CifsUsername": "[email protected]",
    "CifsPassword": "password",
    "CifsShareName": "share3"
}'

Enumerate

Refer to the Enumeration page in REST API for details about the use of enumeration APIs.

Enumerate objects by using GET /v2.0/tenants/[tenant-guid]/datarepositories. The resultant object will appear as:

{
    "Success": true,
    "Timestamp": {
        "Start": "2024-10-21T02:36:37.677751Z",
        "TotalMs": 23.58,
        "Messages": {}
    },
    "MaxResults": 10,
    "IterationsRequired": 1,
    "EndOfResults": true,
    "RecordsRemaining": 16,
    "Objects": [
        {
            "GUID": "example-datarepository",
            ... datarepository details ...
        },
        { ... }
    ],
    "ContinuationToken": "[continuation-token]"
}

Read

To read an object by GUID, call GET /v1.0/tenants/[tenant-guid]/datarepositories/[datarepository-guid]. If the object exists, it will be returned as a JSON object in the response body. If it does not exist, a 404 will be returned with a NotFound error response.

{
    "GUID": "8fface0d-9514-4cf6-b260-827dc1c180f4",
    "TenantGUID": "default",
    "OwnerGUID": "default",
    "Name": "CIFS file server",
    "RepositoryType": "CIFS",
    "IncludeSubdirectories": true,
    "CifsHostname": "windowshost",
    "CifsUsername": "[email protected]",
    "CifsPassword": "password",
    "CifsShareName": "share3",
    "CreatedUtc": "2024-10-22T13:58:43.000000Z"
}

Note: the HEAD method can be used as an alternative to get to simply check the existence of the object. HEAD requests return either a 200/OK in the event the object exists, or a 404/Not Found if not. No response body is returned with a HEAD request.

Update

To update an object by GUID, call PUT /v1.0/tenants/[tenant-guid]/datarepositories/[datarepository-guid] with a fully populated object in the request body. The updated object will be returned to you.

Note: certain fields cannot be modified and will be preserved across updates.

Request body:

{
    "GUID": "8fface0d-9514-4cf6-b260-827dc1c180f4",
    "TenantGUID": "default",
    "OwnerGUID": "default",
    "Name": "My updated CIFS file server",
    "RepositoryType": "CIFS",
    "IncludeSubdirectories": true,
    "CifsHostname": "windowshost",
    "CifsUsername": "[email protected]",
    "CifsPassword": "password",
    "CifsShareName": "share3",
    "CreatedUtc": "2024-10-22T13:58:43.000000Z"
}

Response body:

{
    "GUID": "8fface0d-9514-4cf6-b260-827dc1c180f4",
    "TenantGUID": "default",
    "OwnerGUID": "default",
    "Name": "My updated CIFS file server",
    "RepositoryType": "CIFS",
    "IncludeSubdirectories": true,
    "CifsHostname": "windowshost",
    "CifsUsername": "[email protected]",
    "CifsPassword": "password",
    "CifsShareName": "share3",
    "CreatedUtc": "2024-10-22T13:58:43.000000Z"
}

Delete

To delete an object by GUID, call DELETE /v1.0/tenants/[tenant-guid]/datarepositories/[datarepository-guid]