Data Flows

Object Overview

Data flows are an ordered sequence of steps specifying the trigger that invokes the collection of steps, the entry step, and which steps to run based on the return condition of success, failure, or exception of each step.

Endpoint, URL, and Supported Methods

Objects are managed via the View Orchestrator server API at [http|https]://[hostname]:[port]/v1.0/tenants/[tenant-guid]/dataflows

Supported methods include: GET HEAD PUT DELETE

Structure

Objects have the following structure:

{
    "GUID": "processor",
    "TenantGUID": "default",
    "TriggerGUID": "processor",
    "StepGUID": "processor",
    "Name": "My processing pipeline data flow",
    "Notes": "My notes",
    "LogRetentionDays": 28,
    "CreatedUtc": "2024-07-10T05:10:14.000000Z",
    "Map": null
}

Properties:

GUID GUID globally unique identifier for the object
TenantGUID GUID globally unique identifier for the tenant
TriggerGUID GUID globally unique identifier for the trigger
StepGUID GUID globally unique identifier for the entry step
Name string name of the object
Notes string notes for the data flow
LogRetentionDays int the number of days data flow log files should be retained
CreatedUtc datetime timestamp from creation, in UTC time
Map obj object containing a decision tree outlining the order of steps based on success, failure, or exception criteria

Create

To create, call PUT /v1.0/tenants/[tenant-guid]/dataflows with a populated object using the Orchestrator server.

curl --location --request PUT 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
    "TriggerGUID": "00000000-0000-0000-0000-000000000000",
    "StepGUID": "00000000-0000-0000-0000-000000000000",
    "Name": "Another data flow",
    "Notes": "My notes",
    "LogRetentionDays": 7,
    "Map": {
        "StepGUID": "00000000-0000-0000-0000-000000000000",
        "Success": null,
        "Failure": null,
        "Exception": null
    }
}'

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const createDataFlow = async () => {
  try {
    const response = await orchestrator.createDataFlow({
      TriggerGUID: "00000000-0000-0000-0000-000000000000",
      StepGUID: "00000000-0000-0000-0000-000000000000",
      Name: "Another data flow",
      Notes: "My notes",
      LogRetentionDays: 7,
      Map: {
        StepGUID: "00000000-0000-0000-0000-000000000000",
        Success: null,
        Failure: null,
        Exception: null,
      },
    });
    console.log(response);
  } catch (err) {
    console.log("Error creating data flow:", err);
  }
};
createDataFlow();

Read

To read an individual object by GUID, call GET /v1.0/tenants/[tenant-guid]/dataflows/[dataflow-guid]. If the object exists, it will be returned as a JSON object in the response body. If it does not exist, a 404 will be returned with a NotFound error response. If you wish to include the Map property, include ?inclsub&incldata in the query.

curl --location 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows/92281899-6e74-4b16-b640-9b88042e5e47' \
--header 'Authorization: ••••••'

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const readDataFlow = async () => {
  try {
    const response = await orchestrator.retrieveDataFlow(
      "92281899-6e74-4b16-b640-9b88042e5e47"
    );
    console.log(response);
  } catch (err) {
    console.log("Error reading data flow:", err);
  }
};
readDataFlow();

Note: the HEAD method can be used as an alternative to get to simply check the existence of the object. HEAD requests return either a 200/OK in the event the object exists, or a 404/Not Found if not. No response body is returned with a HEAD request.

Read with steps

To read with steps by GUID, call GET /v1.0/tenants/[tenant-guid]/dataflows/[dataflow-guid]?inclsub.

curl --location 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows/92281899-6e74-4b16-b640-9b88042e5e47?inclsub=null' \
--header 'Authorization: ••••••'

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const readDataFlowWithSteps = async () => {
  try {
    const response = await orchestrator.retrieveDataFlowWithSteps(
      "92281899-6e74-4b16-b640-9b88042e5e47"
    );
    console.log("Data flow:", response);
  } catch (err) {
    console.log("Error reading data flow with steps:", err);
  }
};

readDataFlowWithSteps();

Read with log metadata

To read an individual object with log metadata by GUID, call GET /v1.0/tenants/[tenant-guid]/dataflows/[dataflow-guid]/logs?request=[request-id].

curl --location 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows/00000000-0000-0000-0000-000000000000/logs?request=00000000-0000-0000-0000-000000000000' \
--header 'Authorization: ••••••'

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const readDataFlowLogs = async () => {
  try {
    const response = await orchestrator.retrieveDataFlowLogs(
      "00000000-0000-0000-0000-000000000000",
      "00000000-0000-0000-0000-000000000000"
    );
    console.log("Data flow logs:", response);
  } catch (err) {
    console.log("Error reading data flow logs:", err);
  }
};

readDataFlowLogs();

Read with log file

To read an individual object with log file by GUID, call GET /v1.0/tenants/[tenant-guid]/dataflows/[dataflow-guid]/logfile?request=[request-id].

curl --location 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows/00000000-0000-0000-0000-000000000000/logfile?request=00000000-0000-0000-0000-000000000000' \
--header 'Authorization: ••••••'

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const readDataFlowLogFile = async () => {
  try {
    const response = await orchestrator.retrieveDataFlowLogFile(
      "00000000-0000-0000-0000-000000000000",
      "00000000-0000-0000-0000-000000000000"
    );
    console.log("Data flow log file:", response);
  } catch (err) {
    console.log("Error reading data flow log file:", err);
  }
};
readDataFlowLogFile();

Read with performance data

To read an individual object with performance data by GUID, call GET /v1.0/tenants/[tenant-guid]/dataflows/[dataflow-guid]/logfile?request=[request-id].

curl --location 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows/00000000-0000-0000-0000-000000000000/performance?request=00000000-0000-0000-0000-000000000000' \
--header 'Authorization: ••••••'

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const readDataFlowPerformanceData = async () => {
  try {
    const response = await orchestrator.retrieveDataFlowPerformanceData(
      "00000000-0000-0000-0000-000000000000",
      "00000000-0000-0000-0000-000000000000"
    );
    console.log("Data flow performance data:", response);
  } catch (err) {
    console.log("Error reading data flow performance data:", err);
  }
};
readDataFlowPerformanceData();

Read all

To read all objects, call GET /v1.0/tenants/[tenant-guid]/dataflows, which will return an array of objects.

curl --location 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows' \
--header 'Authorization: ••••••'

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const retrieveAllDataFlows = async () => {
  try {
    const response = await orchestrator.retrieveDataFlows();
    console.log("Data flows:", response);
  } catch (err) {
    console.log("Error reading data flows:", err);
  }
};

retrieveAllDataFlows();

Update

Data flows are immutable and cannot be updated.

Delete

To delete an object by GUID, call DELETE /v1.0/tenants/[tenant-guid]/dataflows/[dataflow-guid].

curl --location --request DELETE 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows/00000000-0000-0000-0000-000000000000' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data ''

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const deleteDataFlow = async () => {
  try {
    const response = await orchestrator.deleteDataFlow(
      "92281899-6e74-4b16-b640-9b88042e5e47"
    );
    console.log("Data flow deleted:", response);
  } catch (err) {
    console.log("Error deleting data flow:", err);
  }
};

deleteDataFlow();

Check existence

To check existence of an object by GUID, call HEAD /v1.0/tenants/[tenant-guid]/dataflows/[dataflow-guid].

curl --location --head 'http://view.homedns.org:8000/v1.0/tenants/00000000-0000-0000-0000-000000000000/dataflows/00000000-0000-0000-0000-000000000000' \
--header 'Authorization: ••••••'

import { ViewOrchestratorSdk } from "view-sdk";

const orchestrator = new ViewOrchestratorSdk(
  "00000000-0000-0000-0000-000000000000", //tenant Id
  "default", //access token
  "http://localhost:8000/" //endpoint
);

const dataFlowExists = async () => {
  try {
    const response = await orchestrator.existsDataFlow(
      "92281899-6e74-4b16-b640-9b88042e5e47"
    );
    console.log("Data flow exists:", response);
  } catch (err) {
    console.log("Error checking data flow existence:", err);
  }
};

dataFlowExists();