Presto API Reference Guide

This page describes the presto (Presto data sources) endpoint.

Note

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Presto workflow

Create a Presto data source.
Search Presto data sources.
Update Presto data sources.

Create a data source

EndpointQuery ParametersPayload ParametersResponse Parameters

Endpoint

Method	Path	Purpose
POST	`/presto/handler`	Save the provided connection information as a data source.

Query Parameters

None.

Payload Parameters

Attribute	Description	Required
private	`boolean` When `false`, the data source will be publicly available in the Immuta UI.	Yes
blobHandler	`array[object]` A list of full URLs providing the locations of all blob store handlers to use with this data source.	Yes
blobHandlerType	`string` Describes the type of underlying blob handler that will be used with this data source (e.g., `MS SQL`).	Yes
recordFormat	`string` The data format of blobs in the data source, such as `json`, `xml`, `html`, or `jpeg`.	Yes
type	`string` The type of data source: `ingested` (metadata will exist in Immuta) or `queryable` (metadata is dynamically queried).	Yes
name	`string` The name of the data source. It must be unique within the Immuta instance.	Yes
sqlTableName	`string` A string that represents this data source's table in the Query Engine.	Yes
organization	`string` The organization that owns the data source.	Yes
category	`string` The category of the data source.	No
description	`string` The description of the data source.	No
owner	`array[object]` Users and groups that should be added as owners to this data source. Profiles must be a list of profile IDs and groups must be a list of group IDs: `{ "profiles": [3, 5], "groups": [4, 1999] }`.	No
expert	`array[object]` Users and groups that should be added as expert users to this data source. Profiles must be a list of profile IDs and groups must be a list of group IDs: `{ "profiles": [87, 199], "groups": [324] }`.	No
ingest	`array[object]` Users and groups that should be added as ingest users to this data source. Profiles must be a list of profile IDs and groups must be a list of group IDs: `{ "profiles": [34, 23], "groups": [32] }`.	No
hasExamples	`boolean` When `true`, the data source contains examples.	No

Response Parameters

Attribute	Description
id	`integer` The handler ID.
dataSourceId	`integer` The ID of the data source.
warnings	`string` This message describes issues with the created data source, such as the data source being unhealthy.
connectionString	`string` The connection string used to connect the data source to Immuta.

Request example

This request creates a Presto data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/presto/handler

Payload example

{
  "handler": {
    "metadata": {
      "staleDataTolerance": 86400,
      "schemaProjectName": "Public",
      "bodataSchemaName": "public",
      "columns": [{
        "name": "c_customer_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_customer_id",
        "dataType": "text",
        "remoteType": "char(16)",
        "nullable": true
      }, {
        "name": "c_current_cdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_hdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_addr_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_shipto_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_sales_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_salutation",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }, {
        "name": "c_first_name",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_last_name",
        "dataType": "text",
        "remoteType": "varchar(30)",
        "nullable": true
      }, {
        "name": "c_preferred_cust_flag",
        "dataType": "text",
        "remoteType": "char(1)",
        "nullable": true
      }, {
        "name": "c_birth_day",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_month",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_year",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_country",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_login",
        "dataType": "text",
        "remoteType": "char(13)",
        "nullable": true
      }, {
        "name": "c_email_address",
        "dataType": "text",
        "remoteType": "varchar(50)",
        "nullable": true
      }, {
        "name": "c_last_review_date",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }],
      "hostname": "example-presto.host.io",
      "port": 8080,
      "ssl": false,
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": "",
      "userFiles": [],
      "database": "your_database",
      "sid": "postgres",
      "table": "customer",
      "schema": "public",
      "bodataTableName": "customer",
      "dataSourceName": "Customer"
    }
  },
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Presto",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true,
    "name": "Customer",
    "sqlTableName": "customer"
  },
  "schemaEvolution": {
    "ownerProfileId": 1,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "connectionString": "your-username@us-east-1/tpc"
  "id": 1,
  "dataSourceId": 1,
  "dataSource": {
    "blobHandler": {
      "accessKey": "REDACTED",
      "url": "https://REDACTED/presto/handler/1",
      "ca": ["-----BEGIN CERTIFICATE-----\nyourdXRhIENcertificate\n-----END CERTIFICATE-----"],
      "manualDictionary": false
    },
    "blobHandlerType": "Presto",
    "recordFormat": "Not Provided",
    "type": "queryable",
    "schemaEvolutionId": 1,
    "columnEvolutionEnabled": true,
    "name": "Customer",
    "sqlTableName": "customer",
    "sqlSchemaName": "public",
    "workspace": null,
    "rowCount": 0,
    "seeded": false,
    "owner": {},
    "documentation": "# Customer",
    "statsExpiration": "2022-07-19T18:55:21.300Z",
    "recordCount": 0,
    "createdBy": 1,
    "policyHandler": null,
    "subscriptionType": "manual",
    "id": 1,
    "description": null,
    "deleted": false,
    "policyHandlerType": "None",
    "subscriptionPolicy": null,
    "globalPolicies": null,
    "status": null,
    "statusInfo": null,
    "expiration": null,
    "catalogMetadata": null,
    "sddTemplate": null,
    "createdAt": "2022-07-19T18:55:21.302Z",
    "updatedAt": "2022-07-19T18:55:21.302Z",
    "tags": []
  },
  "warnings": []
}

Search Presto data sources

Search for handler metadata

EndpointQuery ParametersResponse Parameters

Endpoint

Method	Path	Purpose
GET	`/presto/handler/{handlerId}`	Get the handler metadata associated with the provided handler ID.

Query Parameters

Attribute	Description	Required
handlerId	`integer` The specific handler ID.	Yes
skipCache	`boolean` If `true` the handler cache will be skipped when retrieving the handler data.	No

Response Parameters

Attribute	Description
dataSourceId	`integer` The data source ID.
value	`metadata` Details regarding the handler.

Request example

The following request returns the handler metadata associated with the provided handler ID.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/presto/handler/67

Response example

{
  "dataSourceId": 67,
  "metadata": {
    "ssl": true,
    "port": 1433,
    "query": null,
    "table": "table_108",
    "schema": "elliott_dev",
    "database": "your_database",
    "hostname": "example-presto.host.io",
    "username": "your-username",
    "eventTime": null,
    "dataSourceName": "Dev Table 108",
    "bodataTableName": "table_108",
    "disableClassify": false,
    "highCardinality": "col_1",
    "bodataSchemaName": "dev",
    "columnsNormalized": false,
    "schemaProjectName": "Dev",
    "staleDataTolerance": 2592000
  },
  "type": "odbcHandler",
  "connectionString": "your-username@example-presto.host.io:1433/your_database",
  "remoteTableDescription": "[{\"name\":\"col_1\",\"isPrimaryKey\":null,\"dataType\":\"int\",\"nullable\":\"YES\"},{\"name\":\"col_2\",\"isPrimaryKey\":null,\"dataType\":\"decimal(5,2)\",\"nullable\":\"YES\"},{\"name\":\"col_3\",\"isPrimaryKey\":null,\"dataType\":\"varchar(10)\",\"nullable\":\"YES\"}]",
  "id": 67,
  "createdAt": "2021-09-17T14:15:10.256Z",
  "updatedAt": "2021-09-17T14:15:19.547Z",
  "dbms": {
    "name": "presto"
  }
}

Update Presto data sources

Endpoint	Purpose
`/presto/handler/{handlerId}`	Updates the handler metadata associated with the provided handler ID.
`/presto/bulk`	Updates the data source metadata associated with the provided connection string.
`/presto/handler/{handlerId}/triggerHighCardinalityJob`	Recalculates the high cardinality column for the provided handler ID.

Update handler metadata

EndpointQuery ParametersResponse Parameters

Endpoint

Method	Path	Purpose
PUT	`/presto/handler/{handlerId}`	Updates the handler metadata associated with the provided handler ID. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case it uses the current dictionary.

Query Parameters

Attribute	Description	Required
handlerId	`integer` The specific handler ID.	Yes
skipCache	`boolean` If `true` the handler cache will be skipped when retrieving the handler data.	No

Response Parameters

Attribute	Description
dataSourceId	`integer` The data source ID.
body	`array[object]` Details regarding the handler, including schema, name format, and data source metadata.

Request example

This request updates the data source name to Marketing Data for the data source with the handler ID 67.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example_payload.json
    https://your-immuta-url.com/presto/handler/67

Request payload example

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 67,
    "metadata": {
      "sid": "postgres",
      "ssl": false,
      "port": 8080,
      "query": null,
      "table": "customer",
      "schema": "public",
      "database": "your_database",
      "hostname": "presto-example.io",
      "ephemeral": true,
      "eventTime": null,
      "userFiles": [],
      "dataSourceName": "Marketing Data",
      "bodataTableName": "customer",
      "highCardinality": "c_customer_sk",
      "bodataSchemaName": "public",
      "columnsNormalized": false,
      "schemaProjectName": "Public",
      "staleDataTolerance": 0,
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": "",
      "columns": [{
        "name": "c_customer_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_customer_id",
        "dataType": "text",
        "remoteType": "char(16)",
        "nullable": true
      }, {
        "name": "c_current_cdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_hdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_addr_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_shipto_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_sales_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_salutation",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }, {
        "name": "c_first_name",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_last_name",
        "dataType": "text",
        "remoteType": "varchar(30)",
        "nullable": true
      }, {
        "name": "c_preferred_cust_flag",
        "dataType": "text",
        "remoteType": "char(1)",
        "nullable": true
      }, {
        "name": "c_birth_day",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_month",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_year",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_country",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_login",
        "dataType": "text",
        "remoteType": "char(13)",
        "nullable": true
      }, {
        "name": "c_email_address",
        "dataType": "text",
        "remoteType": "varchar(50)",
        "nullable": true
      }, {
        "name": "c_last_review_date",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }]
    },
    "type": "odbcHandler",
    "connectionString": "presto-example.io:8080/postgres/public",
    "remoteTableDescription": "[{\"Column\":\"c_customer_sk\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_customer_id\",\"Type\":\"char(16)\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_current_cdemo_sk\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_current_hdemo_sk\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_current_addr_sk\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_first_shipto_date_sk\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_first_sales_date_sk\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_salutation\",\"Type\":\"varchar(10)\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_first_name\",\"Type\":\"varchar(20)\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_last_name\",\"Type\":\"varchar(30)\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_preferred_cust_flag\",\"Type\":\"char(1)\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_birth_day\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_birth_month\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_birth_year\",\"Type\":\"integer\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_birth_country\",\"Type\":\"varchar(20)\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_login\",\"Type\":\"char(13)\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_email_address\",\"Type\":\"varchar(50)\",\"Extra\":\"\",\"Comment\":\"\"},{\"Column\":\"c_last_review_date\",\"Type\":\"varchar(10)\",\"Extra\":\"\",\"Comment\":\"\"}]",
    "id": 1,
    "createdAt": "2022-07-19T18:55:21.220Z",
    "updatedAt": "2022-07-19T18:55:23.466Z",
    "dbms": {
      "name": "presto"
    }
  }
}

Response example

{
  "id": 67,
  "ca": ["-----BEGIN CERTIFICATE-----\nMIIuyourcertificate\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "c_customer_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_customer_id",
      "dataType": "text",
      "remoteType": "char(16)",
      "nullable": true
    }, {
      "name": "c_current_cdemo_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_current_hdemo_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_current_addr_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_first_shipto_date_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_first_sales_date_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_salutation",
      "dataType": "text",
      "remoteType": "varchar(10)",
      "nullable": true
    }, {
      "name": "c_first_name",
      "dataType": "text",
      "remoteType": "varchar(20)",
      "nullable": true
    }, {
      "name": "c_last_name",
      "dataType": "text",
      "remoteType": "varchar(30)",
      "nullable": true
    }, {
      "name": "c_preferred_cust_flag",
      "dataType": "text",
      "remoteType": "char(1)",
      "nullable": true
    }, {
      "name": "c_birth_day",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_month",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_year",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_country",
      "dataType": "text",
      "remoteType": "varchar(20)",
      "nullable": true
    }, {
      "name": "c_login",
      "dataType": "text",
      "remoteType": "char(13)",
      "nullable": true
    }, {
      "name": "c_email_address",
      "dataType": "text",
      "remoteType": "varchar(50)",
      "nullable": true
    }, {
      "name": "c_last_review_date",
      "dataType": "text",
      "remoteType": "varchar(10)",
      "nullable": true
    }]
  }
}

Update multiple data sources

EndpointQuery ParametersResponse Parameters

Endpoint

Method	Path	Purpose
PUT	`/presto/bulk`	Updates the data source metadata associated with the provided connection string.

Query Parameters

Attribute	Description	Required
body	`array[object]` This payload includes data source metadata and specifies the connection string.	Yes

Response Parameters

Attribute	Description
bulkId	`integer` The bulk handler metadata ID.
connectionString	`string` The specified connection string.
jobsCreated	`integer` Number of jobs created.

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/presto/bulk

Payload example

The payload below adds a certificate (certificate.json) to the data sources with the provided connection.

{
  "handler": {
    "metadata": {
      "sid": "postgres",
      "ssl": false,
      "port": 8080,
      "database": "your_database",
      "hostname": "presto-example.io",
      "userFiles": [{
        "keyName": "certificate",
        "filename": "576d8b38e9e8bc3749599489408a0b9f605b2a8d.md",
        "userFilename": "certificate.json"
      }],
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": ""
    }
  },
  "connectionString": "presto-example.io:8080/postgres/your_database"
}

Response example

{
  "bulkId": "bulk_ds_update_657dd563e6e746069bf040de5e6909a9",
  "connectionString": "presto-example.io:8080/postgres/your_database",
  "jobsCreated": 4
}

Recalculate high cardinality columns

EndpointQuery ParametersResponse Parameters

Endpoint

Method	Path	Purpose
PUT	`/presto/handler/{handlerId}/triggerHighCardinalityJob`	Recalculates the high cardinality column for the provided handler ID.

Query Parameters

Attribute	Description	Required
handlerId	`integer` The specific handler ID.	Yes

Response Parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

The following request recalculates the high cardinality column for the provided handler ID.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/presto/handler/67/triggerHighCardinalityJob

Response example

25424a50-17df-11ec-b388-0fe1d33b5af1