Important: This documentation covers Yarn 1 (Classic).
For Yarn 2+ docs and migration guide, see yarnpkg.com.

Package detail

@lightdash/databricks-sql

databricks8Apache 2.00.1.8-beta.3TypeScript support: included

Driver for connection to Databricks SQL via Thrift API.

databricks sql, hiveserver2, apache, hive, hs2, client, thrift, nodejs, js, hive, hive js

readme

Databricks SQL Driver for NodeJS (Beta)

http://www.apache.org/licenses/LICENSE-2.0.txt npm test coverage

Description

The Databricks SQL Driver for NodeJS is a Javascript driver for applications that connect to Databricks clusters and SQL warehouses. This project is a fork of Hive Driver which connects via Thrift API.

NOTE: This Driver is Beta.

Documentation

For detailed documentation and usage examples, read the Getting Started guide.

Requirements

  • Node.js 14 or newer

Installation

npm i @databricks/sql

Usage

examples/usage.js

const { DBSQLClient } = require('@databricks/sql');

const client = new DBSQLClient();

client
  .connect({
    host: '********.databricks.com',
    path: '/sql/1.0/endpoints/****************',
    token: 'dapi********************************',
  })
  .then(async (client) => {
    const session = await client.openSession();

    const queryOperation = await session.executeStatement('SELECT "Hello, World!"', { runAsync: true });
    const result = await queryOperation.fetchAll();
    await queryOperation.close();

    console.table(result);

    await session.close();
    await client.close();
  })
  .catch((error) => {
    console.log(error);
  });

Run Tests

Unit tests

You can run all unit tests, or specify a specific test to run:

npm test
npm test <path/to/file.test.js>

e2e tests

Before running end-to-end tests, create a file named tests/e2e/utils/config.local.js and set the Databricks SQL connection info:

{
    host: '***.databricks.com',
    path: '/sql/1.0/endpoints/***',
    token: 'dapi***',
    database: ['catalog', 'database'],
}

Then run

npm run e2e
npm run e2e <path/to/file.test.js>

Contributing

See CONTRIBUTING.md

Issues

If you find any issues, feel free to create an issue or send a pull request directly.

License

Apache License 2.0

changelog

Release History

0.1.x (Unreleased)

  • DBSQLClient.openSession now takes a limited set of options (OpenSessionRequest instead of Thrift's TOpenSessionReq)
  • DBSQLClient.openSession now uses the latest protocol version by default
  • Direct results feature is now available for all IOperation methods which support it. To enable direct results feature, maxRows option should be used
  • FunctionNameRequest type renamed to FunctionsRequest
  • IDBSQLConnectionOptions type renamed to ConnectionOptions
  • IFetchOptions renamed to FetchOptions
  • DBSQLOperation.getSchema will wait for operation completion, like DBSQLOperation.fetchChunk/DBSQLOperation.fetchAll. It also supports the same progress reporting options
  • runAsync option is now available for all operations that support it

0.1.8-beta.2 (2022-09-08)

  • Operations will wait for cluster to start instead of failing
  • Added support for DirectResults, which speeds up data fetches by reducing the number of server roundtrips when possible
  • DBSQLOperation interface simplified: HiveUtils were removed and replaced with new methods DBSQLOperation.fetchChunk/DBSQLOperation.fetchAll. New API implements all necessary waiting and data conversion routines internally
  • Better TypeScript support
  • Thrift definitions updated to support additional Databricks features
  • User-agent string updated; a part of user-agent string is configurable through DBSQLClient's clientId option
  • Connection now uses keep-alive (not configurable at this moment)
  • DBSQLClient now prepends slash to path when needed
  • DBSQLOperation: default chunk size for data fetching increased from 100 to 100.000

Upgrading

DBSQLClient.utils was permanently removed. Code which used utils.waitUntilReady, utils.fetchAll and utils.getResult to get data should now be replaced with the single DBSQLOperation.fetchAll method. Progress reporting, previously supported by utils.waitUntilReady, is now configurable via DBSQLOperation.fetchChunk/DBSQLOperation.fetchAll options. DBSQLOperation.setMaxRows also became an option of methods mentioned above.

0.1.8-beta.1 (2022-06-24)

  • Initial release