Skip to main content

Databricks Data Source

Introduction​

Databricks is a unified analytics platform that helps data engineering and data science teams collaborate on data processing, machine learning, and analytics at massive scale. This topic explains how to connect to Databricks data sources in your Reveal application to visualize and analyze your data.

Prerequisites

Before configuring the Databricks data source in Reveal, you must install the appropriate database driver:

These drivers are essential for establishing connections to your Databricks data source. After installation, ensure the drivers are properly configured according to Databricks documentation.

Server Configuration​

Installation​

Step 1 - Install the Reveal Databricks connector package

For ASP.NET applications, you need to install a separate NuGet package to enable Databricks support:

dotnet add package Reveal.Sdk.Data.Databricks

Step 2 - Register the Databricks data source in your application:

builder.Services.AddControllers().AddReveal( builder =>
{
    builder.DataSources.RegisterDatabricks();
});

Connection Configuration​

// Create a data source provider
public class DataSourceProvider : IRVDataSourceProvider
{
    public async Task<RVDataSourceItem> ChangeDataSourceItemAsync(IRVUserContext userContext, string dashboardId, RVDataSourceItem dataSourceItem)
    {
        // Required: Update the underlying data source
        await ChangeDataSourceAsync(userContext, dataSourceItem.DataSource);

        if (dataSourceItem is RVDatabricksDataSourceItem databricksItem)
        {            
            // Configure specific item properties as needed
            if (databricksItem.Id == "databricks_sales_data")
            {
                databricksItem.Table = "sales_data";
            }
        }
        
        return dataSourceItem;
    }

    public Task<RVDashboardDataSource> ChangeDataSourceAsync(IRVUserContext userContext, RVDashboardDataSource dataSource)
    {
        if (dataSource is RVDatabricksDataSource databricksDS)
        {
            // Configure connection properties
            databricksDS.Host = "your-databricks-instance.cloud.databricks.com";
            databricksDS.HttpPath = "/sql/1.0/warehouses/abc123";
            databricksDS.Port = 443;
            databricksDS.Database = "default";
            databricksDS.Schema = "public";
        }
        
        return Task.FromResult(dataSource);
    }
}
Important

Any changes made to the data source in the ChangeDataSourceAsync method are not carried over into the ChangeDataSourceItemAsync method. You must update the data source properties in both methods. We recommend calling the ChangeDataSourceAsync method within the ChangeDataSourceItemAsync method passing the data source item's underlying data source as the parameter as shown in the examples above.

Authentication​

Authentication for Databricks is handled on the server side using a personal access tokens. For detailed information on authentication options, see the Authentication topic.

public class AuthenticationProvider: IRVAuthenticationProvider
{
    public Task<IRVDataSourceCredential> ResolveCredentialsAsync(IRVUserContext userContext, RVDashboardDataSource dataSource)
    {
        IRVDataSourceCredential userCredential = null;
        if (dataSource is RVDatabricksDataSource)
        {
            // Use Personal Access Token
            userCredential = new RVBearerTokenDataSourceCredential("your_personal_access_token", "your_userid");
        }
        return Task.FromResult<IRVDataSourceCredential>(userCredential);
    }
}

Client-Side Implementation​

On the client side, you only need to specify basic properties like ID, title, and subtitle for the data source. The actual connection configuration happens on the server.

Creating Data Sources​

Step 1 - Add an event handler for the RevealView.onDataSourcesRequested event.

const revealView = new $.ig.RevealView("#revealView");
revealView.onDataSourcesRequested = (callback) => {
    // Add data source here
    callback(new $.ig.RevealDataSources([], [], false));
};

Step 2 - In the RevealView.onDataSourcesRequested event handler, create a new instance of the RVDatabricksDataSource object. Set the title and subtitle properties. After you have created the RVDatabricksDataSource object, add it to the data sources collection.

revealView.onDataSourcesRequested = (callback) => {
    const databricksDS = new $.ig.RVDatabricksDataSource();
    databricksDS.title = "Databricks";
    databricksDS.subtitle = "Data Source";
    
    callback(new $.ig.RevealDataSources([databricksDS], [], false));
};

When the application runs, create a new Visualization and you will see the newly created Databricks data source listed in the "Select a Data Source" dialog.

Creating Data Source Items​

Data source items represent specific datasets within your Databricks data source that users can select for visualization. On the client side, you only need to specify ID, title, and subtitle.

revealView.onDataSourcesRequested = (callback) => {
    // Create the data source
    const databricksDS = new $.ig.RVDatabricksDataSource();
    databricksDS.title = "My Databricks Datasource";
    databricksDS.subtitle = "Databricks";
    
    // Create a data source item
    const databricksDSI = new $.ig.RVDatabricksDataSourceItem(databricksDS);
    databricksDSI.id = "databricks_sales_data";
    databricksDSI.title = "My Databricks Datasource Item";
    databricksDSI.subtitle = "Databricks";

    callback(new $.ig.RevealDataSources([databricksDS], [databricksDSI], false));
};

When the application runs, create a new Visualization and you will see the newly created Databricks data source item listed in the "Select a Data Source" dialog.

Additional Resources​

API Reference​