Contact

Databricks: Automate Jobs and Clusters

USD$ 0.00*

Product number: SB10105
Free Video
Product information "Databricks: Automate Jobs and Clusters"

This integration allows users to perform end-to-end orchestration and automation of jobs and clusters in Databricks environment either in AWS or Azure.  

Key Features:

  • Uses Python module requests to make REST API calls to the Databricks environment.
  • Uses the Databricks URL and the user bearer token to connect with the Databricks environment.
  • With respect to Databricks jobs, this integration can perform the below operations:
    • Create and list jobs.
    • Get job details.
    • Run new jobs.
    • Run submit jobs.
    • Cancel run jobs.
  • With respect to the Databricks cluster, this integration can perform the below operations:
    • Create, start, and restart a cluster.
    • Terminate a cluster.
    • Get a cluster-info.
    • List clusters.
  • With respect to Databricks DBFS, this integration also provides a feature to upload files larger files.

What's New in V1.4.0

Enhancements
  • Added: repair_job option to support repairing failed or incomplete jobs.
  • Added: Support for Databricks multi-task run job outputs in Run Job.
  • Added: repair parameter for run_now and run_submit functions to enable on-demand repair.
  • Added: Enhanced Databricks job logging for improved observability and debugging.
Fixes
  • Fixed: Minor issues with Retry Interval and Max Retry configurations.
  • Fixed: Refactored large functions into smaller, modular units to improve code quality, maintainability, and readability.
 
Product Component: Universal Agent, Universal Controller
Universal Template Name: DataBricks
Version: 1.4.0
Vendor: Databricks
Type: Free
Compatibility : UC/UA 7.1 and above
Support: Community Created