Databricks tips

How to get columns of one or multiple tables?

To get the columns for one or multiple tables, you can combine SHOW TABLES and DESCRIBE EXTENDED commands. The SHOW TABLEs command lists all tables within a schema while the DESCRIBE EXTENDED command ...

Continue Reading β†’

How to define permissions for the jobs in Databricks Asset Bundles?

You can define the permissions with the...permissions block, as follows: # source: https://docs.databricks.com/aws/en/dev-tools/bundles/permissions#define-specific-resource-permissions # ... ...

Continue Reading β†’

How to define a single-node cluster in Databricks Asset Bundles?

The definition for a single-node cluster can look like that: new_cluster: spark_version: 15.4.x-scala2.12 node_type_id: i4i.large autotermination_minutes: 5 runtime_engine: STANDARD...

Continue Reading β†’

How to disable triggers on dev environment with Databricks Asset Bundles?

If you use Databricks Asset Bundles (DAB) in development mode (mode: development), the deployed workflows will be prefixed by the environment name and your users, like in the next picture: Howeve...

Continue Reading β†’