The definition for a single-node cluster can look like that:
new_cluster: spark_version: 15.4.x-scala2.12 node_type_id: i4i.large autotermination_minutes: 5 runtime_engine: STANDARD data_security_mode: SINGLE_USER spark_conf: "spark.databricks.cluster.profile": "singleNode" "spark.master": "local[*, 4]" custom_tags: "ResourceClass": "SingleNode" workload_type: clients: notebooks: false jobs: true num_workers: 0