To get the columns for one or multiple tables, you can combine SHOW TABLES and DESCRIBE EXTENDED commands. The SHOW TABLEs command lists all tables within a schema while the DESCRIBE EXTENDED command returns the list of columns for each of the retrieved tables:
tables = spark.sql('SHOW TABLES IN wfc_catalog.wfc_schema').select('tableName').filter('tableName LIKE "visits_%"').collect() schemas = {} for table in tables: table_name = table.tableName schema = spark.sql(f'DESCRIBE EXTENDED wfc_catalog.wfc_schema.{table_name}').collect() schemas[table_name] = [] for c in schema: if c.col_name == '': break schemas[table_name].append({'name': c.col_name, 'type': c.data_type, 'cooment': c.comment}) display(schemas)