"Warehouse Discovery" is the process of scanning the contents of a warehouse and putting it into the data catalog
“Warehouse Discovery” is the process of scanning and virtualizing all the tables within a preconfigured data warehouse. As the name suggests, Warehouse Discovery looks at a warehouse, brings all the data into the catalog (via data virtualization, so it doesn’t change location), and harvests the metadata from the warehouse to display on the platform and index it in the search engine. Essentially, in one action, we let users represent the contents of their data warehouse(s) in our catalog to make it discoverable.
Warehouses are configured during platform setup. For help configuring a warehouse to the ThinkData Platform, please reach out to our solutions team.
Once the Warehouse has been configured, setting up the discovery process is very simple. Select “add connection” from the create source workflow. Then select the “Warehouse Discovery” type. Give the connection a name (such as “My BigQuery Warehouse”) and select the appropriate warehouse from the External Warehouse dropdown.
Select “Create Connection” to add the connection to the source, then save the source.
To turn on the discovery process, switch the Table Discovery toggle from “paused” to “on”. Once you’ve done this, the platform will begin the process of scanning and virtualizing all the tables within the warehouse.