Working with Databricks and Azure Cosmos DB Analytical Store
Today I would like to share my experience working on one of Quasar Services client’s projects that required a connection to Azure Cosmos DB as a data source. We will be using Azure Cosmos DB Analytical Store because of the obvious reason — we won’t affect transactional store (OLTP) performance running ETL processes. So, we will be using Azure Cosmos DB Analytical Store (OLAP) and downstream systems will consume this data. In my case, the downstream system will be Databricks. Detailed deep dive into Azure Cosmos DB Analytical Store is out of the scope of this post and you can find pretty good readings on the Microsft website ( https://learn.microsoft.com/en-us/azure/cosmos-db/analytical-store-introduction ) and in many other places. I would like to focus in this post on what would be the best option to consume data from Azure Cosmos DB Analytical Store. So first of all, the only way to access Azure Cosmos DB Analytical Store — using Azure Synapse (😲). You can use Change Feed from every...