Showing posts from 2020

Manage Secrets in Azure Databricks Using Azure Key Vault

  To manage credentials   Azure Databricks   offers Secret Management. Secret Management allows users to share credentials in a secure mechanism. Currently Azure Databricks offers two types of   Secret Scopes : Azure Key Vault-backed : To reference secrets stored in an Azure Key Vault, you can create a secret scope backed by Azure Key Vault. Azure Key Vault-backed secrets are only supported for Azure Databricks Premium Plan. Databricks-backed : A Databricks-backed scope is stored in (backed by) an Azure Databricks database. You create a Databricks-backed secret scope using the Databricks CLI (version 0.7.1 and above). Creating Azure Key Vault Open a Web Browser. I am using Chrome. Enter the URL  and hit enter. Sign in into your Azure Account. After successfully logging to Azure Portal, you should see the following screen. Click on "All Services" on the top left corner. Search for  "Azure Key Vault"  in the  "All Services"  search

Analytics Solution with Azure Databricks

  Introduction Databricks is the unified analytics solution powered by Apache Spark, which simplifies data science with a powerful, collaborative, and fully managed machine learning platform. The major analytics solution consists of the following: Collaborative data science : Simplify and accelerate data science by providing a collaborative environment for data science and machine learning models. Reliable data engineering : Large-scale data processing for batch and streaming workloads. Production machine learning : Standardize machine learning life-cycles from experimentation to production. In this guide, you will learn how to perform machine learning using notebooks in Databricks. The following sections will guide you through five steps to build a machine learning model with Databricks. Step One: Login to Databricks The first step is to go to  this link  and click  Try Databricks  on the top right corner of the page. Once you provide the details, it will take you to the following pag

Indexing in Azure Cosmos DB

  Even though Cosmos DB automatically indexes every property by default, understanding how indexing works in Cosmos DB is vital for achieving efficient query performance. In  Azure Cosmos DB , every property in our items are indexed by default. This is fantastic for developers, as this means we don’t have to spend time managing indexing ourselves. However, there may be times where we do want to customize the indexing policy depending on the requirements of our workloads. The purpose of this article is to show you how indexing works in Azure Cosmos DB, what kinds of indexes there are in Cosmos DB and how we can employ different indexing strategies to optimize performance depending on what we’re trying to achieve. How indexing works in Cosmos DB Azure Cosmos DB persists our items within our containers as JSON documents. We can think of these documents as trees and each property in our item as a node within that tree. Say that I have a document for a customer, and that customer has multip