As organizations move and provision resources in Microsoft Azure they find themselves with the need to analyze and monitor resource consumption for trends, review billing data to get a pulse on the organization’s return on cloud spend, and be in a better position to predict and manage operational costs.

Billing and resource usage data in Azure is made available to users in two forms:

  1. Power BI – Using the ‘Azure Enterprise’ content pack for Power B provides a glimpse into the last 12 months azure consumption.
  2. Azure Billing REST APIs – The Usage API and Rate Card API provided by the Azure Resource Manager allow users to retrieve resource consumption and billing information for an organization’s Azure subscription.

Problem

Accessing the billing data by any of the available means is not straightforward. There are a few things to consider with each option. This blog highlights different methods to access azure billing data with Power BI or via other alternative means when Power BI is not the best solution.

Three possible solutions are illustrated below where the first two use Power BI as the end user reporting interface and the third is a custom UI. In this blog ‘Azure Billing and Usage REST API’ will be referred to as ‘API’; and ‘Azure Billing and Usage Data’ will be referred to as ‘data’ to minimize verbosity.

Read More…

The Extract, Transform & Load (ETL) process typically accounts for 70% of an entire data warehousing/data migration effort (according to The Data Warehouse ETL Toolkit). And this proportion represents a major element of the project in terms of risk: the implementation could make or break the project. This fact should be taken into consideration while designing and implementing an ETL solution.

Every staging project has two elements: the ETL itself and a staging data structure. So today I’d like to talk about best practices for standing up a staging area using SQL Server Integration Services [ETL] and hosting a staging database in SQL Server 2012 [DB]. The ‘best practices’ are across three areas: Architecture, Development, and Implementation & Maintenance of the solution. Each area represents key patterns and practices (not a comprehensive list) for the ETL component and the data structure of the staging database.

And since I presume readers are looking for best practices silver bullets, that is what I will try to deliver. What you will not find in this post are turnkey ETL performance optimizations.

Read More…