On a recent project I was tasked with importing data into Power BI from a Hive table. For those of you who are new to Azure or Big Data, Hive is a data warehousing infrastructure for Hadoop which sits in the HDInsight stack on Azure. The primary purpose of Hive is to provide data summarisation, query and analysis for big data sets. In this blog I’m going to take you through the steps and note any Gotchas so that you can connect to Hive using Power BI.
Connecting to Hive
As Hive is part of the Azure HDInsight stack it would be tempting to select the HDInsight or Hadoop connector when you’re getting data. However, note HDFS in brackets beside the Azure HDInsight and Hadoop File options as this means that you’ll be connecting to the underlying data store, which can be Azure Data Lake Store or Azure Blob Storage – both of which use HDFS architectures.
But this doesn’t help when you want to access a Hive table. In order to access a Hive table you will first of all need to install the Hive ODBC driver from Microsoft. Once you’ve downloaded and installed the driver you’ll be able to make your connection to Hive using the ODBC connector in PowerBI.
You will need to input a connection string to connect even though it says optional. The format of the connection string is as follows:
Driver={Microsoft Hive ODBC Driver};Host=hdinsightclustername.azurehdinsight.net;Port=443;Schema=default; RowsFetchedPerBlock=10000; HiveServerType=2; AuthMech=6; DefaultStringColumnLength=200;
One the next screen you’ll be asked to enter a username and password. The credentials used here are not what you use to access Azure but the credentials you created when you set up the HDInsight cluster and use to login to the cluster.
Click connect and you’ll be able to pull through the tables you need into Power BI. Or, if you want to be selective in what is returned, you can write a HiveQL query in the ODBC dialog. It’s also worth noting that at the moment it’s only possible to do an import of Hive Data in Power BI and not perform Direct Query, so if your data set is huge you’ll want to summarise the data or be really selective in what is returned first.
Part 1 – Introduction to Geospatial data
Geospatial data is a niche area that is increasingly becoming utilised in the data analytics
Jan
Logic Apps: Saving files sent via Emails into Blob Storage
Using Microsoft Logic Apps we save files to Azure Blob storage that are sent to
Jan
Create and setup B2B Integration Account in Azure
An integration account allows you to build Logic Apps with enterprise B2B capabilities by adding
Jan
Time to move to the cloud? What to expect when migrating to Azure
Moving to the cloud can be complex. Here’s a simple breakdown you need to know
Dec
Serverless Transformation Data Pipelines with Serverless SQL Pools
What are Serverless SQL Pools? Serverless SQL Pools or SQL on-demand is a serverless distributed
Dec
Basic structure of x12 Edi message
All EDI transactions get defined by EDI message standards. It is vital to have proper
Dec
MultiVendor Cloud Security with the Microsoft Stack
Over Last few years there has been a large rise in identity based attacks and
Nov
Azure Logic Apps B2B EDI integration capabilities
One of the standard and most commonly used suites of protocols for B2B data transfer
Nov