On a recent project I was tasked with importing data into Power BI from a Hive table. For those of you who are new to Azure or Big Data, Hive is a data warehousing infrastructure for Hadoop which sits in the HDInsight stack on Azure. The primary purpose of Hive is to provide data summarisation, query and analysis for big data sets. In this blog I’m going to take you through the steps and note any Gotchas so that you can connect to Hive using Power BI.
Connecting to Hive
As Hive is part of the Azure HDInsight stack it would be tempting to select the HDInsight or Hadoop connector when you’re getting data. However, note HDFS in brackets beside the Azure HDInsight and Hadoop File options as this means that you’ll be connecting to the underlying data store, which can be Azure Data Lake Store or Azure Blob Storage – both of which use HDFS architectures.
But this doesn’t help when you want to access a Hive table. In order to access a Hive table you will first of all need to install the Hive ODBC driver from Microsoft. Once you’ve downloaded and installed the driver you’ll be able to make your connection to Hive using the ODBC connector in PowerBI.
You will need to input a connection string to connect even though it says optional. The format of the connection string is as follows:
Driver={Microsoft Hive ODBC Driver};Host=hdinsightclustername.azurehdinsight.net;Port=443;Schema=default; RowsFetchedPerBlock=10000; HiveServerType=2; AuthMech=6; DefaultStringColumnLength=200;
One the next screen you’ll be asked to enter a username and password. The credentials used here are not what you use to access Azure but the credentials you created when you set up the HDInsight cluster and use to login to the cluster.
Click connect and you’ll be able to pull through the tables you need into Power BI. Or, if you want to be selective in what is returned, you can write a HiveQL query in the ODBC dialog. It’s also worth noting that at the moment it’s only possible to do an import of Hive Data in Power BI and not perform Direct Query, so if your data set is huge you’ll want to summarise the data or be really selective in what is returned first.
How has the travel and transport sector reacted to the Covid-19 crisis?
The Covid-19 pandemic shocked the world. It sent unprecedented ripples of change within numerous sectors
Sep
Databricks for Dummies: An Introduction to Databricks
Databricks is an organisation and industry-leading commercial cloud-based data engineering platform for processing and transforming
Sep
Prepare and pass DP-300 (Administrating relational databases on Microsoft Azure)
Azure Database Administration? An Azure Database Administrator is responsible for implementing, managing, troubleshooting and operating the data
Sep
Resolving merge conflicts in Azure DevOps via extension
Part of the role of source control is to ensure that any simultaneous updates to
Aug
What Adatis and Microsoft’s Cloud Solution Provider (CSP) Licencing can offer you
New licencing announcements from Microsoft: Microsoft has recently announced new updates are now available in
Aug
How To Build Accountable Project Teams
When projects go wrong, as they sometimes do, blame is often shifted to the project
Aug
Microsoft Inspire 2021- A 2-day announcement roundup
Microsoft Inspire 2021- A 2-day announcement roundup Recently Microsoft’s annual Inspire partner conference kicked
Jul
Rapidly Deploying Azure Policies In Terraform
As you may all know Terraform serves a great purpose in deploying resources and infrastructure
Jul