使用JAVA JDBC 连接 Azure HBase Cluster

作者:Rui 发布时间:August 3, 2016 分类:Azure,BIGDATA,HBase,CLOUD 浏览:661

本文章将介绍如何使用Java JDBC 连接到Azure HDInsight HBase群集进行进行Hive查询。

先决条件

  1. 先在Azure创建HBase 群集
  2. 安装JDK 7.0 以上
  3. 安装Maven

Azure Hive JDBC 连接字符串

jdbc:hive2://CLUSTERNAME.azurehdinsight.cn:443/default;ssl=true?hive.server2.transport.mode=http;hive.server2.thrift.http.path=/hive2

将CLUSTERNAME替换为你创建的Azure HDInsight 群集名称

认证

当访问 Azure HDInsight 创建连接时,需要使用用户名和密码进行认证,该用户名和密码即为创建群集时,填写的用户名和密码

DriverManager.getConnection(connectionString,clusterAdmin,clusterPassword);

JDBC 驱动

远程登录到Azure HDInsight 头结点,下载 连接HBase所需的JDBC的jar包,下载的文件为:

C:\apps\dist\hive-0.14.0.2.2.9.1-7\lib\hive-jdbc-0.14.0.2.2.9.1-7-standalone.jar
C:\apps\dist\hadoop-2.6.0.2.2.9.1-7\share\hadoop\common\hadoop-common-2.6.0.2.2.9.1-7.jar
C:\apps\dist\hadoop-2.6.0.2.2.9.1-7\share\hadoop\common\lib\hadoop-auth-2.6.0.2.2.9.1-7.jar

创建Maven项目,并添加JDBC依赖

将上面下载的jar包拷贝到项目中,并在POM中添加以下依赖

<dependency>
    <groupId>hadoop-auth</groupId>
    <artifactId>hadoop-auth</artifactId>
    <version>2.6.0.2.2.9.1-1</version>
    <scope>system</scope>
    <systemPath>${project.basedir}/hadoop-auth-2.6.0.2.2.9.1-1.jar</systemPath>
</dependency>
<dependency>
    <groupId>hadoop-common</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>2.6.0.2.2.9.1-1</version>
    <scope>system</scope>
    <systemPath>${project.basedir}/hadoop-common-2.6.0.2.2.9.1-1.jar</systemPath>
</dependency>
<dependency>
    <groupId>hive-jdbc</groupId>
    <artifactId>hive-jdbc</artifactId>
    <version>2.6.0.2.2.9.1-1</version>
    <scope>system</scope>
    <systemPath>${project.basedir}/hive-jdbc-0.14.0.2.2.9.1-1-standalone.jar</systemPath>
</dependency>                

编写代码,连接Azure HBase群集

String connectionString = "jdbc:hive2://CLUSTERNAME.azurehdinsight.cn:443/default;ssl=true?hive.server2.transport.mode=http;hive.server2.thrift.http.path=/hive2";
try {
    //Load the HiveServer2 JDBC driver
    Class.forName("org.apache.hive.jdbc.HiveDriver");
    Connection conn = DriverManager.getConnection(connectionString, "admin", "Password");
    Statement stmt = conn.createStatement();

    String sql = "describe hivesampletable";
    System.out.println("\nGetting a description of the table:");
    ResultSet result = stmt.executeQuery(sql);
    while (result.next()) {
        System.out.println(result.getString(1) + "\t" + result.getString(2));
    }
    sql = "select count(*) from hivesampletable";
    result = stmt.executeQuery(sql);
    while (result.next()) {
        System.out.println(result.getInt(1));
    }
} catch (SQLException e) {
    e.printStackTrace();
} catch (ClassNotFoundException e) {
    e.printStackTrace();
}

Troubleshooting

SSL “Peer Not Authenticated” error
JDBC使用SSL方式访问Azure HBase服务,并且中国Azure服务使用 Wosign做证书,默认没有在JDK的证书库中,所以需要将Wosign的更证书导入到JDK中作为信任证书,才能够正常访问Azure HDInsight服务
具体导入步骤,请参考:
http://arui.me/index.php/archives/118/

标签: azure

添加新评论 »