Mapr Hive jdbc connection


(Sushant Dinesh) #1

Hello. I am trying to connect the ataccama dq analyzer to hive. I did some research and did not find correct .jar driver files for connection to hive 2.1 installed on mapr cluster. Can someone please help.


(Honza Nguyen) #2

Hi Sushantdinesh,

you cannot connect, because if you check the pathway, you will notice there are no .jars in the folder, hence the red cross. We have it predefined, but the .jars are not bundled with the DQ distribution.
I would recommend downloading Cloudera_HiveJDBC41_2.5.18.1050 driver. Once you have it downloaded, remove the .jars from the classpath and add the downloaded .jars to the classpath.

If you are using kerberos security, you will need to put the krb5.conf to [DQA_HOME]/jre/lib/security.
You will also need to kinit the keytab. You can use the kinit tool, which is in the [DQA_HOME]/jre/bin folder.

Let me know if you need further assistance.

With regards,
Honza.


(Sushant Dinesh) #3

Hello Vietha.

I configured a new Driver from https://www.cloudera.com/downloads/connectors/hive/jdbc/2-5-18.html

I am able to connect. But cannot query or profile the tables.

When I profile: I get 2 errors:
Cannot connect to the database
com.ataccama.dqc.commons.sql.util.PoolingDataSource$_A: Connection to database failed (org.slf4j.helpers.MessageFormatter.format(Ljava/lang/String;Ljava/lang/Object;)Ljava/lang/String;)
at com.ataccama.dqc.commons.sql.util.PoolingDataSource.A(y:364)
at com.ataccama.dqc.commons.sql.util.PoolingDataSource.A(y:665)
at com.ataccama.dqc.commons.sql.util.PoolingDataSource.getConnection(y:3336)
at com.ataccama.dqc.commons.sql.DataSourceImpl.A(y:69)
at com.ataccama.dqc.commons.sql.DataSourceImpl.getBigDataFetchingConnection(y:2614)
at com.ataccama.dqc.tasks.io.jdbc.read.JdbcReaderInstance.A(y:2718)
at com.ataccama.dqc.tasks.io.jdbc.read.JdbcReaderInstance.run(y:305)
at com.ataccama.dqc.processor.internal.runner.ComplexStepNode.runNode(y:202)
at com.ataccama.dqc.processor.internal.runner.B.run(y:2963)
at com.ataccama.dqc.commons.threads.A$_A.run(y:295)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError: org.slf4j.helpers.MessageFormatter.format(Ljava/lang/String;Ljava/lang/Object;)Ljava/lang/String;
at org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:223)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:243)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
at com.cloudera.hiveserver2.hivecommon.api.ZooKeeperEnabledExtendedHS2Factory.createClient(Unknown Source)
at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connect(Unknown Source)
at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.cloudera.hiveserver2.jdbc.common.AbstractDriver.connect(Unknown Source)
at com.ataccama.dqc.commons.sql.util.SimpleDataSource.getConnection(y:3503)
at com.ataccama.dqc.commons.sql.util.SimpleDataSource.getConnection(y:2931)
at com.ataccama.dqc.internal.commons.sql.ConnectionCreator.createResource(y:2319)
at com.ataccama.dqc.internal.commons.sql.ConnectionCreator.createResource(y:3470)
at com.ataccama.dqc.commons.resourcepool.ResourcesPool.B(y:1766)
at com.ataccama.dqc.commons.resourcepool.ResourcesPool.D(y:2145)
at com.ataccama.dqc.commons.resourcepool.A.run(y:1316)
… 1 more

Runtime error in JDBC reader: java.lang.NullPointerException
java.lang.NullPointerException
at com.ataccama.dqc.tasks.io.jdbc.read.JdbcReaderInstance.run(y:1859)
at com.ataccama.dqc.processor.internal.runner.ComplexStepNode.runNode(y:202)
at com.ataccama.dqc.processor.internal.runner.B.run(y:2963)
at com.ataccama.dqc.commons.threads.A$_A.run(y:295)
at java.lang.Thread.run(Thread.java:745)

What is the use of kinit? I ran the exe and it just asked me for my password.

Thank you for your help.

Regards,
-Sushant.


(Honza Nguyen) #4

Hi Sushant,

could you please send me a screen of your configured Apache Hive driver .jars?
Also, could you please share your JDBC connection string? You can omit the servername, I want to see the properties.

And could you please clarify - are you using Kerberos authentication? If you are, the kinit tool will allow you to create a ticket and connect to the Hive locally.

For example: I have a default.keytab file which authenticates me. To profile locally, I will need a ticket via the kinit tool. An example how to create a ticket would be:

kinit.exe -f -k -t default.keytab default@DOMAIN

With regards,
Honza.


(Sushant Dinesh) #5

Hello,
I was unsuccessful in connecting to hive on maprfs.

java.sql.SQLException: [Cloudera]HiveJDBCDriver Error initialized or created transport for authentication: java.net.ConnectException: Connection timed out: connect.
at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
at com.cloudera.hiveserver2.hivecommon.api.ZooKeeperEnabledExtendedHS2Factory.createClient(Unknown Source)
at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connect(Unknown Source)
at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.cloudera.hiveserver2.jdbc.common.AbstractDriver.connect(Unknown Source)
at com.ataccama.dqc.commons.sql.util.SimpleDataSource.getConnection(SimpleDataSource.java:89)
at com.ataccama.dqc.internal.commons.sql.AtcDataSource.getConnection(AtcDataSource.java:46)
at com.ataccama.dqc.commons.sql.util.SimpleDataSource.getConnection(SimpleDataSource.java:73)
at com.ataccama.dqc.internal.commons.sql.AtcDataSource.getConnection(AtcDataSource.java:41)
at com.ataccama.dqc.commons.sql.DataSourceImpl.getRawConnection(DataSourceImpl.java:96)
at com.ataccama.dqc.commons.sql.DataSourceImpl.getNonTransactionalConnection(DataSourceImpl.java:64)
at com.ataccama.dqc.gui.ui.wizards.database.DatabaseConnectionModel.testConnection(y:2171)
at com.ataccama.dqc.gui.ui.wizards.database.NewDatabasePage$2.run(y:1917)
Caused by: com.cloudera.hiveserver2.support.exceptions.GeneralException: [Cloudera]HiveJDBCDriver Error initialized or created transport for authentication: java.net.ConnectException: Connection timed out: connect.
… 13 more
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection timed out: connect
at org.apache.thrift.transport.TSocket.open(TSocket.java:185)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
at com.cloudera.hiveserver2.hivecommon.api.ZooKeeperEnabledExtendedHS2Factory.createClient(Unknown Source)
at com.cloudera.hiveserver2.hivecommon.core.HiveJDBCCommonConnection.connect(Unknown Source)
at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.cloudera.hiveserver2.jdbc.common.AbstractDriver.connect(Unknown Source)
at com.ataccama.dqc.commons.sql.util.SimpleDataSource.getConnection(SimpleDataSource.java:89)
at com.ataccama.dqc.internal.commons.sql.AtcDataSource.getConnection(AtcDataSource.java:46)
at com.ataccama.dqc.commons.sql.util.SimpleDataSource.getConnection(SimpleDataSource.java:73)
at com.ataccama.dqc.internal.commons.sql.AtcDataSource.getConnection(AtcDataSource.java:41)
at com.ataccama.dqc.commons.sql.DataSourceImpl.getRawConnection(DataSourceImpl.java:96)
at com.ataccama.dqc.commons.sql.DataSourceImpl.getNonTransactionalConnection(DataSourceImpl.java:64)
at com.ataccama.dqc.gui.ui.wizards.database.DatabaseConnectionModel.testConnection(y:2171)
at com.ataccama.dqc.gui.ui.wizards.database.NewDatabasePage$2.run(y:1917)
at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:119)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:180)
… 16 more

Please advise.