python利用thrift连接hive

python利用thrift连接hive


分类：
python编程
hadoop+hive+hbase

2012-05-03 15:30
211人阅读
评论(0)
收藏
举报

Thrift是一个跨语言服务部署框架，最初由Facebook于2007年开发，后于2008年进入Apache孵化器(Apache Incubator)。类似于SOAP，COM 和CORBA，Thrift通过定义一个中间定义语言和Thrift代码生成工具，生成指定语言的代码。

目前，Thrift支持C++,Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk和OCaml的代码生成

1. 安装thrift依赖库

yum install automake libtool flex bison pkgconfig gcc-c++ boost-devel libevent-devel zlib-devel python-devel ruby-devel

下载thrift:http://mirror.bjtu.edu.cn/apache//thrift/0.8.0/thrift-0.8.0.tar.gz

安装thrift

tar
-zxvfthrift-0.8.0.tar.gz

cd
thrift-0.8.0

./configure
--with-boost=/usr/local

make&&
make install

我的hive使用的是cloudera cdn3u3版本，python版本为2.7

在使用Python连接hive之前需要将hive中的文件拷贝到python的sys.path中

cp -r $HIVE_PATH/lib/py /usr/local/lib/python2.7/site-packages

[html] view plaincopyprint?

  1. #!/usr/bin/envpython
  2. importsys
  3. fromhive_serviceimportThriftHive
  4. fromhive_service.ttypesimportHiveServerException
  5. fromthriftimportThrift
  6. fromthrift.transportimportTSocket
  7. fromthrift.transportimportTTransport
  8. fromthrift.protocolimportTBinaryProtocol
  9. defhiveExe(sql):
  10. try:
  11. transport=TSocket.TSocket('127.0.0.1',10000)
  12. transport=TTransport.TBufferedTransport(transport)
  13. protocol=TBinaryProtocol.TBinaryProtocol(transport)
  14. client=ThriftHive.Client(protocol)
  15. transport.open()
  16. client.execute(sql)
  17. print"Thereturnvalueis:"
  18. printclient.fetchAll()
  19. print"............"
  20. transport.close()
  21. exceptThrift.TException,tx:
  22. print'%s'%(tx.message)
  23. if__name__=='__main__':
  24. hiveExe("select*fromt_afan_test")
#!/usr/bin/env python
import sys
from hive_service import ThriftHive
from hive_service.ttypes import HiveServerException
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
def hiveExe(sql):
try:
transport = TSocket.TSocket('127.0.0.1', 10000)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = ThriftHive.Client(protocol)
transport.open()
client.execute(sql)
print "The return value is : "
print client.fetchAll()
print "............"
transport.close()
except Thrift.TException, tx:
print '%s' % (tx.message)
if __name__ == '__main__':
hiveExe("select * from t_afan_test")
标签 , ,

发表回复