前段时间的Hadoop小集群搭建了,同时也去Hbase数据库也配置上,今天准备在此环境上再把Hive搭建起来并结合Hbase。中途遇到的问题很头疼。度娘找不到太多实际的资料,再说很多都是基于旧版本的资料,为了方便日后总结先把问题及简单的解决方法记录一下。
Hive的安装直接解压跟Hadoop的一样,修改一下hive-env.sh
中的Hadoop安装目录,这里不详细讲解。
提示1:提示:需要设置数据类型,并初始化HIVE元数据到指定的数据库中。【默认为metastore数据库】,我使用的是Mysql,所以直接修改为Mysql的JDBC驱动。
Exception in thread "main" java.lang.RuntimeException: Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql)
解决方法:
修改以下几处:
1、491行 加入Mysql的主机名或IP地址与端口
489 <property>
490 <name>javax.jdo.option.ConnectionURL</name>
491 <value>jdbc:mysql://10.0.10.160:3306/hive?createDatabaseIfNotExist=true</value>
492 <description>JDBC connect string for a JDBC metastore</description>
493 </property>
2、880行
878 <property>
879 <name>javax.jdo.option.ConnectionDriverName</name>
880 <value>org.apache.derby.jdbc.EmbeddedDriver</value>
881 <description>Driver class name for a JDBC metastore</description>
882 </property>
880 行修改为: com.mysql.jdbc.Driver
3、905行
903 <property>
904 <name>javax.jdo.option.ConnectionUserName</name>
905 <value>APP</value>
906 <description>Username to use against metastore database</description>
907 </property>
905行修改为Mysql中的库名;hive
4、2360行
2358 <property>
2359 <name>hive.stats.dbclass</name>
2360 <value>fs</value>
2361 <description>
2362 Expects one of the pattern in [custom, fs].
2363 The storage that stores temporary Hive statistics. In filesystem based statistics collection ('fs'),
2364 each task writes statistics it has collected in a file on the filesystem, which will be aggregated
2365 after the job has finished. Supported values are fs (filesystem) and custom as defined in StatsSetupCon
st.java.
2366 </description>
2360行修改为:jdbc:mysql
5、476行
474 <property>
475 <name>javax.jdo.option.ConnectionPassword</name>
476 <value>mine</value>
477 <description>password to use against metastore database</description>
478 </property>
476行修改为:mysql数据库连接的密码,我设置的是:123456
提示2:
Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
或者:
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
问题:原因是system:java.io.tmpdir变量在配置文件中无法获取到实际的值,就是找不到路径,正常情况下Hive启动的时候会产生临时文件和日志文件。由于文件无法被创建,所以进程就启动不了。
解决方法:
直接给一个绝对路径。vim hive-site.xml
44行:
44 <value>/home/swper/apache-hive-2.0.0-bin/tmp</value>
49行:
49 <value>/home/swper/apache-hive-2.0.0-bin/tmp/resources</value>
1435行:
1434 <name>hive.querylog.location</name>
1435 <value>/home/swper/apache-hive-2.0.0-bin/tmp</value>
1436 <description>Location of Hive run time structured log file</description>
1437 </property>
3424行:
3422 <property>
3423 <name>hive.server2.logging.operation.log.location</name>
3424 <value>/home/swper/apache-hive-2.0.0-bin/tmp/operation_logs</value>
3425 <description>Top level directory where operation logs are stored if logging functionality is enabled</description>
3426 </property>
修改完后再次启动Hive即可。
提示3:再次启动时会有如下提示。
Logging initialized using configuration in file:/home/swper/apache-hive-2.0.0-bin/conf/hive-log4j2.properties
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
大致意思:在Hive2.0后在Mapreduce的框架上将不再支持,希望考虑使用其它的执行引擎(如tez,spark等。)暂时不知道会有什么影响。