福州至曼谷来回机票价:Hbase无法启动及web查看出现500错误的原因及解决方法

来源：百度文库编辑：偶看新闻时间：2024/04/30 12:03:28

偶然一次停电，导致机房机器重启，于是很多机器静态IP冲突失效。好不容易把网络调整好，Hbase集群却无法正常启动，Hadoop却可以正常使用。大致情况如下：正常启动Hadoop和zookeeper，然后启动Hbase，jps查看一切正常。当打开web页面（60010）查看的时候却出现500错误，错误大致如下：

HTTP ERROR: 500
Trying to contact region server null for region , row ", but failed after 3 attempts.
Exceptions:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.10.11.184:60020 after attempts=1
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.10.11.184:60020 after attempts=1
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.10.11.184:60020 after attempts=1

RequestURI=/master.jsp

Caused by:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server null for region , row ", but failed after 3 attempts.
Exceptions:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.10.11.184:60020 after attempts=1
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.10.11.184:60020 after attempts=1
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.10.11.184:60020 after attempts=1

at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
at org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
at org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:324)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)

查看Hbase master的日志，大致错误如下：

2011-12-08 14:34:54,305 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.10.11.184:60020 could not be reached after 1 tries, giving up.2011-12-08 14:34:54,305 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region location changed. Sleeping.2011-12-08 14:34:55,305 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry finding root region.2011-12-08 14:34:55,307 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.10.11.184:600202011-12-08 14:34:55,308 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.10.11.184:60020 could not be reached after 1 tries, giving up.2011-12-08 14:35:15,404 INFO org.apache.hadoop.hbase.master.ServerManager: 4 region servers, 0 dead, average load 0.33333333333333332011-12-08 14:35:15,484 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scanning meta region {server: 10.10.11.184:60020, regionname: -ROOT-,,0, startKey: <>}2011-12-08 14:35:15,485 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.10.11.184:60020 could not be reached after 1 tries, giving up.2011-12-08 14:35:15,485 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT regionorg.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.10.11.184:60020 after attempts=1 at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:429) at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:918) at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:934) at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:173) at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54) at org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79) at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:153) at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
日志呈现结果大致是无法连接到 /10.10.11.184:60020 ,不能连接到regionServer于是查看regionServer的日志，初次一看感觉没什么错误，就没怎么注意，然后继续纠结在其他地方……后来多次失败后重新审视日志的时候才发现了一些端倪，如下
2011-12-08 22:45:02,365 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Created ZNode /hbase/rs/1323355502342 with data 0:0:0:0:0:0:0:1:600202011-12-08 22:45:02,395 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.regionserver.address=10.10.11.1842011-12-08 22:45:02,395 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: fs.default.name=hdfs://hadoop5-virtual-machine:9000/hbase2011-12-08 22:45:02,395 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.rootdir=hdfs://hadoop5-virtual-machine:9000/hbase2011-12-08 22:45:02,395 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us address to use. Was=0:0:0:0:0:0:0:1:60020, Now=10.10.11.1842011-12-08 22:45:02,510 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Log dir hdfs://hadoop5-virtual-machine:9000/hbase/.logs/hadoop4-virtual-machine,60020,13233555023422011-12-08 22:45:02,521 INFO org.apache.hadoop.hbase.regionserver.HLog: HLog configuration: blocksize=67108864, rollsize=63753420, enabled=true, flushlogentries=100, optionallogflushinternal=10000ms2011-12-08 22:45:02,561 INFO org.apache.hadoop.hbase.regionserver.HLog: New hlog /hbase/.logs/hadoop4-virtual-machine,60020,1323355502342/hlog.dat.13233555025222011-12-08 22:45:02,566 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=RegionServer, sessionId=regionserver/0:0:0:0:0:0:0:1:60020
然后我在slave节点下jps查看，发现hbase相关的regionServer的进程挂掉了，而master的进程还在。
由上蓝色标记字体看出：为什么会出现IP6的地址呢？检查配置文件也没有出错，因为所有的配置文件都是用主机名替代了IP地址。于是就想到了是不是hosts文件的映射出错了。

打开/etc/hosts文件，果不其然，regionServer的主机名映射的是 ::1，

我记得这个主机名和当前IP的映射应该是网卡初始化的时候由NetworkManager自动加上去的，

把 ::1改成 regionServer的实际地址，重启Hbase，访问WEB站点，成功启动！

回顾整个hadoop集群启动失败这个过程，先是断电，然后是静态IP失效，问题应该就是在这个时候产生的，在实效的同试hosts文件被修改。

其实HBASE集群配置相当简单，但由于一些小小的失误，往往是我们自以为不会出错的地方，查看log的时候也不细心，然后以至于浪费我们很多的时间。至于IP与主机名映射的问题是我们配置集群的时候最容易忽视的地方，大家在下次碰到这类相关问题的时候不妨去看看hosts文件。

无法在Web服务器上启动调试无法启动默认web站点(iis) 为什么win98无法使用“按web页查看” 运行C# .net时，出现试图运行时错误，无法在WEB上启动调试，调试失败，因为没有启动集成Windows身份验证请问如何在自己的计算机启动WEB服务器,(C# WEB程序无法打开,说没有启动WEB服务器.) 无法在Web服务器上启动调试。调试失败 win 2000 server中,FTP站点、WEB站点、管理WEB站点无法启动，请问如何做？ vs2005无法在WEB服务器启动调试。WEB服务器配置错误新建ASP.NET WEB应用程序后，启动时（F5）：报无法在 Web 服务器上启动调试。 .net中的web程序在启动时提示:无法开始调试，无法找到Microsoft Internet Explorer VS.NET调试中报错：无法在WEB伺服器上启动侦错无法取得RPC伺服器 web服务器出现500错误怎么办？无法启动机器怀疑主板出现故障机器无法启动。。。。出现下面的提示 illustrator启动后出现无法完成请求电脑断电以后，出现蓝屏，无法启动劲舞团启动出现提示，无法游戏怎样启动web讯雷阿？启动时间出现MEMORG TEST FAIL！无法正常启动 "World Wide Web Pubishing"服务无法启动。怎么回事？急啊，在线等！ asp.net无法在web服务器上启动调试，未将项目设置为进行调试 World Wide Web Publishing 无法启动,提示错误87,参数设置错误无法在 Web 服务器上启动调试。重叠 I/O 操作在进行中。试图运行项目时错误,无法在WEB服务器上启动调式??