阿川CH
学海无涯,上栽上栽!
Toggle navigation
阿川CH
主页
归档
标签
跑tez任务报:Previous writer likely failed to write hdfs://xx, Failing because I am unlikely to write too
Hive
问题排查
2018-10-19 15:57:53
0
0
0
cqc
Hive
问题排查
报这个错有点坑, hive并没把真实的异常错误信息输出,而是返回一个笼统的错`Previous writer likely failed to write hdfs://xx, Failing because I am unlikely to write too`。源码如下: ``` public LocalResource localizeResource( Path src, Path dest, LocalResourceType type, Configuration conf) throws IOException { FileSystem destFS = dest.getFileSystem(conf); // We call copyFromLocal below, so we basically assume src is a local file. FileSystem srcFs = FileSystem.getLocal(conf); if (src != null && !checkPreExisting(srcFs, src, dest, conf)) { // copy the src to the destination and create local resource. // do not overwrite. String srcStr = src.toString(); LOG.info("Localizing resource because it does not exist: " + srcStr + " to dest: " + dest); Object notifierNew = new Object(), notifierOld = copyNotifiers.putIfAbsent(srcStr, notifierNew), notifier = (notifierOld == null) ? notifierNew : notifierOld; // To avoid timing issues with notifications (and given that HDFS check is anyway the // authoritative one), don't wait infinitely for the notifier, just wait a little bit // and check HDFS before and after. if (notifierOld != null && checkOrWaitForTheFile(srcFs, src, dest, conf, notifierOld, 1, 150, false)) { return createLocalResource(destFS, dest, type, LocalResourceVisibility.PRIVATE); } try { destFS.copyFromLocalFile(false, false, src, dest); synchronized (notifier) { notifier.notifyAll(); // Notify if we have successfully copied the file. } copyNotifiers.remove(srcStr, notifier); } catch (IOException e) { if ("Exception while contacting value generator".equals(e.getMessage())) { // HADOOP-13155, fixed version: 2.8.0, 3.0.0-alpha1 throw new IOException("copyFromLocalFile failed due to HDFS KMS failure", e); } LOG.info("Looks like another thread or process is writing the same file"); int waitAttempts = HiveConf.getIntVar( conf, ConfVars.HIVE_LOCALIZE_RESOURCE_NUM_WAIT_ATTEMPTS); long sleepInterval = HiveConf.getTimeVar( conf, HiveConf.ConfVars.HIVE_LOCALIZE_RESOURCE_WAIT_INTERVAL, TimeUnit.MILLISECONDS); // Only log on the first wait, and check after wait on the last iteration. if (!checkOrWaitForTheFile( srcFs, src, dest, conf, notifierOld, waitAttempts, sleepInterval, true)) { LOG.error("Could not find the jar that was being uploaded"); throw new IOException("Previous writer likely failed to write " + dest + ". Failing because I am unlikely to write too."); } } finally { if (notifier == notifierNew) { copyNotifiers.remove(srcStr, notifierNew); } } } return createLocalResource(destFS, dest, type, LocalResourceVisibility.PRIVATE); } ``` 由于tez要加载的第三方是放在本地,如果有多个HiveServer2的实例,比较容易漏掉哪个服务器忘记放第三方包。如果确认不是包的问题,则需要调试跟踪下 最好的解决方法是调试跟进去看异常信息
上一篇:
在hive on tez引擎下,orc split的计算逻辑
下一篇:
JAVA RSA工具类
文档导航