Apache Hive .orig test file and "#### A masked pattern was here ####"


Just a quick information about something in Hive.

If you ever typed:

$ ant clean package test

to run Apache Hive unit tests, you may have seen that Hive sometimes creates two output files.
If you run for example:

$ ant test -Dtestcase=TestCliDriver -Dqfile=alter5.q

Hive sometimes generates a alter5.q.out and a alter5.q.out.orig :

build/ql/test/logs/clientpositive/alter5.q.out
build/ql/test/logs/clientpositive/alter5.q.out.orig

This happens because Hive uses a method to mask any local information, as local time, or local path, with the following sentence:

#### A masked pattern was here ####

So, if you check your .q.out file it should have a bunch of this sentence above covering several local information. This information needs to be covered so that the tests outputs are the same in all computers.

The .q.out.orig file has the original test output, with all the local information non covered.

Out of curiosity, the method to mask the local patterns (private void maskPatterns(String[] patterns, String fname) throws Exception) is located on:

ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java


Comments

  1. Really Good blog post.provided a helpful information.I hope that you will post more updates like thisHadoop Admin Online Training Bangalore

    ReplyDelete
    Replies
    1. Thank you so much! I will add soon a post with Hadoop admin tips. :)

      Delete

Post a Comment

Popular posts from this blog

Apache Mahout

Slope One

Error when using smooth.spline