Changes between Version 15 and Version 16 of Hadoop

Jan 19, 2010, 11:29:30 AM (14 years ago)



  • Hadoop

    v15 v16  
    3636After running the tests, in the reduce phase, we needed a way of sending the results to the centralized management console. The problem was that the classic JUnit runner returner a Result class, that wasn't serializable. So we transformed this class into a byte array, encoded into a Base64 format and send the result in this format and upon receiving it, we would decode the bytes and recreate the object.
     38= How to use it =
     39We present bellow the requirements for running JUnit on top of Hadoop using our solution
     40 * Java
     41 * Hadoop 2.0
     42 * JUnit 4
     44This is a development tool. So its purpose is to be used during development and testing. To run a test class with HadoopRunner, simple adnotate the class with @RunWith(HadoopRunner.class). We provide a Makefile with several basic commands:
     45 * setup - starts the Hadoop daemons
     46 * shutdown - closes the Hadoop daemons
     47 * build - will compile the test classes and the HadoopRunner class and build a jar to be send into the HDFS.
     48 * run - runs the tests. If the RunWith adnotation is missing, this rule will run the test with a usual JUnit runner.
     50We aaded on the SVN a sample test class: [ TestAdd], with 5 test methods. Two of them shoudl pass, the other three should fail. Also, this class needs a simple class: X, just to demonstrate how HadoopRunner loads class at runtime using our resource server.