How to add jar file in hive on AWS server?

1. Create file class Add extends UDF

  • Download library : hive-exec-0.11.0

  • Export file demo.jar .

2.   Connect hive server on AWS.

Use WinSCP to connect hive job of AWS and open command line and input “hive”.

3.  Hive Resources.

Hive can manage the addition of resources to a session where those resources need to be made available at query execution time. The resources can be files, jars, or archives. Any locally accessible file can be added to the session. Once a resource is added to a session, Hive queries can refer to it by its name (in map/reduce/transform clauses) and the resource is available locally at execution time on the entire Hadoop cluster. Hive uses Hadoop’s Distributed Cache to distribute the added resources to all the machines in the cluster at query execution time. Usage:

  • FILE resources are just added to the distributed cache. Typically, this might be something like a transform script to be executed.
  • JAR resources are also added to the Java classpath. This is required in order to reference objects they contain such as UDFs.
  • ARCHIVE resources are automatically unarchived as part of distributing them.

Example:

It is not neccessary to add files to the session if the files used in a transform script are already available on all machines in the Hadoop cluster using the same path name. For example:

 
Here 

 is an executable available on all machines.

 
Here 

 may be accessible via an NFS mount point that’s configured identically on all the cluster nodes. Read more here : https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli

Leave a Reply

Your email address will not be published. Required fields are marked *