[120] | 1 | HDFSPROXY is an HTTPS proxy server that exposes the same HSFTP interface as a |
---|
| 2 | real cluster. It authenticates users via user certificates and enforce access |
---|
| 3 | control based on configuration files. |
---|
| 4 | |
---|
| 5 | Starting up an HDFSPROXY server is similar to starting up an HDFS cluster. |
---|
| 6 | Simply run "hdfsproxy" shell command. The main configuration file is |
---|
| 7 | hdfsproxy-default.xml, which should be on the classpath. hdfsproxy-env.sh |
---|
| 8 | can be used to set up environmental variables. In particular, JAVA_HOME should |
---|
| 9 | be set. Additional configuration files include user-certs.xml, |
---|
| 10 | user-permissions.xml and ssl-server.xml, which are used to specify allowed user |
---|
| 11 | certs, allowed directories/files, and ssl keystore information for the proxy, |
---|
| 12 | respectively. The location of these files can be specified in |
---|
| 13 | hdfsproxy-default.xml. Environmental variable HDFSPROXY_CONF_DIR can be used to |
---|
| 14 | point to the directory where these configuration files are located. The |
---|
| 15 | configuration files of the proxied HDFS cluster should also be available on the |
---|
| 16 | classpath (hdfs-default.xml and hdfs-site.xml). |
---|
| 17 | |
---|
| 18 | Mirroring those used in HDFS, a few shell scripts are provided to start and |
---|
| 19 | stop a group of proxy servers. The hosts to run hdfsproxy on are specified in |
---|
| 20 | hdfsproxy-hosts file, one host per line. All hdfsproxy servers are stateless |
---|
| 21 | and run independently from each other. Simple load balancing can be set up by |
---|
| 22 | mapping all hdfsproxy server IP addresses to a single hostname. Users should |
---|
| 23 | use that hostname to access the proxy. If an IP address look up for that |
---|
| 24 | hostname returns more than one IP addresses, an HFTP/HSFTP client will randomly |
---|
| 25 | pick one to use. |
---|
| 26 | |
---|
| 27 | Command "hdfsproxy -reloadPermFiles" can be used to trigger reloading of |
---|
| 28 | user-certs.xml and user-permissions.xml files on all proxy servers listed in |
---|
| 29 | the hdfsproxy-hosts file. Similarly, "hdfsproxy -clearUgiCache" command can be |
---|
| 30 | used to clear the UGI caches on all proxy servers. |
---|