配置JAVA项目以在不同(虚拟)计算机上使用hadoop

我在Intellij Idea本地有一个maven项目,我想将其设置为使用我在虚拟机上安装的Hadoop.有什么建议吗？

我在本地有Windows 8.1,在虚拟机上有Ubuntu 12.0.4.我已经在那里安装了Hadoop及其工作原理.

编辑：
VM上的Hadoop设置：
core-site.xml

<configuration>
<property>
  <name>hadoop.tmp.dir</name>
  <value>/app/hadoop/tmp</value>
  <description>A base for other temporary directories.</description>
</property>

<property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri's scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri's authority is used to
  determine the host, port, etc. for a filesystem.</description>
</property>

</configuration>

和mapred-site.xml

<configuration>
<property>
  <name>mapred.job.tracker</name>
  <value>localhost:54311</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If "local", then jobs are run in-process as a single map
  and reduce task.
  </description>
</property>

</configuration>

和hdfs-site.xml

<configuration>
<property>
  <name>dfs.replication</name>
  <value>1</value>
  <description>Default block replication.
  The actual number of replications can be specified when the file is created.
  The default is used if replication is not specified in create time.
  </description>
</property>

</configuration>

我的VM(vmware播放器)上的网络是NAT,而IP地址是192.168.35.128.

然后,我在Intellij Idea中有一个简单的Java项目(虽然觉得这很重要,但无论如何…),这是我的配置：

String hdfsUrl = "hdfs://192.168.36.128:54310";
FileSystem hdfs;
final Configuration config = new Configuration();
config.set(FS_DEFAULT_NAME, hdfsUrl);
 try {
            hdfs = FileSystem.get(config);
            if (hdfs != null) {
                hdfsAvailable = true;
            } else {
                throw new IOException("Unable to get hdfs, is NULL");
            }
        } catch (IOException e) {
            logger.warn(e.toString());
        }

当我启动它时,出现以下错误：

java.io.IOException: No FileSystem for scheme: hdfs

所以很明显我缺少了一些东西.我的操作系统是Windows 8,在VM上是Ubuntu.

重要编辑第二条：

telnet 192.168.36.128 54310

是成功的,但没有从应用程序…

解决方法:

由于在类路径中缺少库hadoop-hdfs-*.jar而发生此错误.对于访问HDFS文件系统,如果仅添加hadoop-hdfs-*.jar不能解决此问题,则需要遵循以下依赖性.

hadoop-hdfs-*.jar
hadoop-common-*.jar
dependecy jars inside common/lib directory.

原文地址：https://codeday.me/bug/20191029/1962826.html