写点什么

Hadoop Java api 操作 hdfs(一)

作者:Emperor_LawD
  • 2022 年 5 月 08 日
  • 本文字数:2242 字

    阅读完需:约 7 分钟

Hadoop Java api操作hdfs(一)

新建项目

  • File->New->Module->Maven,选择quickstart


  • 设置文件名和文件地址


  • 点击完成

  • 项目目录

  • 配置pom.xml

  • 添加hadoop依赖包


    <!--添加hadoop依赖-->    <dependency>        <groupId>org.apache.hadoop</groupId>        <artifactId>hadoop-client</artifactId>        <version>${hadoop.version}</version>    </dependency>
复制代码


  • 全局定义hadoop版本


    <!--全局定义hadoop的版本-->    <hadoop.version>2.6.0-cdh5.7.0</hadoop.version>
复制代码


  • 配置好的pom.xml


    <?xml version="1.0" encoding="UTF-8"?>        <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">      <modelVersion>4.0.0</modelVersion>          <groupId>com.syh</groupId>      <artifactId>hadoopdemo</artifactId>      <version>1.0-SNAPSHOT</version>          <name>hadoopdemo</name>      <!-- FIXME change it to the project's website -->      <url>http://www.example.com</url>          <properties>        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>        <maven.compiler.source>1.7</maven.compiler.source>        <maven.compiler.target>1.7</maven.compiler.target>          <!--全局定义hadoop的版本-->        <hadoop.version>2.6.0-cdh5.7.0</hadoop.version>      </properties>      <repositories>          <repository>              <id>cloudera</id>              <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>          </repository>      </repositories>          <dependencies>        <!--添加hadoop依赖-->        <dependency>          <groupId>org.apache.hadoop</groupId>          <artifactId>hadoop-client</artifactId>          <version>${hadoop.version}</version>        </dependency>        <dependency>          <groupId>junit</groupId>          <artifactId>junit</artifactId>          <version>4.11</version>          <scope>test</scope>        </dependency>      </dependencies>          <build>        <pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->          <plugins>            <!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->            <plugin>              <artifactId>maven-clean-plugin</artifactId>              <version>3.1.0</version>            </plugin>            <!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->            <plugin>              <artifactId>maven-resources-plugin</artifactId>              <version>3.0.2</version>            </plugin>            <plugin>              <artifactId>maven-compiler-plugin</artifactId>              <version>3.8.0</version>            </plugin>            <plugin>              <artifactId>maven-surefire-plugin</artifactId>              <version>2.22.1</version>            </plugin>            <plugin>              <artifactId>maven-jar-plugin</artifactId>              <version>3.0.2</version>            </plugin>            <plugin>              <artifactId>maven-install-plugin</artifactId>              <version>2.5.2</version>            </plugin>            <plugin>              <artifactId>maven-deploy-plugin</artifactId>              <version>2.8.2</version>            </plugin>            <!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->            <plugin>              <artifactId>maven-site-plugin</artifactId>              <version>3.7.1</version>            </plugin>            <plugin>              <artifactId>maven-project-info-reports-plugin</artifactId>              <version>3.0.0</version>            </plugin>          </plugins>        </pluginManagement>      </build>    </project>
复制代码


  • 点击图中标志

  • 配置好的 Maven 目录

测试hadoop是否安装成功

  • AppTest文件中写入以下代码


  package com.syh;    import static org.junit.Assert.assertTrue;    import org.junit.Test;  import org.apache.hadoop.fs.*;    /**   * Unit test for simple App.   */  public class AppTest   {      /**       * Rigorous Test :-)       */      @Test      public void shouldAnswerWithTrue()      {          assertTrue( true );      }        @Test      public void testHadoop() {          FileSystem fileSystem = null;      }  }
复制代码


  • 按住Ctrl点击FilrSystem可以看到跳转到 hadoop 目录下,说明安装成功了








发布于: 刚刚阅读数: 4
用户头像

Emperor_LawD

关注

愿能如萤火一般,可以在黑夜里发一点光。 2021.05.19 加入

还未添加个人简介

评论

发布
暂无评论
Hadoop Java api操作hdfs(一)_hadoop_Emperor_LawD_InfoQ写作社区