hive 的主流文件存储格式对比实验

（1）创建表，存储数据格式为TEXTFILE    create table log_text (    track_time string,    url string,    session_id string,    referer string,    ip string,    end_user_id string,    city_id string    )ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE ;
（2）向表中加载数据  load data local inpath '/export/servers/hivedatas/log.data' into table log_text ;
（3）查看表中数据大小，大小为18.1M  dfs -du -h /user/hive/warehouse/myhive.db/log_text;  结果显示:   18.1 M  /user/hive/warehouse/log_text/log.data

（1）创建表，存储数据格式为ORC    create table log_orc(    track_time string,    url string,    session_id string,    referer string,    ip string,    end_user_id string,    city_id string    )ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS orc ;
（2）向表中加载数据  insert into table log_orc select * from log_text ;
（3）查看表中数据大小  dfs -du -h /user/hive/warehouse/myhive.db/log_orc;  结果显示:  2.8 M  /user/hive/warehouse/log_orc/123456_0

1）创建表，存储数据格式为parquet    create table log_parquet(    track_time string,    url string,    session_id string,    referer string,    ip string,    end_user_id string,    city_id string    )ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS PARQUET ;  
2）向表中加载数据  insert into table log_parquet select * from log_text ;
3）查看表中数据大小  dfs -du -h /user/hive/warehouse/myhive.db/log_parquet;  结果显示:  13.1 M  /user/hive/warehouse/log_parquet/123456_0

hive (default)> select count(*) from log_text;_c0100000Time taken: 21.54 seconds, Fetched: 1 row(s)  

hive (default)> select count(*) from log_orc;_c0100000Time taken: 20.867 seconds, Fetched: 1 row(s) 

hive (default)> select count(*) from log_parquet; _c0100000Time taken: 22.922 seconds, Fetched: 1 row(s)

1）建表语句    create table log_orc_none(    track_time string,    url string,    session_id string,    referer string,    ip string,    end_user_id string,    city_id string    )ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS orc tblproperties ("orc.compress"="NONE");2）插入数据  insert into table log_orc_none select * from log_text ;3）查看插入后数据  dfs -du -h /user/hive/warehouse/myhive.db/log_orc_none;  结果显示:  7.7 M  /user/hive/warehouse/log_orc_none/123456_0

1）建表语句    create table log_orc_snappy(    track_time string,    url string,    session_id string,    referer string,    ip string,    end_user_id string,    city_id string    )ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS orc tblproperties ("orc.compress"="SNAPPY");2）插入数据  insert into table log_orc_snappy select * from log_text ;3）查看插入后数据  dfs -du -h /user/hive/warehouse/myhive.db/log_orc_snappy ;  结果显示:   3.8 M  /user/hive/warehouse/log_orc_snappy/123456_04）上一节中默认创建的ORC存储方式，导入数据后的大小为  2.8 M  /user/hive/warehouse/log_orc/123456_0  比Snappy压缩的还小。原因是orc存储文件默认采用ZLIB压缩。比snappy压缩的小。5）存储方式和压缩总结：  在实际的项目开发当中，hive表的数据存储格式一般选择：orc或parquet。压缩方式一般选择snappy。