Week5 作业一

用户头像
Coder
关注
发布于: 2020 年 07 月 08 日

用你熟悉的编程语言实现一致性 hash 算法。

编写测试用例测试这个算法,测试 100 万 KV 数据,10 个服务器节点的情况下,计算这些 KV 数据在服务器上分布数量的标准差,以评估算法的存储负载不均衡性。



import com.google.common.collect.Maps;
import com.google.common.hash.Hashing;
import org.javers.common.collections.Lists;
import java.io.Serializable;
import java.util.*;
import java.util.concurrent.CopyOnWriteArrayList;
public class ConsistentHashMap implements Serializable {
private final int virtualNodes;
private final List<String> servers;
private SortedMap<Integer, String> consistentCircle;
private Map<String, Integer> counter;
public ConsistentHashMap(List<String> servers) {
this.servers = new CopyOnWriteArrayList<>(servers);
this.virtualNodes = 65535 / servers.size();
counter = Maps.newHashMap();
init();
}
public static void main(String[] args) {
ConsistentHashMap consistentHashMap = new ConsistentHashMap(
Lists.asList("127.0.0.1:8080", "127.0.0.1:8081", "127.0.0.1:8082", "127.0.0.1:8083", "127.0.0.1:8084", "127.0.0.1:8085", "127.0.0.1:8086", "127.0.0.1:8087", "127.0.0.1:8088", "127.0.0.1:8089"));
for (int i = 0; i < 1000000; i++) {
consistentHashMap.getServer(UUID.randomUUID().toString());
}
System.out.println(consistentHashMap.standard());
}
private int getHash(String str) {
return Hashing.murmur3_32().hashBytes(str.getBytes()).asInt();
}
private void init() {
consistentCircle = new TreeMap<>();
for (int i = 0; i < servers.size(); i++) {
for (int j = 0; j < virtualNodes; j++) {
String virtualServer = j + "#" + servers.get(i);
consistentCircle.put(getHash(virtualServer), virtualServer);
}
}
}
public String getServer(String key) {
int hash = getHash(key);
SortedMap<Integer, String> map = consistentCircle.tailMap(hash);
if (map.isEmpty()) {
map = consistentCircle;
}
Integer firstKey = map.firstKey();
String server = map.get(firstKey);
server = server.substring(server.indexOf("#") + 1);
counter.put(server, counter.getOrDefault(server, 0) + 1);
return server;
}
public String standard() {
double avg = counter.values().stream().mapToInt(a -> a).average().orElse(0);
double avgStd = counter.values().stream().map(count -> Math.pow(count - avg, 2)).mapToDouble(Double::doubleValue).average().orElse(0d);
return String.format("%.2f", Math.sqrt(avgStd));
}
}

采用Guava的hashing算法来计算hashCode值,最终计算出来的指为:1081.63

发布于: 2020 年 07 月 08 日 阅读数: 71
用户头像

Coder

关注

还未添加个人签名 2018.05.04 加入

还未添加个人简介

评论

发布
暂无评论
Week5 作业一