分布式性能测试框架用例方案设想（一）
发布于: 1 小时前
在近期工作规划中，分布式压测框架提上日程，目前 FunTester 已经具备了一些分布式压测中用到的功能。
例如在执行用例端：利用反射根据方法名执行方法的使用示例、命令行如何执行jar包里面的方法；或者在用例编写端：如何统一接口测试的功能、自动化和性能测试用例、如何在Linux命令行界面愉快进行性能测试；亦或前段时间探索的Groovy反射执行问题：反射执行Groovy类方法NoSuchMethodException解答、Groovy反射invokeMethod传参实践。
目前看已经有了几种粗略的性能测试用例方案，有一些已经进行了实践，有一些已经被我放弃了。分享出来，算是个梳理。
这里的测试用例方案分为两类：用例传递和用例执行。目前我的想法还是通过 HTTP 协议接口完成用例的传递和执行中控制。采用定时任务或者脚本轮询的方式进行执行的控制。目前来看肯定是一个Springboot项目了，这都是后话了。
下面分享第一种设想：
基于 HttpRequestBase 对象的压测场景
这种测试场景应该说非常少了，基于单个或者多个固定的HttpRequestBase对象的分布式压测方案，其实实现起来有点大材小用了。简单的请求，没有参数化规则，没有上下游接口调用，没有前置后置处理，缺乏链路功能支持。说了这么多缺点，下面分享基于HttpRequestBase对象的优点：实现简单，用例传递非常好做。执行起来也直接可以使用框架提供的能力。兼容性好，可以直接从功能用例中提取部分用例然后执行，达到用例多用的目的。
实现 Demo
这里需要区分用例来源。一般来讲，编写单个用例肯定绕不开一张图：
总体分成三部分：请求行、请求头和请求体。依照之前分享过的案例，将一个HttpRequestBase对象拆成三分部。例如我们获取一个请求的方式如下：
public static void main(String[] args) throws UnsupportedEncodingException { HttpPost httpPost = getHttpPost("http://localhost:12345/test/qps"); httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad")); httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET)); }
复制代码
不过这样不利于HttpRequestBase对象在 HTTP 接口中传递，毕竟没有直接用序列化和反序列化的方法。所以我自己写了一个中间对象。
最开始的想法用fastJSON实现：
public static void main(String[] args) { HttpPost httpPost = getHttpPost("http://localhost:12345/test/qps"); httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad")); httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET)); httpPost.addHeader(HttpClientConstant.ContentType_JSON); JSONObject httpResponse = getHttpResponse(httpPost); output(httpResponse); String s = JSON.toJSONString(httpPost); HttpPost httpPost1 = JSON.parseObject(s, httpPost.getClass()); JSONObject httpResponse1 = getHttpResponse(httpPost1); output(httpResponse1); }
复制代码
测试结果如下：
INFO-> 当前用户：oker，工作目录：/Users/oker/IdeaProjects/funtester/,系统编码格式:UTF-8,系统Mac OS X版本:10.16WARN-> 响应体非json格式，已经自动转换成json格式！INFO-> 请求uri：http://localhost:12345/test/qps , 耗时：236 ms , HTTPcode: 200INFO-> ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~＞ {＞ ① . "code":-2,＞ ① . "FunTester":200,＞ ① . "content":"FunTester"＞ }~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~WARN-> 获取post请求参数时异常！WARN-> 获取请求相应失败！请求内容:{requestType='POST', host='', apiName='', uri='http://localhost:12345/test/qps', headers={"Connection":"keep-alive"}, args={}, params={}, json={}, response={}}java.lang.UnsupportedOperationException: public abstract int org.apache.http.params.HttpParams.getIntParameter(java.lang.String,int) at com.alibaba.fastjson.JSONObject.invoke(JSONObject.java:485) ~[fastjson-1.2.62.jar:?] at com.sun.proxy.$Proxy22.getIntParameter(Unknown Source) ~[?:?] at org.apache.http.client.params.HttpClientParamConfig.getRequestConfig(HttpClientParamConfig.java:59) ~[httpclient-4.5.6.jar:4.5.6] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:177) ~[httpclient-4.5.6.jar:4.5.6] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.6.jar:4.5.6] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.6.jar:4.5.6] at com.funtester.httpclient.FunLibrary.getHttpResponse(FunLibrary.java:352) [classes/:?] at com.funtest.javatest.FF.main(FF.java:21) [classes/:?]INFO-> json 对象是空的！
Process finished with exit code 0
复制代码
请求发生错误，其实这里面POST请求实体拷贝会失败。所以这个方法行不通了，只能换一个自己实现的。
public static void main(String[] args) { HttpPost httpPost = getHttpPost("http://localhost:12345/post"); httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad")); httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET)); httpPost.addHeader(HttpClientConstant.ContentType_JSON); JSONObject httpResponse = getHttpResponse(httpPost); output(httpResponse); FunRequest funRequest = FunRequest.initFromRequest(httpPost); HttpRequestBase request = funRequest.getRequest(); JSONObject httpResponse1 = getHttpResponse(request); output(httpResponse1); String text = JSON.toJSONString(funRequest); HttpRequestBase httpRequestBase = FunRequest.initFromString(text).getRequest(); JSONObject httpResponse2 = getHttpResponse(httpRequestBase); output(httpResponse2); }
复制代码
测试结果如下：
INFO-> 当前用户：oker，工作目录：/Users/oker/IdeaProjects/funtester/,系统编码格式:UTF-8,系统Mac OS X版本:10.16WARN-> 响应体非json格式，已经自动转换成json格式！INFO-> 请求uri：http://localhost:12345/post , 耗时：242 ms , HTTPcode: 200INFO-> ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~＞ {＞ ① . "code":-2,＞ ① . "FunTester":200,＞ ① . "content":"FunTester"＞ }~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~WARN-> 响应体非json格式，已经自动转换成json格式！INFO-> 请求uri：http://localhost:12345/post , 耗时：1 ms , HTTPcode: 200INFO-> ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~＞ {＞ ① . "code":-2,＞ ① . "FunTester":200,＞ ① . "content":"FunTester"＞ }~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~WARN-> 响应体非json格式，已经自动转换成json格式！INFO-> 请求uri：http://localhost:12345/post , 耗时：1 ms , HTTPcode: 200WARN-> 响应体非json格式，已经自动转换成json格式！INFO-> 请求uri：http://localhost:12345/post , 耗时：2 ms , HTTPcode: 200INFO-> ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~＞ {＞ ① . "code":-2,＞ ① . "FunTester":200,＞ ① . "content":"FunTester"＞ }~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
Process finished with exit code 0
复制代码
用例创建
用例很简单，就是一个个的HttpRequestBase对象，这里用com.funtester.httpclient.FunRequest类作为中转。
代码生成 FunRequest 对象
static FunRequest getCase001() { HttpPost httpPost = getHttpPost("http://localhost:12345/post"); httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad")); httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET)); httpPost.addHeader(HttpClientConstant.ContentType_JSON); return FunRequest.initFromRequest(httpPost); }
复制代码
这样我们获取一个FunRequest对象即可。
字符串
这里分两种：一种是从代码里面创建FunRequest，用字符串信息保存。
static String getCase001() { HttpPost httpPost = getHttpPost("http://localhost:12345/post"); httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad")); httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET)); httpPost.addHeader(HttpClientConstant.ContentType_JSON); FunRequest funRequest = FunRequest.initFromRequest(httpPost); return JSON.toJSONString(funRequest); }
复制代码
第二种是原本就是字符串，或者数据库中一条记录，通过某种不可描述的方法转换成FunRequest对象。这里就分一下从字符串中读取FunRequest对象的方法，由于情况过于复杂，其他的情况我就不写了，各位有兴趣可以自己实现。
/** * 从字符串中获取请求对象 * @param fun * @return */ static FunRequest initFromString(String fun) { def f = JSON.parseObject(fun) RequestType requestType = RequestType.getInstance(f.requestType) def request = new FunRequest(requestType) request.host = f.host request.path = f.path request.uri = f.uri request.args = f.args request.json = f.json request.params = f.params f.headers.each { request.addHeader(it.name,it.value) } request }
复制代码
用例传输
这个相对接简单多了。用例一旦转换成字符串之后，就可以通过接口上传到master服务，或者由master服务分配给salve服务（暂时Springboot方案）去执行。
上传用例
这里先写一个简单的POST接口上传用例的 Demo。
static boolean updateCase() { HttpPost httpPost = getHttpPost("http://localhost:12345/updatecase"); httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad")); JSONObject json = getJson("name=FunTester"); json.put("FunTester", getCase002()); json.put("各种参数", FunTester); httpPost.setEntity(new StringEntity(json.toString(), DEFAULT_CHARSET)); httpPost.addHeader(HttpClientConstant.ContentType_JSON); JSONObject httpResponse = getHttpResponse(httpPost); return isRight(httpResponse); }
复制代码
分配用例
这里我目前是放弃Socket协议推送用例，而是让salve去master分配好的队列中取用例，当然这里的用例包含必要的运行信息，而不仅仅是一个FunRequest对象。
static JSONObject returnCase() { JSONObject json = getJson("name=FunTester"); json.put("FunTester", getCase002()); json.put("各种参数", "FunTester"); return json; }
复制代码
用例执行
salve拿到用例之后，先去解析响应，然后通过构建多线程任务对象或者list，然后交付给执行框架去完成用例的执行和数据展示和记录。
这里我为了省事儿直接用
RequestThreadTimes模型做了 Demo，其他三种暂时不赘述了。
单 HttpRequestBase 用例
相对简单，虽然可以通过组合多个性能用例来完成多个HttpRequestBase对象的性能压测，但是我非常不推荐这样，所以我在接下来的对象里面拒绝这种用例形式。
static void executeCase(CaseBase caseBase) { FunRequest funRequest = caseBase.getFunRequest(); int thread = caseBase.getThread(); int times = caseBase.getTimes(); int runup = caseBase.getRunup(); String name = caseBase.getName(); RequestThreadTimes requestThreadTimes = new RequestThreadTimes(funRequest.getRequest(), times); Concurrent concurrent = new Concurrent(requestThreadTimes, thread, name); Constant.RUNUP_TIME = runup * 1.0; concurrent.start(); }
复制代码
多 HttpRequestBase 对象
这种用力的话，我采取的方案是caseBase对象中做一下区分，获取用例解析的时候解析成一个list，然后通过线程参数thread从头开始去list取FunRequest对象，构造多线程任务类RequestThreadTimes，完事儿之后交给执行框架执行。
static void executeCase(CaseBase caseBase) { List<FunRequest> funRequests = caseBase.getFunRequests(); int thread = caseBase.getThread(); int times = caseBase.getTimes(); int runup = caseBase.getRunup(); String name = caseBase.getName(); List<ThreadBase> collect = range(thread).mapToObj(f -> new RequestThreadTimes(funRequests.get(f % funRequests.size()).getRequest(), times)).collect(Collectors.toList()); Concurrent concurrent = new Concurrent(collect, name); Constant.RUNUP_TIME = runup * 1.0; concurrent.start(); }
复制代码
FunRequest 全部代码
package com.funtester.httpclient
import com.alibaba.fastjson.JSONimport com.alibaba.fastjson.JSONObjectimport com.funtester.base.bean.RequestInfoimport com.funtester.base.exception.RequestExceptionimport com.funtester.config.HttpClientConstantimport com.funtester.config.RequestTypeimport com.funtester.frame.Saveimport com.funtester.frame.SourceCodeimport com.funtester.utils.Timeimport org.apache.commons.lang3.StringUtilsimport org.apache.http.Headerimport org.apache.http.HttpEntityimport org.apache.http.client.methods.HttpPostimport org.apache.http.client.methods.HttpPutimport org.apache.http.client.methods.HttpRequestBaseimport org.apache.http.client.methods.RequestBuilderimport org.apache.logging.log4j.LogManagerimport org.apache.logging.log4j.Logger/** * 重写FunLibrary，使用面对对象思想,不用轻易使用set属性方法,可能存在BUG */class FunRequest extends SourceCode implements Serializable, Cloneable {
private static final long serialVersionUID = -4153600036943378727L
private static Logger logger = LogManager.getLogger(FunRequest.class)
/** * 请求类型，true为get，false为post */ RequestType requestType
/** * 请求对象 */ HttpRequestBase request
/** * host地址 */ String host = EMPTY
/** * 接口地址 */ String path = EMPTY
/** * 请求地址,如果为空则由host和path拼接 */ String uri = EMPTY
/** * header集合 */ List<Header> headers = new ArrayList<>()
/** * get参数 */ JSONObject args = new JSONObject()
/** * post参数,表单 */ JSONObject params = new JSONObject()
/** * json参数,用于POST和put */ JSONObject json = new JSONObject()
/** * 响应,若没有这个参数,从将funrequest对象转换成json对象时会自动调用getresponse方法 */ JSONObject response = new JSONObject()
/** * 构造方法 * * @param requestType */ private FunRequest(RequestType requestType) { this.requestType = requestType }
/** * 获取get对象 * * @return */ static FunRequest isGet() { new FunRequest(RequestType.GET) }
/** * 获取post对象 * * @return */ static FunRequest isPost() { new FunRequest(RequestType.POST) }
/** * 获取put请求对象 * @return */ static FunRequest isPut() { new FunRequest(RequestType.PUT) }
/** * 获取delete请求对象 * @return */ static FunRequest isDelete() { new FunRequest(RequestType.DELETE) }
/** * 设置host * * @param host * @return */ FunRequest setHost(String host) { this.host = host this }
/** * 设置接口地址 * * @param path * @return */ FunRequest setpath(String path) { this.path = path this }
/** * 设置uri * * @param uri * @return */ FunRequest setUri(String uri) { this.uri = uri this }
/** * 添加get参数 * * @param key * @param value * @return */ FunRequest addArgs(Object key, Object value) { args.put(key, value) this }
/** * 添加post参数 * * @param key * @param value * @return */ FunRequest addParam(Object key, Object value) { params.put(key, value) this }
/** * 添加json参数 * * @param key * @param value * @return */ FunRequest addJson(Object key, Object value) { json.put(key, value) this }
/** * 添加header * * @param key * @param value * @return */ FunRequest addHeader(Object key, Object value) { headers << FunLibrary.getHeader(key.toString(), value.toString()) this }
/** * 添加header * * @param header * @return */ FunRequest addHeader(Header header) { headers.add(header) this }
/** * 批量添加header * * @param header * @return */ FunRequest addHeader(List<Header> header) { header.each {h -> headers << h} this }
/** * 增加header中cookies * * @param cookies * @return */ FunRequest addCookies(JSONObject cookies) { headers << FunLibrary.getCookies(cookies) this }
FunRequest addHeaders(List<Header> headers) { this.headers.addAll(headers) this }
FunRequest addHeaders(JSONObject headers) { headers.each {x -> this.headers.add(FunLibrary.getHeader(x.getKey().toString(), x.getValue().toString())) } this }
FunRequest addArgs(JSONObject args) { this.args.putAll(args) this }
FunRequest addParams(JSONObject params) { this.params.putAll(params) this }
FunRequest addJson(JSONObject json) { this.json.putAll(json) this }
/** * 获取请求响应，兼容相关参数方法，不包括file * * @return */ JSONObject getResponse() { response = response.isEmpty() ? FunLibrary.getHttpResponse(request == null ? getRequest() : request) : response response }
/** * 获取请求对象 * * @return */ HttpRequestBase getRequest() { if (request != null) request if (StringUtils.isEmpty(uri)) uri = host + path switch (requestType) { case RequestType.GET: request = FunLibrary.getHttpGet(uri, args) break case RequestType.POST: request = !params.isEmpty() ? FunLibrary.getHttpPost(uri + FunLibrary.changeJsonToArguments(args), params) : !json.isEmpty() ? FunLibrary.getHttpPost(uri + FunLibrary.changeJsonToArguments(args), json.toString()) : FunLibrary.getHttpPost(uri + FunLibrary.changeJsonToArguments(args)) break case RequestType.PUT: request = FunLibrary.getHttpPut(uri, json) break case RequestType.DELETE: request = FunLibrary.getHttpDelete(uri) break case RequestType.PATCH: request = FunLibrary.getHttpPatch(uri, params) default: break } for (Header it : headers) { if (it.getName() != HttpClientConstant.ContentType_JSON.getName()) request.addHeader(it) } logger.debug("请求信息：{}", new RequestInfo(this.request).toString()) request }
FunRequest setHeaders(List<Header> headers) { this.headers = headers this }
FunRequest setArgs(JSONObject args) { this.args = args this }
FunRequest setParams(JSONObject params) { this.params = params this }
FunRequest setJson(JSONObject json) { this.json = json this }
@Override FunRequest clone() { initFromRequest(this.getRequest()) }
@Override String toString() { return "{" + "requestType='" + requestType.getName() + '\'' + ", host='" + host + '\'' + ", path='" + path + '\'' + ", uri='" + uri + '\'' + ", headers=" + FunLibrary.header2Json(headers).toString() + ", args=" + args.toString() + ", params=" + params.toString() + ", json=" + json.toString() + ", response=" + response.toString() + '}' }
/** * 将请求对象转成curl命令行 * @return */ String toCurl() { StringBuffer curl = new StringBuffer("curl -w HTTPcode%{http_code}:代理返回code%{http_connect}:数据类型%{content_type}:DNS解析时间%{time_namelookup}:%{time_redirect}:连接建立完成时间%{time_pretransfer}:连接时间%{time_connect}:开始传输时间%{time_starttransfer}:总时间%{time_total}:下载速度%{speed_download}:speed_upload%{speed_upload} ") curl << " -X ${requestType.getName()} " headers.each { curl << " -H '${it.getName()}:${it.getValue().replace(SPACE_1, EMPTY)}'" } switch (requestType) { case RequestType.GET: args.each { curl << " -d '${it.key}=${it.value}'" } break case RequestType.POST: if (!params.isEmpty()) { curl << " -H Content-Type:application/x-www-form-urlencoded" params.each { curl << " -F '${it.key}=${it.value}'" } } if (!json.isEmpty()) { curl << " -H \"Content-Type:application/json\"" //此处多余,防止从外部构建curl命令 json.each { curl << " -d '${it.key}=${it.value}'" } } break default: break } curl << " ${uri}" // curl << " --compressed" //这里防止生成多个curl请求,批量生成有用 curl.toString() }
/** * 将请求对象转成curl命令行 * @param requestBase * @return */ static String reqToCurl(HttpRequestBase requestBase) { initFromRequest(requestBase).toCurl() }
/** * 从requestbase对象从初始化funrequest * @param base * @return */ static FunRequest initFromRequest(HttpRequestBase base) { FunRequest request = null String method = base.getMethod() String uri = base.getURI().toString() RequestType requestType = RequestType.getInstance(method) List<Header> headers = Arrays.asList(base.getAllHeaders()) if (requestType == requestType.GET) { request = isGet().setUri(uri).addHeaders(headers) } else if (requestType == RequestType.POST) { HttpPost post = (HttpPost) base HttpEntity entity = post.getEntity() if (entity == null) { request = isPost().setUri(uri).addHeader(headers) } else { Header type = entity.getContentType() String value = type == null ? EMPTY : type.getValue() String content = FunLibrary.getContent(entity) if (value.equalsIgnoreCase(HttpClientConstant.ContentType_TEXT.getValue()) || value.equalsIgnoreCase(HttpClientConstant.ContentType_JSON.getValue())) { request = isPost().setUri(uri).addHeaders(headers).addJson(JSONObject.parseObject(content)) } else if (value.equalsIgnoreCase(HttpClientConstant.ContentType_FORM.getValue())) { request = isPost().setUri(uri).addHeaders(headers).addParams(getJson(content.split("&"))) } } } else if (requestType == RequestType.PUT) { HttpPut put = (HttpPut) base String content = FunLibrary.getContent(put.getEntity()) request = isPut().setUri(uri).addHeaders(headers).setJson(JSONObject.parseObject(content)) } else if (requestType == RequestType.DELETE) { request = isDelete().setUri(uri) } else { RequestException.fail("不支持的请求类型!") } return request }
/** * 从字符串中获取请求对象 * @param fun * @return */ static FunRequest initFromString(String fun) { def f = JSON.parseObject(fun) RequestType requestType = RequestType.getInstance(f.requestType) def request = new FunRequest(requestType) request.host = f.host request.path = f.path request.uri = f.uri request.args = f.args request.json = f.json request.params = f.params f.headers.each { request.addHeader(it.name,it.value) } request }
static HttpRequestBase doCopy(HttpRequestBase base) { (HttpRequestBase) RequestBuilder.copy(base).build() }
/** * 拷贝HttpRequestBase对象 * @param base * @return */ static HttpRequestBase cloneRequest(HttpRequestBase base) { initFromRequest(base).getRequest() }
/** * 保存请求和响应 * @param base * @param response */ static void save(HttpRequestBase base, JSONObject response) { FunRequest request = initFromRequest(base) request.setResponse(response) Save.info("/request/" + Time.getDate().substring(8) + SPACE_1 + request.getUri().replace(OR, CONNECTOR).replaceAll("https*:_+", EMPTY), request.toString()) }
}
复制代码
Have Fun ~ Tester ！
FunTester，一群有趣的灵魂，腾讯云 &Boss 认证作者，GDevOps 官方合作媒体。
点击阅读阅文，查看 FunTester 历史原创集合
划线
评论
复制
发布于: 1 小时前阅读数: 4
版权声明: 本文为 InfoQ 作者【FunTester】的原创文章。
原文链接:【http://xie.infoq.cn/article/07887453b50be0deac3a98bd3】。文章转载请联系作者。
FunTester
关注
公众号：FunTester，Have Fun, Tester！ 2020.10.20 加入
Have Fun,Tester！
评论