极客大学 - 架构师训练营第七周作业

关注
发布于: 2020 年 11 月 07 日
作业一﻿
性能压测问题性能压测的时候，随着并发压力的增加，系统响应时间和吞吐量如何变化，为什么？
答：系统性能压测一般包括：性能测试、负载测试和压力测试三个阶段。
系统响应时间：表示一个请求从开始到结束的时间。反应了系统的快慢。
吞吐量：表示系统在单位时间内，处理的请求数。体现了系统的处理能力。
并发数：系统能够同时处理的请求数。反应了系统的负载能力
要了解它们之间变化，首先我们要明白我们首先现来看一下吞吐量与并发数以及响应时间的关系：吞吐量 = (1000 * 并发数)/响应时间(ms)
根据这个关系，在正常逐渐增加压力的情况下进行测试我们有以下的三个结论:
随着并发数量的增加，在性能测试阶段，系统运行的响应时间会缓慢增加，吞吐量则会快速上升
当系统的某项或多项性能指标达到安全临界时，如果并发压力继续增加，系统的响应时间增速会加快，吞吐量增速变缓甚至出现下降
继续增加并发压力，系统进入压力测试阶段，由于超过安全负载，响应时间会极速增加，吞吐量也会快速下降，直至系统崩溃无法提供服务
由于真实环境的流量具有不稳定性，为了能够更好地获取系统性能指标，还需要进行稳定性测试。如果不均匀地增加系统并发压力，由于大量缓存失效等原因，系统的压力临界值会提前到来。我们也可以根据课件里的TPS以及响应时间的变化示意图来看一看
﻿
﻿
﻿
﻿
性能压测工具用你熟悉的编程语言写一个 Web 性能压测工具，输入参数：URL，请求总次数，并发数。输出参数：平均响应时间，95% 响应时间。用这个测试工具以 10 并发、100 次请求压测 www.baidu.com。
﻿
由于我在海外，所以用了google.com来进行测试。
﻿
作业环境
Programming language: Python3.7
OS: Linux
Github url: https://github.com/tonylixu/pyload_tester
﻿
HttpResponse.py
请求类，用来存储一次请求的内容和响应时间
"""HTTP response class"""
class HttpResponse:
    """A Class to hold request content and response time"""
    def __init__(self):
        """Initialize content and response time"""
        self._content = ''
        self._response_time = ''
    @property
    def content(self):
        return self._content
    @content.setter
    def content(self, value):
        self._content = value
    @property
    def response_time(self):
        return self._response_time
    @response_time.setter
    def response_time(self, value):
        self._response_time = value
    def print_response_time(self):
        """Print response time in certain format"""
        return f'HttpResponse{{response time = {self.response_time}ms }}'
    def print_response_content(self):
        """Print response content in certain format"""
        return f'HttpResponse{{response content = {self.content} }}'
if __name__ == '__main__':
    hr = HttpResponse()
    hr.print_response_time()
﻿
HttpClient.py
封装url请求的类
import requests
import HttpResponse
class HttpClient:
    def __init__(self, url):
        self.url = url
    def send_request(self) -> HttpResponse:
        """Send request to url
        :return: HttpResponse obj
        """
        response = HttpResponse.HttpResponse()
        try:
            request = requests.get(self.url)
        except requests.exceptions.RequestException as e:
            raise SystemExit(e)
        response.content = request.content
        response.response_time = request.elapsed.total_seconds()
        return response
﻿
LoadTester.py
压力测试类，定义压测的方法和统计数据
import HttpClient
import HttpResponse
class LoadTester:
    def __init__(self, url, sbc, times):
        self.url = url
        self.times = times
    def test_url(self):
        resp_list = []
        for i in range(self.times):
            client = HttpClient.HttpClient(self.url)
            resp = client.send_request()
            resp_list.append(resp)
        return resp_list
    @staticmethod
    def get_average_response_time(response_list):
        total_time, size = 0, len(response_list)
        for resp in response_list:
            total_time += resp.response_time
        return total_time/size
﻿
run_test.py
主要运行程序
import argparse
import LoadTester
from concurrent.futures import ThreadPoolExecutor, as_completed
import numpy as np
﻿
﻿
def create_load_tester(url, concurrent, times):
    return LoadTester.LoadTester(url, concurrent, times)
﻿
﻿
def calculate_95_percentile(response_lists):
    """Calculate 95 percentile time"""
    time_list = []
    for r_list in response_lists:
        for r in r_list:
            time_list.append(r.response_time)
    return np.percentile(time_list, 95)
﻿
﻿
def calculate_average(response_lists):
    """Calculate average time"""
    time_list = []
    for r_list in response_lists:
        for r in r_list:
            time_list.append(r.response_time)
    return np.average(time_list)
﻿
﻿
def parse_command_line_arguments():
    parser = argparse.ArgumentParser(description='Process tester argument')
    parser.add_argument('-u', dest='url', required=True, help='Test url')
    parser.add_argument('-c', dest='concurrent', type=int, required=True, help='Concurrent number')
    parser.add_argument('-t', dest='times', type=int, required=True, help='Total tests')
    return parser.parse_args()
﻿
﻿
def main():
    # Retrieve command line arguments
    args = parse_command_line_arguments()
    url, concurrent, times = args.url, args.concurrent, args.times
﻿
    # Create load tester
    lt = create_load_tester(url, concurrent, times)
﻿
    # Start load tests
    response_lists, tasks = [], []
    with ThreadPoolExecutor(max_workers=2) as executor:
        for i in range(concurrent):
            tasks.append(executor.submit(lt.test_url))
        for future in as_completed(tasks):
            response_lists.append(future.result())
﻿
    # Process test results
    ninty_five_percentile = calculate_95_percentile(response_lists)
    average_time = calculate_average(response_lists)
    print(f'Concurrent {concurrent} visit {url}, total visits {times} times, pressure tests results:\n'
          f'Average response time: {average_time}\n'
          f'95% response time: {ninty_five_percentile}')
﻿
﻿
if __name__ == "__main__":
    main()
﻿
运行结果:
﻿