设计一个简单的压力测试工具

本篇是架构师训练营的一道作业题,记录一下。

0x01 问题

架构师训练营第七周的作业有一道题:

用你熟悉的编程语言写一个Web性能压测工具,输入参数:URL,请求总数,并发数。 输出参数:平均响应时间,95%响应时间。 用这个测试工具以10并发、100次请求压测www.baidu.com

0x02 分析需求

首先,程序接受压测的参数,命令行工具更合适,所以考虑用命令行工具的方式实现;

其次,程序需要对参数所指定的URL按参数指定的并发数和总请求数进行测试,最后还要输出统计指标,这里需要协调并发数和请求数之间的任务处理。

一个是要保证每轮测试的并发请求数等于并发数,另一个是要保证总的请求数全部处理完成。

最后就是统计指标结果的计算,要注意计算中途的精度不能掉。

0x03 编码实现

首先是Worker的封装,负责处理压测的并发数和总请求数之间的协调逻辑,以及统计指标的输出。

package worker

import "fmt"

type worker struct {
	url           string
	totalReqNum   int
	concurrentNum int
	jobsCh        chan struct{}
	resultCh      chan int64
}

func NewWorker(url string, concurrentNum int, totalReqNum int) *worker {
	return &worker{
		url:           url,
		concurrentNum: concurrentNum,
		totalReqNum:   totalReqNum,
		jobsCh:        make(chan struct{}, totalReqNum),
		resultCh:      make(chan int64, totalReqNum),
	}
}

type WorkFunc interface {
	DoWork() int64
}

func (w *worker) BuildWorker(wf WorkFunc) {
	for i := 1; i <= w.concurrentNum; i++ {
		go doWork(wf, w.jobsCh, w.resultCh)
		//fmt.Println("worker ", i, " initialized")
	}
}

func (w *worker) BuildJobs() {
	for i := 0; i < w.totalReqNum; i++ {
		w.jobsCh <- struct{}{}
		//fmt.Println("add job ", i+1)
	}
}

func (w *worker) PrintStatistic() {
	totalRespTime := int64(0)
	nfpRespTime := int64(0)
	nfpCount := int(float64(w.totalReqNum) * 0.95)
	for i := 0; i < w.totalReqNum; i++ {
		t := <-w.resultCh
		totalRespTime += t
		if nfpCount >= i {
			nfpRespTime += t
		}
	}

	fmt.Println("")
	fmt.Printf("avg response time:\t%.2Fs\n", calcRespTime(totalRespTime, w.totalReqNum))
	fmt.Printf("95%% response time:\t%.2Fs\n", calcRespTime(nfpRespTime, nfpCount))
	fmt.Println("")
}

func calcRespTime(totalNanoTime int64, totalCount int) float64 {
	return float64(totalNanoTime)/float64(totalCount)/float64(1000000000)
}

func doWork(wf WorkFunc, jobs <-chan struct{}, respTimeCh chan<- int64) {
	for range jobs {
		respTime := wf.DoWork()
		respTimeCh <- respTime
		//fmt.Println("resp time: ", respTime)
	}
}

测试用例:

package worker

import (
	"github.com/magiconair/properties/assert"
	"testing"
	"time"
)

type mockWorkFunc struct {
	now int64
}

func (m *mockWorkFunc) DoWork() int64 {
	return m.now
}

func TestWorker_BuildWorker(t *testing.T) {
	total := 1
	w := &worker{
		url:           "http://www.example.com",
		totalReqNum:   total,
		concurrentNum: 1,
		jobsCh:        make(chan struct{}, total),
		resultCh:      make(chan int64, total),
	}
	now := time.Now().Unix()

	w.BuildWorker(&mockWorkFunc{now: now})

	w.jobsCh <- struct{}{}

	res := <-w.resultCh
	assert.Equal(t, res, now)
}

func TestWorker_BuildJobs(t *testing.T) {
	total := 5
	w := &worker{
		url:           "http://www.example.com",
		totalReqNum:   total,
		concurrentNum: 1,
		jobsCh:        make(chan struct{}, total),
		resultCh:      make(chan int64, total),
	}

	w.BuildJobs()

	var count int

	for range w.jobsCh {
		count++
		if count == total {
			break
		}
	}

	assert.Equal(t, count, total)
}

func TestWorker_PrintStatistic(t *testing.T) {
	total := 5
	w := &worker{
		url:           "http://www.example.com",
		totalReqNum:   total,
		concurrentNum: 1,
		jobsCh:        make(chan struct{}, total),
		resultCh:      make(chan int64, total),
	}

	now := time.Now().Unix()
	w.BuildWorker(&mockWorkFunc{now: now})
	w.BuildJobs()
	w.PrintStatistic()
}

worker对压测的任务通过WorkFunc接口来隔离,在Golang中也可以直接传入一个函数(更函数式),这里用接口来隔离再面向对象。

UrlWorkFun处理具体的测试任务:

package worker

import (
	"fmt"
	"net/http"
	"time"
)

type urlWorkFunc struct {
	url string
}

func NewUrlWorkFunc(url string) WorkFunc {
	return &urlWorkFunc{url: url}
}

func (u *urlWorkFunc) DoWork() int64 {
	start := time.Now().UnixNano()
	_, err := http.Get(u.url)
	if err != nil {
		fmt.Println(err)
	}
	//time.Sleep(1 * time.Second)
	end := time.Now().UnixNano()

	return end - start
}

测试用例:

package worker

import (
	"github.com/magiconair/properties/assert"
	"testing"
)

func Test_DoWork(t *testing.T) {
	u := &urlWorkFunc{url: "https://www.baidu.com"}
	latency := u.DoWork()
	assert.Equal(t, latency >= 0, true)
}

main函数处理命令行参数的校验,以及业务逻辑的协调:

package main

import (
	"flag"
	"fmt"
	"net/url"
	"pressure-test-toy/pkg/worker"
)

var (
	targetURL     = flag.String("url", "", "target URL for pressure test")
	concurrentNum = flag.Int("concurrentNum", 1, "concurrency number")
	totalReqNum   = flag.Int("totalReqNum", 1, "total request number")
)

func main() {
	flag.Parse()
	if *targetURL == "" {
		flag.Usage()
		return
	}

	_, err := url.ParseRequestURI(*targetURL)
	if err != nil {
		fmt.Println("invalid target url: ", *targetURL)
		return
	}

	u := worker.NewUrlWorkFunc(*targetURL)
	w := worker.NewWorker(*targetURL, *concurrentNum, *totalReqNum)

	w.BuildWorker(u)
	w.BuildJobs()
	w.PrintStatistic()
}

以上即为全部的实现代码。

0x04 编译

编写Makefile以编译代码:

#!make

# version
version := "v0.0.1"

# app name
app := "pttoy"

# platform
darwin := ${app}-darwin.${version}.bin
linux := ${app}-linux.${version}.bin

default: darwin

linux:

	CGO_ENABLE=0 GOOS=linux GOARCH=amd64 go build -o "${linux}" cmd/main.go
	upx ${linux}

darwin:

	CGO_ENABLE=0 GOOS=darwin GOARCH=amd64 go build -o "${darwin}" cmd/main.go
	upx ${darwin}

默认编译Mac下的可执行文件,要编译Linux下的可执行文件使用make linux即可。

0x05 运行

经过上面的编译会得到可以执行文件:pttoy-darwin.v0.0.1.bin

直接运行,或加上-h参数都会打印出使用帮助。

➜  pressure-test-toy git:(master) ./pttoy-darwin.v0.0.1.bin
Usage of ./pttoy-darwin.v0.0.1.bin:
  -concurrentNum int
    	concurrency number (default 1)
  -totalReqNum int
    	total request number (default 1)
  -url string
    	target url for pressure test

url参数不正确时,会打印使用帮助:

➜  pressure-test-toy git:(master) ./pttoy-darwin.v0.0.1.bin -url aaa -concurrentNum 1 -totalReqNum 2
invalid target url:  aaa

正确传参时,输出如下:

➜  pressure-test-toy git:(master) ./pttoy-darwin.v0.0.1.bin -url https://www.baidu.com -concurrentNum 10 -totalReqNum 100

avg response time:	0.08s
95% response time:	0.07s

上面的输出即为题目要求的以10个并发,总共100次请求压测www.baidu.com的结果。

全部代码已经上传到了代码仓库,链接在下面。

Reference:
  1. 代码仓库