Pokémon

Steve Desmond

Ithaca Web People

May 2, 2017

Load Testing
and Benchmarking
Web Applications

Steve Desmond

Ithaca Web People

May 2, 2017

Topics

• What is it?
• Why do it?
• How to do it?
• "Live" demos

< Disclaimer />

What are you talking about, Steve?

• Load Testing

Putting demand on a software system or computing device and measuring its response under both normal and anticipated peak load conditions -Wikipedia

• Benchmarking

Load testing a component or system to determine its performance characteristics -Testing Performance Ltd

Browser Request Breakdown

1. DNS resolution
2. Connect
3. Send request
4. Receive response
5. Parse
6. Render

Measurements

• Throughput

  requests per second (or minute)

• Latency

  response time in ms (or μs)

But why?

• Page views and conversions [1]
• Hosting costs
• Complexity
• Go green, save the world

Pop Quiz

What is Google's recommended maximum server response time?

10 ms
200 ms


500 ms
2 s

https://developers.google.com/speed/docs/insights/Server

https://www.nngroup.com/articles/response-times-3-important-limits/

Pop Quiz

How well do you know your web apps' latency and/or throughput?

No idea
General good/bad range


Numbers for some actions
Very well

Topics

• What?
• Why?
• How?
• Demos

Set a performance budget

1. How many concurrent users do we have most of the time?
2. What is the maximum number of concurrent users we expect?
3. How fast should (or conversely, how slow can) the site be under each of these conditions?

Creating a PerfLab

Pretend it's a science experiment

• Consistent, dedicated environment
• As similar as possible to production
• Minimize variables (network, CPU, etc.)
• Like unit tests, isolate what you're testing

Microsoft's ASP.NET "SmurfLab"

8-port Netgear XS708E 10-Gigabit switch

https://github.com/aspnet/benchmarks/

Microsoft's ASP.NET "SmurfLab"

Entity Framework "SmurfLab"

wrk

• multi-threaded cross-platform CLI
• written in C = minimal overhead
• HTTP GET via URL arg
• other actions, cookies, etc. via lua scripts

						
							~$ wrk {url}
						
					

Multi-threading side-note

1 thread

2 threads

(juggling metaphor)

8 threads

wrk options

• --latency: show distribution
• -t{n}: threads (default: 2)
• -c{n}: connections (default: 10)
• -d{n}: duration (default: 10s)
• -s: use lua script

						
							#post.lua
							wrk.method = "POST"
							wrk.body   = "foo=bar&baz=quux"
							wrk.headers["Content-Type"] = "application/x-www-form-urlencoded"
						
						
							~$ wrk -t1 -c1 -d10 -s post.lua http://192.168.1.100:5000/my-form
						
					

wrk demo

						~$ 
						
							Running 10s test @ http://192.168.1.100:5000
							  2 threads and 10 connections
						
						
							  Thread Stats   Avg      Stdev     Max   +/- Stdev
								Latency   229.35us  125.26us   6.34ms   96.25%
								Req/Sec    14.53k   596.05    15.33k    94.06%
							  Latency Distribution
							  	 50%  215.00us
								 75%  280.00us
								 90%  322.00us
								 99%  397.00us
							  292028 requests in 10.10s, 34.26MB read
							Requests/sec:  28915.24
							Transfer/sec:      3.39MB
							~$ 
					

Apache JMeter

• Java app with "Enterprise" UI + headless runner
• Better simulation of human interaction
• Visual output, flat file exports

Enterprise?

JMeter Demo

Load Test Toolbox

• multi-threaded cross-platform CLI
• .NET Core w/ChartJS+Node for graph output
• Focused on latency, throughput is a variable

2 tools

• Drill

  								
  									~$ ./drill.sh http://192.168.1.100:5000 100 10 drill-chart.png
  								
  							

• Hammer

  								
  									~$ ./hammer.sh http://192.168.1.100:5000 1 500 hammer-chart.png
  								
  							

Load Test Toolbox demo

						~$ 
						1: 0.71 ms
						2: 0.69 ms
						3: 0.69 ms
						4: 0.65 ms
						
							5: 0.66 ms
							~$ 
						
					

Topics

• What?
• Why?
• How?
• Demos

Now what?

Make it fast

But how?

Thanks for images

• Bulbapedia
• Pokémon Wiki