An all to often forgotten part of building services/applications is making sure to load test it. This process usually requires someone to manually test a set of endpoints, or if you not from the early 2000’s, to write some automated testing. There are so mamy ways of doing automated testing that range from simple to very in depth. Usually a company will have a QA team who can deal with this sort of thing. However, in smaller companies or for those just wanting to test their own application, we need to rely on other methods.

That is where Artillery comes in…

What is Artillery

So what is it? Well Artillery is a very capable, lightweight & user friendly tool for load testing and functional testing of your application. More information can be found on https://artillery.io/

To install Artillery via NPM (Node Package Manager), run the following command from your machine:

npm i -g artillery

Artillery is Open Source, but there is also a paid subscription for those wanting a enterprise solution for testing. They seem to have a good update and release routine, to keep adding in new useful features.

Lets have a little fun

So how does this help us? Lets say you have an API or website that needs some load testing. You can either use the quick run feature for a smoke test. All that is needed is a couple parameters and the endpoint that you want to test. That will run a simple test, by sending a set number of requests and at the set arrival rates to that endpoint. Once complete, it will output the results of the test in the terminal.

Here is an example of a quick run:

artillery quick --count 10 -n 20 https://bytesize.blog

This quick test will create 10 “virtual users”, each of which will send 20 requests to the home page of ByteSize blog.

This is fine as a very basic test, however a better option is to use a config file that has more complex requests and logic within it. This file needs to be in YAML format and you can set it to do some really interesting things. For example: if you have a varying number off API endpoints that you want run tests over, you can set these up in different work flows, which can contain different arrival rates, with different number of virtual users and for different durations. If the endpoints require authentication, then you can set this up in the config file as well.

Here is an example of a test.yml config file:

config:
  target: 'https://bytesize.blog'
  phases:
    - duration: 60
      arrivalRate: 10
scenarios:
  - flow:
    - get:
        url: '/'

To execute this you would use:

artillery run test.yml

After running the above test the results will be outputed to the terminal and at the bottom will be a “Summary report”. This has the following information in it:

  • Scenarios launched is the number of virtual users created in the preceding 10 seconds (or in total)
  • Scenarios completed is the number of virtual users that completed their scenarios in the preceding 10 seconds (or in the whole test). Note: this is the number of completed sessions, not the number of sessions started and completed in a 10 second interval.
  • Requests completed is the number of HTTP requests and responses or WebSocket messages sent
  • Request time is in milliseconds, and p95 and p99 values are the 95th and 99th percentile values (a request latency p99 value of 500ms means that 99 out of 100 requests took 500ms or less to complete).
  • Codes provides the breakdown of HTTP response codes received.

When running it from my machine I get the following report:

Summary report @ 07:50:33(+0100) 2020-04-26
  Scenarios launched:  600
  Scenarios completed: 600
  Requests completed:  600
  Mean response/sec: 9.94
  Response time (msec):
    min: 86.8
    max: 444.2
    median: 108.4
    p95: 176.5
    p99: 293.4
  Scenario counts:
    0: 600 (100%)
  Codes:
    200: 600

Artillery is a very cool tool to have in your toolkit, and even better is that it’s completely free, which is actually quite something considering how feature rich it is. It can be used directly on your local machine or, even better, as part of your CI/CD pipeline.

I hope you found this short article helpful in some way. If you are in need of a simple, small and powerful tool to do loadtesting, then Artillery is a great choice. So have a go and let me know what you think. So until next time, bye for now.