Greg Heartsfield home

hS3, with ByteString

Galois has released an updated haskell HTTP module, supporting ByteString requests and responses, instead of just String. This provides a significant performance improvement, and dramatically reduces memory usage. Since hS3 uses HTTP for all network communication, it wasn’t really usable for files larger than a few megabytes. I’ve pushed a new released of hS3 (0.4), and did some quick measurements to see the improvements.

I generated a 1MB file of random data, and uploaded it to S3 using HTTP-3000 (String), and HTTP-4000 (ByteString.Lazy). Real time was limited by network bandwidth, so there was no improvement in that regard. User (CPU) time was greatly improved:

HTTP-3000 (String):         35.859s
HTTP-4000 (ByteString.Lazy): 0.072s

Using the GHC profiler, I also uploaded a 5MB file using the old and new libraries, which demonstrates the reduced memory requirements. The vertical axis is the same on both graphs, but I’ve had to reduce the scale of the horizontal (time) axis on the second graph, since the ByteString version only required about 1/500th CPU to execute.

hS3 benchmark with String type hS3 benchmark with ByteString type

Validate XHTML Validate CSS