Throughput comparison of Wasabi vs Amazon S3

With the removal of egress fees using Wasabi for S3 compatible storage is a very cost effective competitor to Amazon S3.

I wanted to verify if Wasabi could live up to their claims of being faster than S3 storage.  You can download a copy of their benchmark report  and take a look at their method and results, but I'll summarize:

1. They claim lower latency for operations/higher number of operations per second than S3.
2. In their testing with only 10 threads with very small objects (1MB) they claim to be 3x faster then S3
3. For larger objects (10MB) they claim to achieve the same thoughput as S3.

I wasn't interested in their claims of lower latency/higher operations, I'm purely interested in throughput, which they didn't test in their white paper.

Wasabi has only one datacenter currently in US East. 

I set up the following tests with comparisons to S3 East and S3 West.
Each test was conducted with 40 threads per client with 40 large objects of approx. 4GB per file.
Each client had a single 10Gb NIC which limited the performance of S3 West from AWS West.
S3 is capable of much higher throughput than documented below if all traffic remains within the same data center.

1. Upload and download throughput to/from AWS West
2. Upload and download throughput from Azure East
3. Multi client (3 clients) download throughput to simulate a real world deployment

Results:

Client	Wasabi	S3 East	S3 West
EC2 West Put	222	234	451
Azure East Put	430	399	287
EC2 West Get	352	436	809
Azure East Get	495	585	387
Multi client download (3 clients)	571	927	1329

From the results and graph you can see that for a single client Wasabi is able to sustain approximately the same throughput as S3.  S3 has a significant throughput advantage over Wasabi if you are in the same AWS data center, which is expected.The result that is concerning is that Wasabi can only sustain approx. 5Gbps throughput across all clients.  i.e. if you have 10 clients they will only be able to achieve 500Mbps each, etc.   For may applications this would be acceptable, but keep this in mind before choosing Wasabi as your storage platform.It is possible that as Wasabi scales out they will scale out the platform bandwidth, but given the low cost it is also possible that this is the expected maximum performance.From a compatibility perspective Wasabi claim to be 100% compatible with S3 however they only currently implement a limited subset of the S3 user and bucket policy.  I strongly advise testing your required policies and see if they are implemented yet.Update: Wasabi responded that they throttle on a per account basis.  I was able to verify that by creating a second account I was able to achieve 915Mbps download speed with 3 clients (2 accessing one bucket and 1 accessing the other).  While this isn't useful for a single client it does at least show that the limit is in software and not a hard limitation of the platform.