Storage service speed testing (Windows and OS X)

Tony · March 3, 2016, 7:53pm

I’ve been doing a little bit of testing across the various services for upload speed and reliability. I thought some folks might be interested in testing, themselves.

Here are basic “upload speed testers” for both Windows and MacOS. These aren’t perfect or supremely accurate, but they should give you a general idea of your own performance for a specific source and allow you to compare across different sources.

To use them:

Open a command prompt or terminal
Change the directory to the source you want to test (ex: odrive\Dropbox).
Copy and paste the appropriate single-line command below
Adjust the test file size by changing the sizeinmb value. Low sizes are not going to be very accurate because of the “looseness” in some areas of these tests.
Adjust the number of files by changing the numfiles value.
Hit enter

Windows:
powershell -command "& {$sizeinmb = 10;$numfiles = 10;$randstr = ([char[]]([char]'a'..[char]'z') + 0..9 | sort {get-random})[0..5] -join '';$tmpdir = \"$pwd\~tmpuploadtest_$randstr\";$tmpdirstart = \"$pwd\tmpuploadtest_$randstr\";mkdir -Force $tmpdir > $null;$filebasename = \"$($sizeinmb)MB.file\";$sizeinbytes = $sizeinmb * 1024 * 1024;for($i = 1; $i -le $numfiles; $i++){$thefile = \"$tmpdir\$filebasename\" + $i;fsutil file createnew $thefile $sizeinbytes > $null;};Start-Sleep -m 8000;ren $tmpdir $tmpdirstart;echo \"\",\"Waiting for odrive\";$locked = 0;while($locked -ne 1) { $files = Get-ChildItem $tmpdirstart; for ($i=0; $i -lt $files.Count; $i++) { try { [IO.File]::OpenWrite($files[$i].FullName).close(); } catch { $locked = 1; }} Start-Sleep -m 50};echo \"Starting upload ....\";$starttime = [int64](([datetime]::UtcNow)-(get-date \"1/1/1970\")).TotalSeconds;Start-Sleep -m 5000;while($locked -ne 0) { $locked = 0; $files = Get-ChildItem $tmpdirstart; for ($i=0; $i -lt $files.Count; $i++) { try { [IO.File]::OpenWrite($files[$i].FullName).close(); } catch { $locked = 1; }} echo "uploading..."; Start-Sleep -m 1500};$endtime = [int64](([datetime]::UtcNow)-(get-date \"1/1/1970\")).TotalSeconds;$totaltime = $endtime - $starttime; $mbpersec = $sizeinmb * $numfiles / $totaltime; $mins = $totaltime / 60;echo \"$mins minutes\";echo \"$($mbpersec) MB/sec\";rm -Force -Recurse $tmpdirstart;}"

MacOS:
sizeinmb=10; numfiles=10; randstr=$(openssl rand -hex 3); tmpdir="$(pwd)/._tmpuploadtest_$randstr"; tmpdirstart="$(pwd)/tmpuploadtest_$randstr"; echo ""; mkdir "$tmpdir"; filebasename=$sizeinmb"MB.file"; sizeinbytes=$(($sizeinmb * 1024 * 1024));for (( i=1; i<=$numfiles; i++)) do thefile="$tmpdir/$filebasename$i"; < /dev/zero head -c $sizeinbytes > "$thefile"; done; echo "Waiting for odrive"; mv "$tmpdir" "$tmpdirstart"; while [[ -z $(lsof -nPl -c odrive | grep "$tmpdirstart") ]]; do sleep .05; done; starttime=$(date +%s); echo "Starting upload ...."; while [[ -n $(lsof -nPl -c odrive | grep "$tmpdirstart") ]]; do sleep .05; done; endtime=$(date +%s); totaltime=$(($endtime - $starttime)); mbpersec=$(awk "BEGIN {printf \"%.2f\",${sizeinmb}*${numfiles}/${totaltime}}"); mins=$(awk "BEGIN {printf \"%.2f\",${totaltime}/60}"); echo "$mins minutes"; echo "$mbpersec MB/sec"; rm -r "$tmpdirstart";

Tony · March 8, 2016, 3:25am

I found a little time to day to run some tests. Here are the results:

Single large file upload test:
File Size: 512MB
Instance: CenturyLink Server running Windows 2012R2 (US West. Santa Clara)

Amazon Cloud Drive:
Upload profile: Pretty smooth. Highs of 130Mbps. Lows of 105Mbps.
Upload Stats:
0.633333333333333 minutes
13.4736842105263 MB/sec

Box:
Upload profile: A bit rough, but no dramatic highs and lows. Highs of ~35Mbps with lows of ~15Mbps
Upload Stats:
2.86666666666667 minutes
2.97674418604651 MB/sec

Dropbox:
Upload profile: Somewhat jagged. Highs of ~50Mbps with lows of ~15Mbps. Some drops into sub 500Kbps range, but not often.
Upload stats:
2.83333333333333 minutes
3.01176470588235 MB/sec

Google Drive:
Upload profile: Fairly smooth, but so fast that there isn’t much of one. Highs of ~430Mbps lows of ~380Mpbs
Upload Stats:
0.183333333333333 minutes
46.5454545454545 MB/sec

OneDrive:
Upload profile: Very Jagged. Highs of 50Mbps with lows of sub-500Kbps
Upload stats:
6.68333333333333 minutes
1.27680798004988 MB/sec

OneDrive for Business:
Upload profile: Very Jagged. Highs of 55Mbps with lows of sub-500Kbps
Upload stats:
5.88333333333333 minutes
1.45042492917847 MB/sec

Single large file upload test:
File Size: 512MB
Instance: Amazon EC2 running Windows 2012R2 (us-east-1a)

Amazon Cloud Drive:
Upload profile: Pretty smooth and fast. Highs of 700Mbps. Lows of 400Mbps. Amazon Cloud Drive takes a while, relatively, to respond after everything is uploaded, so the transfer portion is actually much quicker (~45MBps).
Upload Stats:
0.6 minutes
14.2222222222222 MB/sec

Google Drive:
Upload Profile. Pretty smooth and fast. Highs of 385Mbps with lows of 210Mbps.
Upload Stats:
0.3 minutes
28.4444444444444 MB/sec

rafhob · March 8, 2016, 2:57pm

Nice stats here @Tony! Thanks for having time to do this tests. Its really interesting!

Lasse · March 11, 2016, 8:58am

Great script!

I tried it out on a couple of my linked services and thought I’d share the results.

Amazon Cloud Drive

1.05263157894737 MB/sec (20MB 1st run)
1 MB/sec (20MB 2nd run)
0.508905852417303 MB/sec (200MB)
0.941176470588235 MB/sec (512MB)

Dropbox

1.05263157894737 MB/sec (20MB 1st run)
1 MB/sec (20MB 2nd run)
0.956937799043062 MB/sec (200MB)
0.906194690265487 MB/sec (512MB)

Google Drive

2.85714285714286 MB/sec (20MB 1st run)
6.66666666666667 MB/sec (20MB 2nd run)
10 MB/sec (200MB)
6.56410256410256 MB/sec (512MB)

OneDrive

0.192307692307692 MB/sec (20MB 1st run)
0.454545454545455 MB/sec (20MB 2nd run)
0.516795865633075 MB/sec (200MB)
0.591907514450867 MB/sec (512MB)

I’m on a 250/250Mbps fibre connection and currently using OneDrive as my primary storage, but from these results it looks like I should be using Google Drive!

Edit: Inspired by your test @Tony, I did another one - this time with 512MB file. Again Google Drive by far looks like the best choice for me.

Tony · March 11, 2016, 3:24pm

Thanks for the results! Google definitely seems to have a lock on upload speeds.

This is just one factor of many for determining which storage is right for you, but it could be one of the most important, depending on your needs.

I need to get a download test in here too.

peter · March 11, 2016, 5:54pm

The other thing to consider is concurrent requests. Beyond absolute speed, the requests rate and success rate can also have an impact on big jobs. The right storage is use case dependent.

Tony · March 11, 2016, 6:15pm

Concurrent requests is the next iteration for upload testing. A little more complex to do while trying to keep everything native and in a one-liner, though.

I know that some of the services that are currently shown to be much slower in the single file test will fare better with concurrent transfers. OneDrive, for example, has a very choppy upload profile. Concurrent uploads would smooth that out so that the valleys are less dramatic and frequent.

For large single file uploads, though, it is going to be hard to beat what Google Drive is currently showing.

dougblair · March 15, 2016, 9:43pm

Just thought I would mention that the new Throttling function in release 5127 (Mac) appears to work exactly as advertised, Thank you! For what it’s worth, I upload a LOT of video data to Amazon using odrive, and the speed on my systems, using Tony’s speed check script, is

Amazon Cloud Drive
Unlimited - 2.22 MB/sec
Normal - 2.00 MB/sec
Limited - 1.02 MB/sec

ISP: Wide Open West in Chicago suburbs, 320Mbps down, 20 Mbps up

Does 2.2 MegaBYTES/sec sound reasonable with 20 MegaBITS/sec provided by ISP?

Tony · March 15, 2016, 11:01pm

Thanks for the data and feedback. 2.22 is a little under 18Mbps, so pretty close to your theoretical max.

Lasse · March 16, 2016, 7:45am

Tests of concurrent request and download speed would be appreciated!

Maybe you should even consider implementing a benchmark in the odrive app?

Tony · March 17, 2016, 1:39am

Been sitting on this for a day or two, waiting to get a chance to get to the OS X counterpart. That hasn’t happened yet, so I’ll post this now. This can be used for single or multi-file upload on Windows:

powershell -command "& {$sizeinmb = 10;$numfiles = 10;$randstr = ([char[]]([char]'a'..[char]'z') + 0..9 | sort {get-random})[0..5] -join '';$tmpdir = \"$pwd\~tmpuploadtest_$randstr\";$tmpdirstart = \"$pwd\tmpuploadtest_$randstr\";mkdir -Force $tmpdir > $null;$filebasename = \"$($sizeinmb)MB.file\";$sizeinbytes = $sizeinmb * 1024 * 1024;for($i = 1; $i -le $numfiles; $i++){$thefile = \"$tmpdir\$filebasename\" + $i;fsutil file createnew $thefile $sizeinbytes > $null;};Start-Sleep -m 8000;ren $tmpdir $tmpdirstart;echo \"\",\"Waiting for odrive\";$locked = 0;while($locked -ne 1) { $files = Get-ChildItem $tmpdirstart; for ($i=0; $i -lt $files.Count; $i++) { try { [IO.File]::OpenWrite($files[$i].FullName).close(); } catch { $locked = 1; }} Start-Sleep -m 50};echo \"Starting upload ....\";$starttime = [int64](([datetime]::UtcNow)-(get-date \"1/1/1970\")).TotalSeconds;Start-Sleep -m 5000;while($locked -ne 0) { $locked = 0; $files = Get-ChildItem $tmpdirstart; for ($i=0; $i -lt $files.Count; $i++) { try { [IO.File]::OpenWrite($files[$i].FullName).close(); } catch { $locked = 1; }} Start-Sleep -m 500};$endtime = [int64](([datetime]::UtcNow)-(get-date \"1/1/1970\")).TotalSeconds;$totaltime = $endtime - $starttime; $mbpersec = $sizeinmb * $numfiles / $totaltime; $mins = $totaltime / 60;echo \"$mins minutes\";echo \"$($mbpersec) MB/sec\";rm -Force -Recurse $tmpdirstart;}"

Adjust $sizeinmb and $numfiles to your preference.

I did some limited testing and the sources that were performing poorly are performing much better under concurrent transfer loads. I expected large improvement, but some of the results are pretty dramatic (~10x!).

Lasse · March 17, 2016, 8:05am

This is great - thanks!

Just tried it out - first on build 5085, but then I saw you released build 5095 with a very important change: Concurent upload/download performance improvement.

Amazon Cloud Drive

20*10MB: 0.840336134453782 MB/sec (build 5085)
20*10MB: 0.966183574879227 MB/sec (build 5095)
200*1MB: 2.5 MB/sec (build 5095)

Dropbox

20*10MB: 0.743494423791822 MB/sec (build 5085)
20*10MB: 1.49253731343284 MB/sec (build 5095)
200*1MB: 2.8169014084507 MB/sec (build 5095)

Google Drive

20*10MB: 11.1111111111111 MB/sec (build 5085)
20*10MB: 11.7647058823529 MB/sec (build 5095)
200*1MB: 4.34782608695652 MB/sec (build 5095)

OneDrive

20*10MB: 0.593471810089021 MB/sec (build 5085)
20*10MB: 3.17460317460317 MB/sec (build 5095)
200*1MB: 1.8018018018018 MB/sec (build 5095)

So for me it still looks like Google Drive is the best choice!

Tony · March 30, 2016, 11:43pm

Finally got a little time this afternoon to whip up the OS X multi-file uploader. I’ve done very limited testing on this so let me know if you see any issues:

sizeinmb=10; numfiles=10; randstr=$(openssl rand -hex 3); tmpdir="$(pwd)/._tmpuploadtest_$randstr"; tmpdirstart="$(pwd)/tmpuploadtest_$randstr"; echo ""; mkdir "$tmpdir"; filebasename=$sizeinmb"MB.file"; sizeinbytes=$(($sizeinmb * 1024 * 1024));for (( i=1; i<=$numfiles; i++)) do thefile="$tmpdir/$filebasename$i"; < /dev/zero head -c $sizeinbytes > "$thefile"; done; echo "Waiting for odrive"; mv "$tmpdir" "$tmpdirstart"; while [[ -z $(lsof -nPl -c odrive | grep "$tmpdirstart") ]]; do sleep .05; done; starttime=$(date +%s); echo "Starting upload ...."; while [[ -n $(lsof -nPl -c odrive | grep "$tmpdirstart") ]]; do sleep .05; done; endtime=$(date +%s); totaltime=$(($endtime - $starttime)); mbpersec=$(awk "BEGIN {printf \"%.2f\",${sizeinmb}*${numfiles}/${totaltime}}"); mins=$(awk "BEGIN {printf \"%.2f\",${totaltime}/60}"); echo "$mins minutes"; echo "$mbpersec MB/sec"; rm -r "$tmpdirstart";

Adjust sizeinmb and numfiles to your liking.

Smudge · April 3, 2016, 12:57am

I’ve modified the script to make a few improvements, including fixing a bug. Instead of using /dev/zero it uses /dev/random. When using zeros and tested a second time, the files were cached because it was the same content.

Google: 20x 10MB @ 2.44MB/sec = 1.37 mins
Google: 20x 10MB @40.00MB/sec = 0.08 mins

Here are my results after the changes, including the speed in megabits instead of megabytes since most people know their internet speeds in megabits.

It will list the results as directory, count, size, speed, time (min:sec)

Google:  1x512MB @23.41Mb/sec = 02:55
Amazon: 20x 10MB @14.68Mb/sec = 01:49
Amazon: 20x 20MB @20.78Mb/sec = 02:34
Amazon:  1x512MB @19.79Mb/sec = 03:27

I have added the option to save this code as a script, then you can pass it arguments to change the count and size of the test files if you wish. If not, it will default to 20 files of 10MB.

Save the following script to odrivetest.sh. The first argument is the count and the second argument is the size. For example odrivetest.sh 15 30 would run it with 15 files of 30MB each.

#!/bin/bash
count=${1:-20}; size=${2:-10}; bytes=$(($size * 1000 * 1000));
echo -n "Preparing test ${count}x${size}MB files:";
dir="$(pwd)"; randstr=$(openssl rand -hex 3);
tmpdir="$dir/._tmpuploadtest_$randstr";
tmpdirstart="$dir/tmpuploadtest_$randstr";
mkdir "$tmpdir";
for (( i=1; i<=$count; i++));
do name=$(awk "BEGIN {printf \"test-%03d.file\",$i}");
echo -n "C"; < /dev/random head -c $bytes > "$tmpdir/$name"; done;
echo -n "M"; mv "$tmpdir" "$tmpdirstart";
echo -n "W";
while [[ -z $(lsof -nPl -c odriveapp | grep "$tmpdirstart") ]];
do sleep .05; done;
echo -n "^"; starttime=$(date +%s);
while [[ -n $(lsof -nPl -c odriveapp | grep "$tmpdirstart") ]];
do sleep .05; done; endtime=$(date +%s); echo "!";
secs=$(($endtime - $starttime));
speed=$(awk "BEGIN {printf \"%.2f\",${size}*${count}*8/${secs}}");
dir=${dir##*/}
printf "%s:%3sx%3sMB @%-5sMb/sec = %02d:%02d\n" $dir $count $size $speed $((($secs/60)%60)) $(($secs%60));
rm -r "$tmpdirstart";

Tony · April 4, 2016, 6:22pm

Thanks for the refinements!

The bug you reported is interesting. Since the files are located in a different path (random string appended to the directory), there should be no caching or insta-sync type of logic involved. I had originally used /dev/random but switched to /dev/zero because it was faster and shouldn’t have any duplicate/caching/insta-sync drawbacks (and didn’t in my testing). In fact, if this is happening I want to figure out how since our expectation is that it should not.

One of the drawbacks to this script is that it is looking at things from the “outside”, so there are cases where false reporting can occur. Since the script is using open file handles as a determination of processing, if one of the files in the set fails (and moves to waiting), the script can “complete” without actually completing. This will give similar results to what you saw in that first set of tests above.

brenton · March 7, 2019, 8:06pm

Hey @Tony , I’m wondering if this speed test should still work with the current version of odrive? Or if there is another way to do this? I’m on MacOS Mojave and when I run the test it sticks on “Waiting for odrive” forever.

Tony · March 8, 2019, 5:58am

Hi @brenton,
I hadn’t looked at this in a very long time, but it does still work. The name of the application on MacOS changed from odriveapp to odrive, which is why you were seeing it hang. I changed it and edited the first post so that the information is all in one place at the top.

This was originally done before the CLI was available. Now that is it, I would like to revisit this and update, when/if I can get a chance to do so. For now this should be okay for basic upload comparison for your storage.