Hi there,
First time poster, long-time user of Odrive.
I recently decided to try using the AWS S3 service with Odrive, as I am a big of S3. I learned a couple of things that I wanted to share. (This MAY be Mac OS/X specific, I warned ya!)
DO: Make sure you have a security policy set up in IAM. I would recommend creating a brand new user account in IAM so that it has the minimum privileges necessary (good system admin practice). Once the account is created, attach the following IAM policy to it:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::yourusername-odrive"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::yourusername-odrive/*"
]
}
]
}
That should allow that account to access Odrive.
DON’T: try to sync tens of thousands of files to Amazon S3.
I learned this one the hard way. I do some software development, and have a few Node.js projects that, when all of their modules (and sub modules) are added up, came out to about 40,000 files across 900 Megs of data. It turns out that S3 isn’t the best when it comes to uploading many small files. I ended up having to let Odrive run overnight to sync up all those files, then the following evening let it run another night to unsync those files.
SUGGESTION: Use something like Duplicity to tar up your entire development directory into just a few large files, and write a script that does that and puts those files into the Odrive directory that gets synced to S3. You’ll thank me later.
To get to this point, it took me a few evenings, so I thought I would share what I learned, and hopefully it will save anyone else wanting to use S3 with Odrive some frustration.
If anyone has questions about S3, feel free to hit me up in the comments, I’m a huge fan of AWS and its services.
– Doug