Better AWS S3 Integration
AWS S3 is a great option and I've gotten it working using an IAM role w policies to access that specific S3 bucket. What I'm suggesting is that instead of S3 being used as just a file "dump", there could be an optional setting for "Serve content directly from S3" (as well as perhaps another option available for "serve directly using cloudfront" - which would have a notice on the extra cost).
The idea is that instead of serving assets from the Mattermost instance itself, the objects in S3 could have presigned URL's set on them (with another setting for default expiry) ensuring that only the users who have access to the specific object (image) can access it.
I was going to look over the source and see if I could implement this in a branch and submit a pull request, but if anyone else is interested in working on this do let me know!
Jack Britchford
shared this idea
-
Jack Britchford
commented
I haven't used Go extensively, but I've integrated S3 presigned URL's and using crypto to generate the presigned URL's locally on the server with set expiries and the signature used by AWS to validate that the expiry/request data/etc is all valid. I'm sure most AWS SDK's will have a function for generating presigned URL's but the difficulty may be in integrating it with rendering the images (using the AWS s3 endpoints on the outputted page as apposed to the /api/ ones. I also noticed there's thumbnails and different scaled images, so there could also be an option "store thumbnails" to also process thumbnails locally and put them on the file store in S3. If I'm not mistaken, currently it requests all the images from S3 (on the host machine, host machine being the box/boxes running Mattermost) and they serve them back. Bandwidth from AWS EC2 to S3 is nothing but- when you're then serving that back from multiple EC2 instances behind a load balancer, that is quite wasteful when S3 can handle all of them. I'm also not sure if any headers are being sent correctly with the images to ensure they're cached correctly (it would also be nice, if there aren't any currently) to control the cache of users images (I imagine the static images are cached indefinitely but, would be nice to be able to set content max-age on the images/thumbnails that are in chat (if that isn't already possible)