Crates.io meeting agenda 2019-10-31 20:00 UTC (Discord, 30 min)

[pietroalbini]

Please edit to add agenda items, or comment if unable to edit.

• Setup a S3 lifecycle policy for the database dumps. (pietro)
• Use deploy keys to commit to the index instead of access tokens. (pietro)
• Discuss updating Ember, and ember-data in particular (@jtgeibel)
• Legal takedown requests and Upload all content to s3 as public and immutable crates.io#1871 (@smarnach)
• Change the meeting time not to be during EU evening? (pietro)

[carols10cents]

* Setup a S3 lifecycle policy for the database dumps. (pietro)

@pietroalbini I think we're overwriting the same db-dump.tar.gz file every time we take a new snapshot? Does s3 keep previous versions or something like that?

[pietroalbini]

@carols10cents yeah indeed, the whole crates-io bucket is configured to keep previous versions of each file indefinitely. This is something we definitely want for most of the repository (to prevent issues with accidental deletions of .crate files), but we should configure auto-deletion after a while for that file.

[smarnach]

There is another, orthogonal problem with the database dumps. We don't set any cache-related headers for them, so CloudFront will cache them for 24 hours by default. If the first download via a particular CloudFront edge cache happens shortly before the new dump becomes available, all requests via this edge server will see the old version for another 24 hours.

Do we need to sovle this in the uploader code (by adding an Expires header), or is it possible to somehow configure this in the bucket settings as well?

[pietroalbini]

Also added "Use deploy keys to commit to the index instead of access tokens".

There is another, orthogonal problem with the database dumps. We don't set any cache-related headers for them, so CloudFront will cache them for 24 hours by default. If the first download via a particular CloudFront edge cache happens shortly before the new dump becomes available, all requests via this edge server will see the old version for another 24 hours.

Do we need to sovle this in the uploader code (by adding an Expires header), or is it possible to somehow configure this in the bucket settings as well?

I'll look at what I can do on the CloudFront side.