@anonymous: Use the somenet scripts they seem to do the trick.

@anonymous: Use/enhance the somenet-scripts, they seem to do the trick.

I desperately need this!!!!!!

Just a follow-up: The repo of the cleaner changed to:

https://git.somenet.org/pub/jan/mattermost-privileged.git

Its running every night on at least 3 instances: 2* TE and one EE10.

Currently cleaning/doing this (tonight's run):

Mattermost DB cleanup script. Tested on 5.19

Enforcing enhanced system policy ...
* [0.02232] 0 deleted 31 days+ not used guest-account(s).
* [1.17975] 0 deleted 31 days+ not verified account(s).
* [1.22323] 0 deleted 12 months+ inactive non-public one-user-channel(s).
* [1.22547] 0 deleted breaking channel(s).

Enforcing system policy ...
* [1.55687] 220 deleted all system spam message(s).
* [1.75893] 5 deleted old BOT-AUTODELETE-FAST message(s).
* [1.95629] 0 deleted old BOT-AUTODELETE-SLOW message(s).
* [2.20882] 1 deleted AUTODELETE-PROP-DAY message(s).
* [2.44673] 0 deleted AUTODELETE-PROP-WEEK message(s).

Deleting old unused/history data ...
* [2.45021] 0 deleted unused (implicitly created) DM/Group channel(s).
* [2.61361] 3994 deleted old audit entries.
* [2.61563] 1 deleted old job entries.
* [2.63317] 1279 deleted old linkmetadata entries.
* [2.67874] 45 deleted old channelmemberhistory entries.

Deleting 'deleted'/'archived' entries ...
* [02.8313] 51 deleted 'deleted' post(s).
* [2.83459] 0 deleted 'deleted' channel(s).
* [2.83618] 0 deleted 'deleted' emoji(s).
* [02.8384] 0 deleted 'deleted' teammember(s).
* [2.83949] 0 deleted 'deleted' slash command(s).
* [02.8406] 0 deleted 'deleted' outgoing webhook(s).
* [2.84178] 0 deleted 'deleted' incoming webhook(s).

Deleting orphaned entries ...
* [2.87812] 0 deleted orphaned private channel(s).
* [3.45347] 0 deleted orphaned post(s).
* [4.28844] 3 deleted orphaned post-comment(s).
* [4.87566] 0 deleted orphaned reaction(s).
* [4.92934] 0 deleted orphaned channelmember(s).
* [4.97739] 0 deleted orphaned channelmemberhistory entr(y/ies).
* [4.98494] 0 deleted orphaned teammember(s).
* [4.99011] 0 deleted orphaned user status entr(y/ies).
* [4.99968] 0 deleted orphaned session(s).
* [5.35271] 4 deleted orphaned file reference(s).

* [5.38729] 0 deleted orphaned direct_channel_show preference(s).
* [05.4077] 0 deleted orphaned channel_approximate_view_time preference(s).
* [5.41207] 0 deleted orphaned favorite_channel preference(s).
* [5.94737] 0 deleted orphaned flagged_post preference(s).

Fixing stuff ...
* [6.02034] 521 inserted public channel(s) into 'materialized view'.
* [6.02641] 0 deleted public channel group_channel_show preference(s). THIS IS A BUGFIX. SHOULD BE 0!
* [6.67347] 11 updated channel totalmsgcount.
* [7.31338] 94 updated non-empty channel lastpostat.
* [7.41214] 1201 updated channelmember msgcount/lastviewedat. (good enough)

*** committed ***

* db-size: 557 MB

Data / Database / File system pruning seems like extremely basic functionality, on any level. This should hardly be "Enterprise-only". And to ensure data integrity and consistency, it seems like the proper place is an internal Mattermost tool (as well as one or more API endpoint/s)

one of 2 issues preventing us to move from slack to mattermost :(
would be cool to have config if files are really deleted or soft deleted
and then allow team owner to purge soft deleted files trough console
but also a menu on a file itself "soft delete" - "hard delete" so you don't have to delete the post, you can only delete a file

(second issue for us is rather trivial - having custom sounds in client, but that's important for our users so..)

This isn't just a suggestion. To me, this is a major "fail" of Mattermost development to not allow files to be deleted permanently, as part of the message deletion process or to allow an attachment to be deleted within a message without having to delete the message itself. The issue of ever-increasing HDD space is one that needs to be addressed, not as a recommendation, but as a bug. A soft delete only is not a feature but a bug.

I cannot emphasize this enough. In fact, I've tried almost all of the other open source alternatives out there and adding this fix to delete files would make this the perfect solution, IMHO.

I found that this is partly because:
A, file records are not even soft deleted in the database, even if the post they were originally attached to IS deleted
B, files are physically kept in the data directory

Other than that the used storage will grow indefinitely, it doesn't even seem to make sense. Deleting the post will make the attachment disappear from the UI and unavailable/unusable for all users.

Notice below, how the post has "deleteat" timestamp set in public.posts table, but it is 0 on the attachment in public.fileinfo. public.fileinfo.deleteat should either be set by a database trigger to the same timestamp that of the original post's delete time, or set on both records from code as well as the file physically deleted. We already have the file location stored in public.fileinfo, so it doesn't seem very complicated.

mattermost=> select p.id post_id, p.deleteat post_delete_time, f.name file_name, f.size file_size, f.deleteat file_delete_time from public.posts p join public.fileinfo f on f.postid = p.id where p.deleteat > 0 and coalesce(f.deleteat,0) = 0 order by p.deleteat;
-[ RECORD 1 ]----+----------------------------
post_id | gfhts4kfybfh9ecn68mdksjjsc
post_delete_time | 1500029064303
file_name | test file
file_size | 15
file_delete_time | 0
-[ RECORD 2 ]----+----------------------------
post_id | nxkeu31sdpfepy1t3nmuy758by
post_delete_time | 1500086864483
file_name | 15000846037071752086964.jpg
file_size | 6204621
file_delete_time | 0

I would like to see a configurable expiration of data that is older than x days or months, that applies to Database and Files.