I suggest you ...

EE: Data retention and prune/reset/clear channel content options

For compliance, add the ability to delete specific messaging entries from the user interface, including:

- Clear a specific channel of all its content
- Delete all posts from a specific user
- Delete all posts from a specific user in a specific channel
- Delete all posts from/before/after a specific date
- Configure a data retention time

219 votes
Vote
Sign in
(thinking…)
Sign in with: facebook google
Signed in as (Sign out)
You have left! (?) (thinking…)
Anonymous shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

30 comments

Sign in
(thinking…)
Sign in with: facebook google
Signed in as (Sign out)
Submitting...
  • Tobias commented  ·   ·  Flag as inappropriate

    Why is this exclusively a feature in the largest license? In theory, Mattermost should always meet the legal requirements, especially in the EU, without having to pay extra for it.

  • Marco Descher commented  ·   ·  Flag as inappropriate

    This is really ridiculous. I started to use Mattermost through its shipping with Gitlab, and collect logs through it. I consider going on E10 for the push notifications, but requiring E20 for simply clearing logs is really not matching what I think of mattermost overall - and I got a mug from you for developing an extension ...

  • Ingo Fischer commented  ·   ·  Flag as inappropriate

    E20 only? Really? This is a very basic feature for business users and companies (data protection stuff and legal requirements from the country), so it should be available in E10!!

  • someone commented  ·   ·  Flag as inappropriate

    As we didnt expect that after nearly 2 years after the initial request anything would come out of this, we just finished writing the database+filesystem cleanup by ourself ...

    Our tests indicate, that these 2 scripts do their jobs on MM 3.8 perfectly.
    But please beware that you may lose support, if you run this.
    Make DB-Backups and test by yourself, before throwing this at live-data.

    (I so hope, this form has markup/formatting support)

    DB-Cleaner-Python script:
    ```
    #!/usr/bin/env python3

    import psycopg2
    import psycopg2.extras

    dbconnstring = "host=... dbname=... user=... password=..."

    dbconn = psycopg2.connect(dbconnstring)

    # channels
    cur = dbconn.cursor(cursor_factory=psycopg2.extras.DictCursor)
    cur.execute("DELETE FROM posts where channelid in (SELECT id FROM channels where deleteat <> 0)")
    print("Deleted "+str(cur.rowcount)+" 'deleted' channel post(s).")
    cur.execute("DELETE FROM channelmembers WHERE channelid in (SELECT id FROM channels WHERE deleteat <> 0)")
    print("Deleted "+str(cur.rowcount)+" 'deleted' channel member(s).")
    cur.execute("DELETE FROM channels WHERE deleteat <> 0")
    print("Deleted "+str(cur.rowcount)+" 'deleted' channel(s).")
    dbconn.commit()
    print("* committed")

    # posts
    cur = dbconn.cursor(cursor_factory=psycopg2.extras.DictCursor)
    cur.execute("DELETE FROM posts where deleteat <> 0")
    print("Deleted "+str(cur.rowcount)+" 'deleted' post(s).")
    dbconn.commit()
    print("* committed")

    # orphaned files
    cur = dbconn.cursor(cursor_factory=psycopg2.extras.DictCursor)
    cur.execute(
    "DELETE FROM fileinfo WHERE id not in "+
    "(SELECT unnest(string_to_array(replace(replace(replace(fileids, '\"', ''), ']', ''), '[', ''), ',')) FROM posts WHERE fileids <> '[]')"
    )
    print("Deleted "+str(cur.rowcount)+" orphaned file reference(s).")
    dbconn.commit()
    print("* committed")

    # slash commands
    cur = dbconn.cursor(cursor_factory=psycopg2.extras.DictCursor)
    cur.execute("DELETE FROM commands where deleteat <> 0")
    print("Deleted "+str(cur.rowcount)+" 'deleted' slash command(s).")
    dbconn.commit()
    print("* committed")
    ```

    Filesystem-Cleaner:
    ```
    #!/usr/bin/env python3

    import psycopg2
    import psycopg2.extras
    import os
    import pprint

    dbconnstring = "host=... dbname=... user=... password=..."
    media_root = "/srv/mattermost/mattermost/data/"

    # get all db referenced files.
    dbconn = psycopg2.connect(dbconnstring)
    cur = dbconn.cursor(cursor_factory=psycopg2.extras.DictCursor)
    cur.execute("SELECT unnest(ARRAY[path, thumbnailpath, previewpath]) FROM fileinfo")

    db_files = cur.fetchall()
    db_files = set([val for sublist in db_files for val in sublist])
    #pprint.pprint(db_files)

    # get paths of physical files.
    fs_files = set()
    for relative_root, dirs, files in os.walk(media_root):
    for file_ in files:
    # Compute the relative file path to the media directory, so it can be compared to the values from the db
    relative_file = os.path.join(os.path.relpath(relative_root, media_root), file_)
    fs_files.add(relative_file)
    #pprint.pprint(fs_files)

    # diff files and (show info to) delete them
    diff_files = fs_files - db_files
    del_files = [f for f in diff_files if not f.startswith('users/')]

    #pprint.pprint(del_files)

    if del_files:
    for file_ in del_files:
    print('rm "'+os.path.join(media_root, file_)+'"')
    # os.remove(os.path.join(media_root, file_))

    # delete all empty folders
    for relative_root, dirs, files in os.walk(media_root, topdown=False):
    for dir_ in dirs:
    if not os.listdir(os.path.join(relative_root, dir_)):
    print('rmdir "'+os.path.join(relative_root, dir_)+'"')
    # os.rmdir(os.path.join(relative_root, dir_))

    ```

  • someone commented  ·   ·  Flag as inappropriate

    Please split this feature into 2:
    - A way (API) to delete old posts (per channel/team/whatever). (we dont particularly care about this part)
    - A CLI-command to permanently delete old database + filesystem contents.

    This is a major blocker to mattermost deployment for us, as we are required to permanently/irrecoverably delete any messages and files deleted (by user or admin) within "reasonable time".

    Where "reasonable time" is defined as: the deleted file/message may no longer be included in daily backups.

    Basically we run this for all our production software: "cleanup databases && cleanup filesystems && do backups" every night.

    We are considering to hack this by ourself, as it currently seems to be totally safe to delete everything that has deletedAT > 0 from database + purge the audit table.
    As it seems to be important for more than just us, we also want to +1 this as a core-feature.

  • Anonymous commented  ·   ·  Flag as inappropriate

    We also need this feature, as we share some sensitive information via mattermost too.

    It also would be great to specify which channels you want to purge (we have a separate KB channel and I don't want to purge this channel).

  • Anonymous commented  ·   ·  Flag as inappropriate

    This is the number 9 most top voted request (and related to it is #3 "Off the record chats"). And this request has been open for 19 months already, which probably means a "no," is it? Does not look like the company listens to their users... or even checks this suggestion board.

    Too busy getting mentioned in *Wired* to implement vital features? Bragging in the blog about “a great choice wherever specialist teams need highly secure messaging,” of using Mattermost in the area obsessed with both data retention and shredding? Sorry, not buying that. TV show is a TV show, and real life is real life. All show and no go.

  • Jochen Oonincx commented  ·   ·  Flag as inappropriate

    If I cannot use a function such as this I am not allowed to set up Mattermost at all. Mainly because we have to deal with organisations in the following sectors:

    - Government (requirement)
    - Financial (requirement)
    - Network & Security (Wanted, but not strictly required)
    - Power Plants (requirement)
    - Aviation (requirement)

    I love the ease of use of Mattermost, however I cannot deploy it unless this feature is included. The only other workaround is to keep resetting the entire environment every interval and rebuild everything and re-invite users. Which is just a real annoyance.

  • Gwynn commented  ·   ·  Flag as inappropriate

    It might be not so critical if all posts would be saved encrypted in database. But all posts in plain texts, and it is critical security and privacy hole.

  • Anonymous commented  ·   ·  Flag as inappropriate

    One more vote for this. This is absolutely required in corporate use. Another option would be to make sure that a new user never sees older conversation history, only from the moment his account was created.

    Think about a very typical scenario: People are talking about a customer / associate / whatever in a not so nice way on a company internal channel. Or for example a failed business case involving that customer. Two months later that associate happens to be hired in the company, given a Mattermost access and he browses old chat history. Not definitely a good thing.

    New users just cannot be allowed to access old discussions written before they joined, period. People should be able to assume that only the people at the moment in the chat channel are able to ever read the messages.

  • Anonymous commented  ·   ·  Flag as inappropriate

    And now there are 117 votes. So, how significant is it now?? The users here wrote plenty of good(!) reasons for this feature.

  • dissapointed user commented  ·   ·  Flag as inappropriate

    The reply from Mattermost.org sounds plainly ignorant!! Not every person who lacks the option of deleting message history have time and energy to google for appropriate forum to post it's concerns! We are company of 70 people and started using Mattermost few month ago and our first reaction to lack of history erase option was complaining to our administrator who simply replied that it is not possible. So not all concerned users end up in this forum!

  • Anonymous commented  ·   ·  Flag as inappropriate

    I work for an NGO that does almost all it's internal communication through a self-hosted Mattermost instance. We fear that a breach of our server can have damaging effects, similar to the effects of the Podesta email leak. We try to minimize the consequences of a breach as much as we can, and minimizing the amount of information we have to protect helps a lot. This means that we can greatly benefit from the possibility to (automatically) delete messages after a certain time.

  • Anonymous commented  ·   ·  Flag as inappropriate

    This is sorely needed. Especially for cleaning up bot messages...I have to segregate human and bot channels because the human conversations would get lost after a few days of jenkins results.

  • Dan Farmer commented  ·   ·  Flag as inappropriate

    As someone who works with sensitive data that has legislative requirements foisted on me (not like I want them, but I have no choice!), not having this is a real burden.

    The cron hack below doesn't help unless it's officially supported feature, but if someone cut-n-pastes the wrong data into MM that I need to kill off, I'm stuck.) This is preventing me from using this for full production services.

← Previous 1

Feedback and Knowledge Base