Automatically mute users who attempt to post a slur, fixes #8062 (#8177)

* Initial psuedo-code for checking for slurs in messages

* Initial working prototype for blocking posting of slurs. Moved check from group.js to the chat api. Still needs: to permanently revoke chat privileges, to notify the moderators, a better method for checking for the blacklisted words, and a way to get the real list of words to check.

* Permanently revoke chat privileges when attempting to post a slur.

* Removed console logs

* Fixing rebase

* Do not moderate private groups

* Moved slur check to a generic check for banned words function

* Moved list of slurs to a separate file, fixed misplacement of return in ContainsBannedWords() function

* Slurs are blocked in both public and private groups

* Added code to send a slack message for slurs

* Fixed formatting issues

* Incorporated tectContainsBannedWords() function from PR 8197, added an argument to specify the list of banned words to check

* Added initial tests for blocking slurs and revoking chat priviliges

* Uncommented line to save revoked privileges

* Check that privileges are revoked in private groups

* Moved code to email/slack mods to chat api file

* Switched to BadRequest instead of NotFound error

* Restore chat privileges after test

* Using official placeholder slur

* Fixed line to export sendSubscriptionNotification function for slack

* Replaced muteUser function in user methods with a single line in the chat controller file

* Reset chatRevoked flag to false in a single line

* Switched method of setting chatRevoked flag so that it is updated locally and in the database

* First attempt at the muteUser function: revokes user's chat privileges and notifies moderators

* Manual merge for cherry-pick

* Initial working prototype for blocking posting of slurs. Moved check from group.js to the chat api. Still needs: to permanently revoke chat privileges, to notify the moderators, a better method for checking for the blacklisted words, and a way to get the real list of words to check.

* Permanently revoke chat privileges when attempting to post a slur.

* Removed console logs

* Created report to be sent to moderators via email

* Do not moderate private groups

* Moved slur check to a generic check for banned words function

* Moved list of slurs to a separate file, fixed misplacement of return in ContainsBannedWords() function

* Slurs are blocked in both public and private groups

* Added code to send a slack message for slurs

* Fixed formatting issues

* Incorporated tectContainsBannedWords() function from PR 8197, added an argument to specify the list of banned words to check

* Added initial tests for blocking slurs and revoking chat priviliges

* Uncommented line to save revoked privileges

* Check that privileges are revoked in private groups

* Moved code to email/slack mods to chat api file

* Switched to BadRequest instead of NotFound error

* Restore chat privileges after test

* Using official placeholder slur

* Fixed line to export sendSubscriptionNotification function for slack

* Replaced muteUser function in user methods with a single line in the chat controller file

* Reset chatRevoked flag to false in a single line

* Switched method of setting chatRevoked flag so that it is updated locally and in the database

* Removed some code that got re-added after rebase

* Tests for automatic slur muting pass but are incomplete (do not check that chatRevoked flag is true)

* Moved list of banned slurs to server side

* Added warning to bannedSlurs file

* Test chat privileges revoked when posting slur in public chat

* Fix issues left over after rebase (I hope)

* Added code to test for revoked chat privileges after posting a slur in a private group

* Moved banned slur message into locales message

* Added new code to check for banned slurs (parallels banned words code)

* Fixed AUTHOR_MOTAL_URL in sendTxn for slur blocking

* Added tests that email sent on attempted slur in chat post

* Created context for slur-related-tests, fixed sandboxing of email. Successfully tests that email.sendTxn is called, but the email content test fails

* commented out slack (for now) and cleaned up tests of sending email

* Successfully tests that slur-report-to-mods email is sent

* Slack message is sent, and testing works, but some user variables seem to only work when found in chat.js and passed to slack

* Made some fixes for lint, but not sure what to do about the camel case requirement fail, since that's how they're defined in other slack calls

* Slack tests pass, skipped camelcase check around those code blocks

* Fixed InternalServerError caused by slack messaging

* Updated chat privileges revoked error

* fix(locale): typo correction
This commit is contained in:
Alyssa Batula
2017-07-19 17:06:15 -04:00
committed by Sabe Jones
parent 89ee8b1648
commit c350665076
6 changed files with 296 additions and 22 deletions

View File

@@ -10,11 +10,17 @@ import {
TAVERN_ID,
} from '../../../../../website/server/models/group';
import { v4 as generateUUID } from 'uuid';
import * as email from '../../../../../website/server/libs/email';
import { IncomingWebhook } from '@slack/client';
import nconf from 'nconf';
const BASE_URL = nconf.get('BASE_URL');
describe('POST /chat', () => {
let user, groupWithChat, member, additionalMember;
let testMessage = 'Test Message';
let testBannedWordMessage = 'TEST_PLACEHOLDER_SWEAR_WORD_HERE';
let testSlurMessage = 'message with TEST_PLACEHOLDER_SLUR_WORD_HERE';
before(async () => {
let { group, groupLeader, members } = await createAndPopulateGroup({
@@ -166,6 +172,114 @@ describe('POST /chat', () => {
});
});
context('banned slur', () => {
beforeEach(() => {
sandbox.spy(email, 'sendTxn');
sandbox.stub(IncomingWebhook.prototype, 'send');
});
afterEach(() => {
sandbox.restore();
});
it('errors and revokes privileges when chat message contains a banned slur', async () => {
await expect(user.post(`/groups/${groupWithChat._id}/chat`, { message: testSlurMessage})).to.eventually.be.rejected.and.eql({
code: 400,
error: 'BadRequest',
message: t('bannedSlurUsed'),
});
// Email sent to mods
await sleep(0.5);
expect(email.sendTxn).to.be.calledOnce;
expect(email.sendTxn.args[0][1]).to.be.eql('slur-report-to-mods');
// Slack message to mods
expect(IncomingWebhook.prototype.send).to.be.calledOnce;
/* eslint-disable camelcase */
expect(IncomingWebhook.prototype.send).to.be.calledWith({
text: `${user.profile.name} (${user.id}) tried to post a slur`,
attachments: [{
fallback: 'Slur Message',
color: 'danger',
author_name: `${user.profile.name} - ${user.auth.local.email} - ${user._id}`,
title: 'Slur in Test Guild',
title_link: `${BASE_URL}/#/options/groups/guilds/${groupWithChat.id}`,
text: testSlurMessage,
// footer: sandbox.match(/<.*?groupId=group-id&chatId=chat-id\|Flag this message>/),
mrkdwn_in: [
'text',
],
}],
});
/* eslint-enable camelcase */
// Chat privileges are revoked
await expect(user.post(`/groups/${groupWithChat._id}/chat`, { message: testMessage})).to.eventually.be.rejected.and.eql({
code: 401,
error: 'NotAuthorized',
message: t('chatPrivilegesRevoked'),
});
// Restore chat privileges to continue testing
user.flags.chatRevoked = false;
await user.update({'flags.chatRevoked': false});
});
it('does not allow slurs in private groups', async () => {
let { group, members } = await createAndPopulateGroup({
groupDetails: {
name: 'Party',
type: 'party',
privacy: 'private',
},
members: 1,
});
await expect(members[0].post(`/groups/${group._id}/chat`, { message: testSlurMessage})).to.eventually.be.rejected.and.eql({
code: 400,
error: 'BadRequest',
message: t('bannedSlurUsed'),
});
// Email sent to mods
await sleep(0.5);
expect(email.sendTxn).to.be.calledThrice;
expect(email.sendTxn.args[2][1]).to.be.eql('slur-report-to-mods');
// Slack message to mods
expect(IncomingWebhook.prototype.send).to.be.calledOnce;
/* eslint-disable camelcase */
expect(IncomingWebhook.prototype.send).to.be.calledWith({
text: `${members[0].profile.name} (${members[0].id}) tried to post a slur`,
attachments: [{
fallback: 'Slur Message',
color: 'danger',
author_name: `${members[0].profile.name} - ${members[0].auth.local.email} - ${members[0]._id}`,
title: 'Slur in Party - (private party)',
title_link: undefined,
text: testSlurMessage,
// footer: sandbox.match(/<.*?groupId=group-id&chatId=chat-id\|Flag this message>/),
mrkdwn_in: [
'text',
],
}],
});
/* eslint-enable camelcase */
// Chat privileges are revoked
await expect(members[0].post(`/groups/${groupWithChat._id}/chat`, { message: testMessage})).to.eventually.be.rejected.and.eql({
code: 401,
error: 'NotAuthorized',
message: t('chatPrivilegesRevoked'),
});
// Restore chat privileges to continue testing
members[0].flags.chatRevoked = false;
await members[0].update({'flags.chatRevoked': false});
});
});
it('does not error when sending a message to a private guild with a user with revoked chat', async () => {
let { group, members } = await createAndPopulateGroup({
groupDetails: {