Help:Abuse Filter rules
Abuse Filter rules
This page provides information about the Abuse Filter rules its use.
|
This help page is TestWiki how-to guide. It details processes or procedures of some aspect(s) of TestWiki's norms and practices. It is not one of TestWikis's policies or guidelines. |
The rules are a custom language. They are formatted similar to conditionals in a C/Java/Perl-like language.
Strings[edit]
You can specify a literal by placing it in single or double quotes (for strings), or by typing it in as-is (for numbers, both floating-point and integer).
You can get linebreaks with \n
, tab characters with \t
, and you can also escape the quote character with a backslash.
Use the +
(plus) symbol to concatenate two literal strings or the values of two vars with a string value.
- Examples
"This is a string"
'This is also a string'
'This string shouldn\'t fail'
"This string\nHas a linebreak"
1234
1.234
-123
User-defined variables[edit]
You can define custom variables for ease of understanding with the assign symbol :=
in a line (closed by ;
) within a condition.
Such variables may use letters, underscores, and numbers (apart from the first character) and are case-insensitive.
Example (from w:Special:AbuseFilter/79):
(
line1:="(\{\{(r|R)eflist|\{\{(r|R)efs|<references\s?/>|</references\s?>)";
rcount(line1, removed_lines)
) > (
rcount(line1, added_lines)
)
Arrays[edit]
AbuseFilter has support for non-associative arrays, which can be used like in the following examples.
<translate> Caution:</translate> | Expressions like page_namespace in [14, 15] may not work as expected. This one will evaluate to true also if page_namespace is 1, 4, or 5. For more information and possible workarounds, please see T181024. |
my_array := [ 5, 6, 7, 10 ];
my_array[0] == 5
length(my_array) == 4
int( my_array ) === 4 // Same as length
float( my_array ) === 4.0 // Counts the elements
string(my_array) == "5\n6\n7\n10\n" // Note: the last linebreak could be removed in the future
5 in my_array == true
'5' in my_array == true
'5\n6' in my_array == true // Note: this is due to how arrays are casted to string, i.e. by imploding them with linebreaks
1 in my_array == true // Note: this happens because 'in' casts arguments to strings, so the 1 is caught in '10' and returns true.
my_array[] := 57; // This appends an element at the end of the array
my_array === [ 5, 6, 7, 10, 57 ]
my_array[2] := 42; // And this is for changing an element in the array
my_array === [ 5, 6, 42, 10, 57 ]
Comments[edit]
You can specify comments using the following syntax:
/* This is a comment */
Arithmetic[edit]
You can use basic arithmetic symbols to do arithmetic on variables and literals with the following syntax:
-
– Subtract the right-hand operand from the left-hand operand.+
– Add the right-hand operand to the left-hand operand.*
– Multiply the left-hand operand by the right-hand operand./
– Divide the left-hand operand by the right-hand operand.**
– Raise the left-hand operand to the exponential power specified by the right-hand operand.%
– Return the remainder given when the left-hand operand is divided by the right-hand operand.
The type of the returned result is the same that would be returned by PHP, for which a lot of documentation may be found online. More exhaustive examples may be found in this AF parser test.
Example | Result |
---|---|
1 + 1 |
2 |
2 * 2 |
4 |
1 / 2 |
0.5 |
9 ** 2 |
81 |
6 % 5 |
1 |
Boolean operations[edit]
You can match if and only if all of a number of conditions are true, one of a number of conditions are true, or one and only one of all conditions are true.
x | y
— OR – returns true if one or more of the conditions is true.x & y
— AND – returns true if both of the conditions are true.x ^ y
— XOR – returns true if one, and only one of the two conditions is true.!x
— NOT – returns true if the condition is not true.
Examples
Code | Result |
---|---|
1 | 1
|
true |
1 | 0
|
true |
0 | 0
|
false |
1 & 1
|
true |
1 & 0
|
false |
0 & 0
|
false |
1 ^ 1
|
false |
1 ^ 0
|
true |
0 ^ 0
|
false |
!1
|
false |
!0
|
true |
Simple comparisons[edit]
You can compare variables with other variables and literals with the following syntax:
<
,>
– Return true if the left-hand operand is less than/greater than the right-hand operand respectively. Watch out: operands are casted to strings and, like it happens in PHP,null < any number === true
andnull > any number === false
.<=
,>=
– Return true if the left-hand operand is less than or equal to/greater than or equal to the right-hand operand respectively. Watch out: operands are casted to strings and, like it happens in PHP,null <= any number === true
andnull >= any number === false
.==
(or=
),!=
– Return true if the left-hand operand is equal to/not equal to the right-hand operand respectively.===
,!==
– Return true if the left-hand operand is equal to/not equal to the right-hand operand AND the left-hand operand is the same/not the same data type to the right-hand operand respectively.
Example | Result |
---|---|
1 == 2 |
false |
1 <= 2 |
true |
1 >= 2 |
false |
1 != 2 |
true |
1 < 2 |
true |
1 > 2 |
false |
2 = 2 |
true |
'' == false |
true |
'' === false |
false |
1 == true |
true |
1 === true |
false |
['1','2','3'] == ['1','2','3'] |
true |
[1,2,3] === [1,2,3] |
true |
['1','2','3'] == [1,2,3] |
true |
['1','2','3'] === [1,2,3] |
false |
[1,1,''] == [true, true, false] |
true |
[] == false & [] == null |
true |
['1'] == '1' |
false[1] |
Built-in variables[edit]
The abuse filter passes various variables by name into the parser. These variables can be accessed by typing their name in, in a place where a literal would work. You can view the variables associated with each request in the abuse log.
Variables from AbuseFilter[edit]
Variables always available[edit]
Description | Name | Data type | Notes |
---|---|---|---|
Action | action |
string | One of the following: edit, move, createaccount, autocreateaccount, delete, upload[2], stashupload[3] |
Unix timestamp of change | timestamp |
string | int(timestamp) gives you a number with which you can calculate the date, time, day of week, etc. |
Database name of the wiki | wiki_name |
string | For instance, this is "enwiki" on the English Wikipedia, and "itwikiquote" on the Italian Wikiquote. |
Language code of the wiki | wiki_language |
string | For instance, this is "en" on the English Wikipedia, and "it" on the Italian Wikiquote. Multi-lingual wikis like Commons, Meta, and Wikidata will also report as "en". |
Edit count of the user | user_editcount |
integer/null | Null only for unregistered users. |
Name of the user account (IP in case the user is not registered) | user_name |
string | For "createaccount" and "autocreateaccount" actions, use
accountname if you want the name of the account being created. |
Type of the user account | user_type |
string | The type of the user, which will be one of ip , temp (if the user is using a temporary account ), named , external , or unknown .
|
Time email address was confirmed | user_emailconfirm |
string/null | In the format: YYYYMMDDHHMMSS. Null if the email wasn't confirmed. |
Age of the user account | user_age |
integer | In seconds. 0 for unregistered users. |
Whether the user is blocked | user_blocked |
boolean | True for blocked registered users. Also true for edits from blocked IP addresses, even if the editor is a registered user who is not blocked. False otherwise. This doesn't differentiate between partial and sitewide blocks.
|
Groups (including implicit) the user is in | user_groups |
array of strings | see Special:ListGroupRights |
Rights that the user has | user_rights |
array of strings | see Special:ListGroupRights |
Page ID | article_articleid |
integer | <translate> (deprecated)</translate> Use page_id instead.
|
Page ID (can be seen via "page information" link in sidebar) | page_id |
integer | This is 0 for new pages, but it is unreliable when inspecting past hits. If you need an exact result when inspecting past hits, use "page_age == 0" to identify new page creation. (note that it is slower, though.) This issue has been fixed in 9369d08, merged on September 11th 2023. |
Page namespace | article_namespace |
integer | <translate> (deprecated)</translate> Use page_namespace instead.
|
Page namespace | page_namespace |
integer | refers to namespace index . Check for namespace(s) using expressions like "page_namespace == 2" or "equals_to_any(page_namespace, 1, 3)" |
Page age (in seconds) | page_age |
integer | the number of seconds since the first edit (or 0 for new pages). This is reliable, but it tends to be slow; consider using page_id if you don't need much precision.
|
Page title (without namespace) | article_text |
string | <translate> (deprecated)</translate> Use page_title instead.
|
Page title (without namespace) | page_title |
string | |
Full page title | article_prefixedtext |
string | <translate> (deprecated)</translate> Use page_prefixedtitle instead.
|
Full page title | page_prefixedtitle |
string | |
Edit protection level of the page | article_restrictions_edit |
string | <translate> (deprecated)</translate> Use page_restrictions_edit instead.
|
Edit protection level of the page | page_restrictions_edit |
array of strings | |
Move protection level of the page | article_restrictions_move |
string | <translate> (deprecated)</translate> Use page_restrictions_move instead.
|
Move protection level of the page | page_restrictions_move |
array of strings | |
Upload protection of the file | article_restrictions_upload |
string | <translate> (deprecated)</translate> Use page_restrictions_upload instead.
|
Upload protection of the file | page_restrictions_upload |
array of strings | |
Create protection of the page | article_restrictions_create |
string | <translate> (deprecated)</translate> Use page_restrictions_create instead.
|
Create protection of the page | page_restrictions_create |
array of strings | |
Last ten users to contribute to the page | article_recent_contributors |
array of strings | <translate> (deprecated)</translate> Use page_recent_contributors instead.
|
Last ten users to contribute to the page | page_recent_contributors |
array of strings | This tends to be slow (see #Performance). Try to put conditions more likely evaluate to false before this one, to avoid unnecessarily running the query. This value is empty if the user is the only contributor to the page(?), and only scans the last 100 revisions |
First user to contribute to the page | article_first_contributor |
string | <translate> (deprecated)</translate> Use page_first_contributor instead.
|
First user to contribute to the page | page_first_contributor |
string | This tends to be slow (see #Performance).[4] Try to put conditions more likely evaluate to false before this one, to avoid unnecessarily running the query. |
Variables available for some actions[edit]
Description | Name | Data type | Notes |
---|---|---|---|
Edit summary/reason | summary |
string | Summaries automatically created by MediaWiki ("New section", "Blanked the page", etc.) are created after the filter checks the edit, so they will never actually catch, even if the debugger shows that they should. The variable contains whatever the user sees in the edit summary window, which may include MediaWiki preloaded section titles.[5] |
minor_edit |
Disabled, and set to false for all entries between 2016 and 2018.[6] | ||
Old page wikitext, before the edit | old_wikitext |
string | This variable can be very large. Consider using removed_lines if possible to improve performance.
|
New page wikitext, after the edit | new_wikitext |
string | This variable can be very large. Consider using added_lines if possible to improve performance.
|
Unified diff of changes made by edit | edit_diff |
string | |
Unified diff of changes made by edit, pre-save transformed | edit_diff_pst |
string | This tends to be slow (see #Performance). Checking both added_lines and removed_lines is probably more efficient.[7]
|
New page size | new_size |
integer | |
Old page size | old_size |
integer | |
Size change in edit | edit_delta |
integer | |
Lines added in edit, pre-save transformed | added_lines_pst |
array of strings | Use added_lines if possible, which is more efficient.
|
Lines added in edit | added_lines |
array of strings | includes all lines in the final diff that begin with + |
Lines removed in edit | removed_lines |
array of strings | |
All external links in the new text | all_links |
array of strings | This tends to be slow (see #Performance). |
Links in the page, before the edit | old_links |
array of strings | This tends to be slow (see #Performance). |
All external links added in the edit | added_links |
array of strings | This tends to be slow (see #Performance). Consider checking against added_lines first, then check added_links so that fewer edits are slowed down. This follows MediaWiki's rules for external links . Only unique links are added to the array. Changing a link will count as 1 added and 1 removed link.
|
All external links removed in the edit | removed_links |
array of strings | This tends to be slow (see #Performance). Consider checking against removed_lines first, then check removed_links so that fewer edits are slowed down. This follows MediaWiki's rules for external links . Only unique links are added to the array. Changing a link will count as 1 added and 1 removed link.
|
New page wikitext, pre-save transformed | new_pst |
string | This variable can be very large. |
Parsed HTML source of the new revision | new_html |
string | This variable can be very large. Consider using added_lines if possible to improve performance.
|
New page text, stripped of any markup | new_text |
string | This variable can be very large. Consider using added_lines if possible to improve performance.
|
old_html |
Disabled for performance reasons. | ||
old_text |
Disabled for performance reasons. | ||
⧼abusefilter-edit-builder-vars-page-last-edit-age⧽ | page_last_edit_age |
integer or null |
null when the page does not exist
|
SHA1 hash of file contents | file_sha1 |
string | [2] |
Size of the file in bytes | file_size |
integer | The file size in bytes[2] |
Width of the file in pixels | file_width |
integer | The width in pixels[2] |
Height of the file in pixels | file_height |
integer | The height in pixels[2] |
Bits per color channel of the file | file_bits_per_channel |
integer | The amount of bits per color channel[2] |
MIME type of the file | file_mime |
string | The file MIME type.[2] |
Media type of the file | file_mediatype |
string | The file media type.[8][2] |
Page ID of move destination page | moved_to_articleid |
integer | <translate> (deprecated)</translate> Use moved_to_id instead.
|
Page ID of move destination page | moved_to_id |
integer | |
Title of move destination page | moved_to_text |
string | <translate> (deprecated)</translate> Use moved_to_title instead.
|
Title of move destination page | moved_to_title |
string | |
Full title of move destination page | moved_to_prefixedtext |
string | <translate> (deprecated)</translate> Use moved_to_prefixedtitle instead.
|
Full title of move destination page | moved_to_prefixedtitle |
string | |
Namespace of move destination page | moved_to_namespace |
integer | |
Move destination page age (in seconds) | moved_to_age |
integer | |
⧼abusefilter-edit-builder-vars-movedto-last-edit-age⧽ | moved_to_last_edit_age
|
integer or null |
null when the target page does not exist
|
Edit protection level of move destination page | moved_to_restrictions_edit |
array of string | Same as page_restrictions_edit , but for the target of the move.
|
Move protection level of move destination page | moved_to_restrictions_move |
array of string | Same as page_restrictions_move , but for the target of the move.
|
Upload protection of move destination file | moved_to_restrictions_upload |
array of string | Same as page_restrictions_upload , but for the target of the move.
|
Create protection of move destination page | moved_to_restrictions_create |
array of string | Same as page_restrictions_create , but for the target of the move.
|
Last ten users to contribute to move destination page | moved_to_recent_contributors |
array of strings | Same as page_recent_contributors , but for the target of the move.
|
First user to contribute to move destination page | moved_to_first_contributor |
string | Same as page_first_contributor , but for the target of the move.
|
Namespace of move source page | moved_from_namespace |
integer | |
Title of move source page | moved_from_text |
string | <translate> (deprecated)</translate> Use moved_from_title instead.
|
Title of move source page | moved_from_title |
string | |
Full title of move source page | moved_from_prefixedtext |
string | <translate> (deprecated)</translate> Use moved_from_prefixedtitle instead.
|
Full title of move source page | moved_from_prefixedtitle |
string | |
Page ID of move source page | moved_from_articleid |
integer | <translate> (deprecated)</translate> Use moved_from_id instead.
|
Page ID of move source page | moved_from_id |
integer | |
Move source page age (in seconds) | moved_from_age |
integer | |
⧼abusefilter-edit-builder-vars-movedfrom-last-edit-age⧽ | moved_from_last_edit_age |
integer | |
Edit protection level of move source page | moved_from_restrictions_edit |
array of string | Same as page_restrictions_edit , but for the page being moved.
|
Move protection level of move source page | moved_from_restrictions_move |
array of string | Same as page_restrictions_move , but for the page being moved.
|
Upload protection of move source file | moved_from_restrictions_upload |
array of string | Same as page_restrictions_upload , but for the page being moved.
|
Create protection of move source page | moved_from_restrictions_create |
array of string | Same as page_restrictions_create , but for the page being moved.
|
Last ten users to contribute to move source page | moved_from_recent_contributors |
array of strings | Same as page_recent_contributors , but for the page being moved.
|
First user to contribute to move source page | moved_from_first_contributor |
string | Same as page_first_contributor , but for the page being moved.
|
Account name (on account creation) | accountname |
string | |
Content model of the old revision | old_content_model
|
string | See Help:ChangeContentModel for information about content model changes |
Content model of the new revision | new_content_model
|
string | See Help:ChangeContentModel for information about content model changes |
Protected variables[edit]
A variable can be considered protected. For instance, on wikis with temporary accounts enabled, IPs are considered PII and access to them must be restricted.
Protected variables and filters that use them are only accessible to maintainers with the abusefilter-access-protected-vars
right.
Using a protected variable flags the filter as protected as well.
The filter subsequently cannot be unprotected, even if it no longer actively uses a protected variable, as its historical logs will remain available.
The default protected variables are defined in AbuseFilterProtectedVariables
in extension.json
.
Description | Name | Data type | Notes |
---|---|---|---|
⧼abusefilter-edit-builder-vars-user-unnamed-ip⧽ | user_unnamed_ip
|
string | User IP for anonymous users/temporary accounts This returns
null for registered users. |
Variables from other extensions[edit]
Description | Name | Data type | Values | Added by |
---|---|---|---|---|
Global groups that the user is in | global_user_groups
|
array | Extension:CentralAuth | |
Global edit count of the user | global_user_editcount
|
integer | Extension:CentralAuth | |
Global groups that the user is in (on account creation) | global_account_groups
|
array | Available only when action is createaccount (then it is always empty) or autocreateaccount .
|
Extension:CentralAuth |
Global edit count of the user (on account creation) | global_account_editcount
|
integer | Available only when action is createaccount (then it is always zero) or autocreateaccount .
|
Extension:CentralAuth |
OAuth consumer used to perform this change | oauth_consumer
|
integer | Extension:OAuth | |
Page ID of Structured Discussions board | board_articleid
|
integer | <translate> (deprecated)</translate> Use board_id instead.
|
Extension:StructuredDiscussions |
Page ID of Structured Discussions board | board_id
|
integer | Extension:StructuredDiscussions | |
Namespace of Structured Discussions board | board_namespace
|
integer | refers to namespace index | Extension:StructuredDiscussions |
Title (without namespace) of Structured Discussions board | board_text
|
string | <translate> (deprecated)</translate> Use board_title instead.
|
Extension:StructuredDiscussions |
Title (without namespace) of Structured Discussions board | board_title
|
string | Extension:StructuredDiscussions | |
Full title of Structured Discussions board | board_prefixedtext
|
string | <translate> (deprecated)</translate> Use board_prefixedtitle instead.
|
Extension:StructuredDiscussions |
Full title of Structured Discussions board | board_prefixedtitle
|
string | Extension:StructuredDiscussions | |
Source text of translation unit | translate_source_text
|
string | Extension:Translate | |
Target language for translation | translate_target_language
|
string | This is the language code, like en for English.
|
Extension:Translate |
Whether or not the change was made through a Tor exit node | tor_exit_node
|
boolean | true if the action comes from a tor exit node. | Extension:TorBlock |
Whether or not a user is editing through the mobile interface | user_mobile
|
boolean | true for mobile users, false otherwise. | Extension:MobileFrontend |
⧼abusefilter-edit-builder-vars-user-app⧽ | user_app
|
boolean | true if the user is editing from the mobile app, false otherwise. | Extension:MobileApp |
Page views[1] | article_views
|
integer | <translate> (deprecated)</translate> Use page_views instead.
|
Extension:HitCounters |
Page views[2] | page_views
|
integer | the amount of page views | Extension:HitCounters |
Source page views[3] | moved_from_views
|
integer | the amount of page views of the source page | Extension:HitCounters |
Target page views[4] | moved_to_views
|
integer | the amount of page views of the target page | Extension:HitCounters |
Whether the IP address is blocked using the stopforumspam.com list[5] | sfs_blocked
|
boolean | Whether the IP address is blocked using the stopforumspam.com list | Extension:StopForumSpam |
Notes[edit]
When action='move'
, only the summary
, action
, timestamp
and user_*
variables are available.
The page_*
variables are also available, but the prefix is replaced by moved_from_
and moved_to_
, that represent the values of the original article name and the destination one, respectively.
For example, moved_from_title
and moved_to_title
instead of page_title
.
Since MediaWiki 1.28 (gerrit:295254), action='upload'
is only used when publishing an upload, and not for uploads to stash.
A new action='stashupload'
is introduced, which is used for all uploads, including uploads to stash.
This behaves like action='upload'
used to, and only provides file metadata variables (file_*
).
Variables related to the page edit, including summary
, new_wikitext
and several others, are now available for action='upload'
.
For every file upload, filters may be called with action='stashupload'
(for uploads to stash), and are always called with action='upload'
; they are not called with action='edit'
.
Filter authors should use action='stashupload' | action='upload'
in filter code when a file can be checked based only on the file contents – for example, to reject low-resolution files – and action='upload'
only when the wikitext parts of the edit need to be examined too – for example, to reject files with no description.
This allows tools that separate uploading the file and publishing the file (e.g. UploadWizard or upload dialog) to inform the user of the failure before they spend the time filling in the upload details.
Performance[edit]
As noted in the table above, some of these variables can be very slow.
While writing filters, remember that the condition limit is not a good metric of how heavy filters are.
For instance, variables like *_recent_contributors
or *_links
always need a DB query to be computed, while *_pst
variables will have to perform parsing of the text, which again is a heavy operation; all these variables should be used very, very carefully.
For instance, on Italian Wikipedia it's been observed that, with 135 active filters and an average of 450 used conditions, filters execution time was around 500ms, with peaks reaching 15 seconds.
Removing the added_links
variable from a single filter, and halving the cases when another filter would use added_lines_pst
brought the average execution time to 50ms.
More specifically:
- Use
_links
variables when you need high accuracy and checking for "http://..." in other variables (for instance,added_lines
) could lead to heavy malfunctioning; - Use
_pst
variables when you're really sure that non-PST variables aren't enough. You may also conditionally decide which one to check: if, for instance, you want to examine a signature, check first ifadded_lines
contains~~~
; - In general, when dealing with these variables, it's always much better to consume further conditions but avoid computing heavy stuff. In order to achieve this, always put heavy variables as last conditions.
Last but not least, note that whenever a variable is computed for a given filter, it'll be saved and any other filter will immediately retrieve it. This means that one single filter computing this variable counts more or less as dozens of filters using it.
Keywords[edit]
The following special keywords are included for often-used functionality:
like
(ormatches
) returns true if the left-hand operand matches the glob pattern in the right-hand operand.in
returns true if the right-hand operand (a string) contains the left-hand operand. Note: empty strings are not contained in, nor contain, any other string (not even the empty string itself).contains
works likein
, but with the left and right-hand operands switched. Note: empty strings are not contained in, nor contain, any other string (not even the empty string itself).rlike
(orregex
) andirlike
return true if the left-hand operand matches (contains) the regex pattern in the right-hand operand (irlike
is case insensitive).if ... then ... end
if ... then ... else ... end
... ? ... : ...
true
,false
,null
Examples
Code | Result | Comment |
---|---|---|
"1234" like "12?4"
|
True | |
"1234" like "12*"
|
True | |
"foo" in "foobar"
|
True | |
"foobar" contains "foo"
|
True | |
"o" in ["foo", "bar"]
|
True | Due to the string cast |
"foo" regex "\w+"
|
True | |
"a\b" regex "a\\\\b"
|
True | To look for the escape character backslash using regex you need to use either four backslashes or two \x5C . (Either works fine.)
|
"a\b" regex "a\x5C\x5Cb"
|
True |
Functions[edit]
A number of built-in functions are included to ease some common issues.
They are executed in the general format functionName( arg1, arg2, arg3 )
, and can be used in place of any literal or variable.
Its arguments can be given as literals, variables, or even other functions.
name | description |
---|---|
lcase |
Returns the argument converted to lower case. |
ucase |
Returns the argument converted to upper case. |
length |
Returns the length of the string given as the argument. If the argument is an array, returns its number of elements. |
string |
Casts to string data type. If the argument is an array, implodes it with linebreaks. |
int |
Casts to integer data type. |
float |
Casts to floating-point data type. |
bool |
Casts to boolean data type. |
norm |
Equivalent to rmwhitespace(rmspecials(rmdoubles(ccnorm(arg1)))) .
|
ccnorm
|
Normalises confusable/similar characters in the argument, and returns a canonical form. A list of characters and their replacements can be found on git, e.g. ccnorm( "Eeèéëēĕėęě3ƐƷ" ) === "EEEEEEEEEEEEE" .[9] The output of this function is always uppercase. While not expensive, this function isn't cheap either, and could slow a filter down if called many times.
|
ccnorm_contains_any
|
Normalises confusable/similar characters in all its arguments, and returns true if the first string contains any string from the following arguments (unlimited number of arguments, logic OR mode). A list of characters and their replacements can be found on git. Due to the usage of ccnorm , this function can be slow if passed too many arguments.
|
ccnorm_contains_all
|
Normalises confusable/similar characters in all its arguments, and returns true if the first string contains every string from the following arguments (unlimited number of arguments, logic AND mode). A list of characters and their replacements can be found on git. Due to the usage of ccnorm , this function can be slow if passed too many arguments.
|
specialratio |
Returns the number of non-alphanumeric characters divided by the total number of characters in the argument. |
rmspecials |
Removes any special characters in the argument, and returns the result. Does not remove whitespace. (Equivalent to s/[^\p{L}\p{N}\s]//g.) |
rmdoubles |
Removes repeated characters in the argument, and returns the result. |
rmwhitespace |
Removes whitespace (spaces, tabs, newlines). |
count |
Returns the number of times the needle (first string) appears in the haystack (second string). If only one argument is given, splits it by commas and returns the number of segments. |
rcount
|
Similar to count but the needle uses a regular expression instead. Can be made case-insensitive by letting the regular expression start with "(?i)". Please note that, for plain strings, this function can be up to 50 times slower than count [10], so use that one when possible.
|
get_matches
|
MW 1.31+ Looks for matches of the regex needle (first string) in the haystack (second string). Returns an array where the 0 element is the whole match and every [n] element is the match of the n'th capturing group of the needle. Can be made case-insensitive by letting the regular expression start with "(?i)". If a capturing group didn't match, that array position will take value of false.
|
ip_in_range |
Returns true if user's IP (first string) matches the specified IP range (second string, can be in CIDR notation, explicit notation like "1.1.1.1-2.2.2.2", or a single IP). Only works for anonymous users. Supports both IPv4 and IPv6 addresses. |
ip_in_ranges |
Returns true if user's IP (first string) matches any of the specified IP ranges (following strings in logic OR mode, can be in CIDR notation, explicit notation like "1.1.1.1-2.2.2.2", or a single IP). Only works for anonymous users. Supports both IPv4 and IPv6 addresses.
|
contains_any |
Returns true if the first string contains any string from the following arguments (unlimited number of arguments in logic OR mode). If the first argument is an array, it gets casted to string. |
contains_all |
Returns true if the first string contains every string from the following arguments (unlimited number of arguments in logic AND mode). If the first argument is an array, it gets casted to string. |
equals_to_any |
Returns true if the first argument is identical (=== ) to any of the following ones (unlimited number of arguments). Basically, equals_to_any(a, b, c) is the same as a===b | a===c , but more compact and saves conditions.
|
substr |
Returns the portion of the first string, by offset from the second argument (starts at 0) and maximum length from the third argument (optional). |
strlen |
Same as length .
|
strpos
|
Returns the numeric position of the first occurrence of needle (second string) in the haystack (first string), starting from offset from the third argument (optional, default is 0). This function may return 0 when the needle is found at the beginning of the haystack, so it might be misinterpreted as false value by another comparative operator. The better way is to use === or !== for testing whether it is found. Differently from PHP's strpos(), which returns false when the needle is not found, this function returns -1 when the needle is not found.
|
str_replace |
Replaces all occurrences of the search string with the replacement string. The function takes 3 arguments in the following order: text to perform the search on, text to find, replacement text. |
str_replace_regexp |
Replaces all occurrences of the search string with the replacement string using regular expressions. The function takes 3 arguments in the following order: text to perform the search on, regular expression to match, replacement expression. |
rescape |
Returns the argument with some characters preceded with the escape character "\", so that the string can be used in a regular expression without those characters having a special meaning. |
set |
Sets a variable (first string) with a given value (second argument) for further use in the filter. Another syntax: name := value .
|
set_var |
Same as set .
|
Examples[edit]
Code | Result | Comment |
---|---|---|
length( "Wikipedia" )
|
9 | |
lcase( "WikiPedia" )
|
wikipedia | |
ccnorm( "w1k1p3d14" )
|
WIKIPEDIA | ccnorm output is always uppercase
|
ccnorm( "ωɨƙɩᑭƐƉ1α" )
|
WIKIPEDIA | |
ccnorm_contains_any( "w1k1p3d14", "wiKiP3D1A", "foo", "bar" )
|
true | |
ccnorm_contains_any( "w1k1p3d14", "foo", "bar", "baz" )
|
false | |
ccnorm_contains_any( "w1k1p3d14 is 4w3s0me", "bar", "baz", "some" )
|
true | |
ccnorm( "ìíîïĩїį!ľ₤ĺľḷĿ" )
|
IIIIIII!LLLLLL | |
norm( "!!ω..ɨ..ƙ..ɩ..ᑭᑭ..Ɛ.Ɖ@@1%%α!!" )
|
WIKIPEDAIA | |
norm( "F00 B@rr" )
|
FOBAR | norm removes whitespace, special characters and duplicates, then uses ccnorm
|
rmdoubles( "foobybboo" )
|
fobybo | |
specialratio( "Wikipedia!" )
|
0.1 | |
count( "foo", "foofooboofoo" )
|
3 | |
count( "foo,bar,baz" )
|
3 | |
rmspecials( "FOOBAR!!1" )
|
FOOBAR1 | |
rescape( "abc* (def)" )
|
abc\* \(def\) | |
str_replace( "foobarbaz", "bar", "-" )
|
foo-baz | |
str_replace_regexp( "foobarbaz", "(.)a(.)", "$2a$1" )
|
foorabzab | |
ip_in_range( "127.0.10.0", "127.0.0.0/12" )
|
true | |
ip_in_ranges( "127.0.10.0", "10.0.0.0/8", "127.0.0.0/12" )
|
true | |
contains_any( "foobar", "x", "y", "f" )
|
true | |
get_matches( "(foo?ba+r) is (so+ good)", "fobaaar is soooo good to eat" )
|
['fobaaar is soooo good', 'fobaaar', 'soooo good'] |
Order of operations[edit]
Operations are generally done left-to-right, but there is an order to which they are resolved. As soon as the filter fails one of the conditions, it will stop checking the rest of them (due to short-circuit evaluation) and move on to the next filter. The evaluation order is:
- Anything surrounded by parentheses (
(
and)
) is evaluated as a single unit. - Turning variables/literals into their respective data. (e.g.,
page_namespace
to 0) - Function calls (
norm
,lcase
, etc.) - Unary
+
and-
(defining positive or negative value, e.g.-1234
,+1234
) - Keywords (
in
,rlike
, etc.) - Boolean inversion (
!x
) - Exponentiation (
2**3 → 8
) - Multiplication-related (multiplication, division, modulo)
- Addition and subtraction (
3-2 → 1
) - Comparisons (
<
,>
,==
) - Boolean operations (
&
,|
,^
) - Ternary operator (
... ? ... : ...
) - Assignments (
:=
)
Examples[edit]
A & B | C
is equivalent to(A & B) | C
, not toA & (B | C)
. In particular, bothfalse & true | true
andfalse & false | true
evaluates totrue
.A | B & C
is equivalent to(A | B) & C
, not toA | (B & C)
. In particular, bothtrue | true & false
andtrue | false & false
evaluates tofalse
.added_lines rlike "foo" + "|bar"
is wrong, useadded_lines rlike ("foo" + "|bar")
instead.
Condition counting[edit]
The condition limit is (more or less) tracking the number of comparison operators + number of function calls entered.
Further explanation on how to reduce conditions used can be found at Help:Abuse Filter.
Exclusions[edit]
Although the AbuseFilter examine function will identify "rollback" actions as edits, the AbuseFilter will not evaluate rollback actions for matching.[11]
Useful links[edit]
Notes[edit]
- ↑ Comparing arrays to other types will always return false, except for the example above
- ↑ 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 The only variables currently available for file uploads (action='upload') are user_*, page_*, file_sha1, file_size, file_mime, file_mediatype, file_width, file_height, file_bits_per_channel (the last five were only added since the release for MediaWiki 1.27 gerrit:281503). All the file_* variables are unavailable for other actions (including action='edit').
- ↑ Since MediaWiki 1.28 (gerrit:295254)
- ↑ Several filters (12) that use this variable have showed up in the AbuseFilterSlow Grafana dashboard (requires logstash access to view). Moving this variable to towards the end of the filter seemed to help.
- ↑ See phabricator:T191722
- ↑ Deprecated with this commit and disabled with this one.
- ↑ Some filters using this variable have showed up in the AbuseFilterSlow Grafana dashboard (example, requires logstash access). For instance, instead of using
"text" in edit_diff_pst
(or evenedit_diff
), consider something like"text" in added_lines & !("text" in removed_lines)
- ↑ See the source code for a list of types.
- ↑ Be aware of phab:T27619. You can use Special:AbuseFilter/tools to evaluate
ccnorm( "your string" )
to see which characters are transformed. - ↑ https://3v4l.org/S6IGP
- ↑ T24713 - rollback not matched by AF