attempt to straighten text ui out

.gitignore

@@ -1 +1,2 @@
 hardcoded_variables.py
+__pycache__

README.md

@@ -6,6 +6,64 @@ A Matrix moderation tool to make managing a Synapse server easier.
 Contact me at [@michael:perthchat.org](https://matrix.to/#/@michael:perthchat.org) if you get stuck or have an edit in mind.
 
 
+***
+## List of Functions
+
+This tool abstracts the Synapse API so you can perform common moderation functions easier and in batch.
+
+Here is a preview of the CLI interface:
+```
+##########################
+# MATRIX MODERATION TOOL #
+##########################
+
+A tool for making common Synapse moderation tasks easier. Created by @PC-Admin.
+
+----------------------------------------------
+
+#### User Account Commands ####			    	#### Room Commands ####
+1) Deactivate a user account.			    	50) List details of a room.
+2) Deactivate multiple user accounts.			51) List the members of a room.
+3) Create a user account.			        52) Export the state events of a room.
+4) Create multiple user accounts.		    	53) Export the state events of multiple rooms.
+5) Reset a users password.			        54) List rooms in public directory.
+6) Whois user account.				        55) Remove a room from the public directory.
+7) Whois multiple user accounts.		    	56) Remove multiple rooms from the public directory.
+8) Query user account.				        57) Redact a room event.
+9) Query multiple user accounts.		    	58) List/Download all media in a room.
+10) List room memberships of user.		    	59) Download media from multiple rooms.
+11) Promote a user to server admin.		    	60) Quarantine all media in a room.
+12) List all user accounts.			        61) Shutdown a room.
+13) Quarantine all media a users uploaded.		62) Shutdown multiple rooms.
+14) Collect account data.			        63) Delete a room.
+15) List account pushers.			        64) Delete multiple rooms.
+16) Get rate limit of a user account.			65) Purge the event history of a room to a specific timestamp.
+17) Set rate limit of a user account.			66) Purge the event history of multiple rooms to a specific timestamp.
+18) Delete rate limit of a user account.		67) Get blocked status for room.
+19) Check if user account exists.		    	68) Block a room.
+20) Shadow ban a user.				        69) Unblock a room.
+21) Find a user by their 3PID.
+
+#### Server Commands ####					#### ipinfo.io ####
+100) Delete and block a specific media.				140) Analyse a users country of origin.
+101) Purge remote media repository up to a certain date.	141) Analyse multiple users country of origin.
+102) Prepare database for copying events of multiple rooms.
+103) Show last 10 reported events.				#### Report Generation ####
+104) Get all reported events.					150) Generate user report.
+105) Get details of a reported event.				151) Lookup homeserver admin contact details.
+								152) Send a test email (to yourself).
+#### rdlist - General ####					153) Send a test Matrix message (to yourself).
+120) Block all rooms with specific rdlist tags.			154) Send test incident reports (to yourself).
+121) Get rdlist tags for a room.
+
+#### rdlist - Recommended Tags ####
+For rdlist rooms with recommended tags, the following actions are available:
+130) Collect User Reports on local accounts in rdlist rooms.
+131) Send Incident Reports on remote accounts in rdlist rooms.
+132) Block/Purge all rdlist rooms.
+```
+
+
 ***
 ## Licensing
 
@@ -15,17 +73,26 @@ This work is published under the MIT license, for more information on this licen
 ***
 ## Setup script
 
-You can hard code the server URL, federation port and access token into the [hardcoded_variables.py](./hardcoded_variables.py) file for faster use, it will prompt you for these values if you don't.
+Firstly, you need hard code the 'server URL', 'federation port' and 'access token' into the [hardcoded_variables.py](./hardcoded_variables.py) file 
+```
+$ cp ./hardcoded_variables.py.sample ./hardcoded_variables.py
+$ nano ./hardcoded_variables.py
+```
 
 Your access token can be found in Element > Settings > Help & About, your user account must first be upgraded to a server admin.
 
 This script also requires you to install the following PIP packages:
 ```
-pip3 install python-whois
+$ pip3 install python-whois && \
-pip3 install requests
+pip3 install requests && \
 pip3 install matrix-nio
 ```
 
+***
+## Running the script
+
+`$ python3 moderation_tool.py`
+
 
 ***
 ## Upgrade user to 'server admin'
@@ -77,38 +144,6 @@ With the popular [matrix-docker-ansible-deploy](https://github.com/spantaleev/ma
 `matrix_synapse_admin_enabled: true`
 
 
-***
-## Roadmap
-
-To do:
-1) Add the following functions:
-- https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#find-a-user-based-on-their-id-in-an-auth-provider
-- https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#find-a-user-based-on-their-third-party-id-threepid-or-3pid
-- https://github.com/matrix-org/synapse/blob/master/docs/admin_api/delete_group.md
-2) Add fully automated (should just return a web link and decryption password) reporting functions for users:
-- Description of why the report was made (what happened), include key information
-- User's ID - DONE
-- Whois Data - DONE
-- Account Data - DONE
-- Query Data - DONE
-- Pushers List - DONE
-- IPs + ipinfo Data - DONE
-- List of the rooms the user is participating in, divided into 1:1 conversations and larger rooms - DONE
-- Any other usernames associated with that IP
-- Timestamp for when illegal material was accessed
-- Description of report format and contents (to guide the reader)
-- Collect state event dumps of recently read rooms as well (as they may have looked at other suss rooms recently)
-3) Have recommended rdlist function:
-- return a list of offending accounts and the tags they accessed (for creating incident_dict's)
-- add the shadowban function to prevent members alerting others after mass shutdowns - DONE
-4) Only email reportID in incident report?
-5) Add a room report function to create a properly formatted report for rdlist
-6) Expand the incident reporting to also issue reports over Matrix
-7) Automated public room joining and reminder if reporting email is not available?
-8) Refine ipinfo module to also return extra details about the IP
-9) Make existing functions compatible with JSON formatted inputs
-
-
 ***
 ## rdlist Functionality
 
@@ -118,97 +153,13 @@ This script can automatically load and block/purge abusive rooms from rdlist, ma
 
 If you are running a public server, please dm me at [@michael:perthchat.org](https://matrix.to/#/@michael:perthchat.org) and I can invite you to the 'Legion of Janitors' room.
 
-Once you have read access to the [rdlist repository](https://code.glowers.club/loj/rdlist), simply run this moderation script like so:
+For more information on rdlist related function consult the [support document](./docs/rdlist_functions.md).
-```
-$ python3 moderation_tool.py 
-
-Please select one of the following options:
-...
-Please enter a number from the above menu, or enter 'q' or 'e' to exit.
-
-121
-
-@mod_team:perthchat.org account already exists. Resetting account password.
-
-Ensuring @mod_team:perthchat.org account is a server admin.
-
-rdlist repo already cloned...
-Fetching origin
-rdlist repo is up-to-date, no need to pull changes.
-
-Using recommended rdlist tags. Rooms matching the following tags will be purged and/or blocked:
-['hub_room_links', 'hub_room_trade', 'preban', 'degen_misc', 'beastiality', 'degen_porn', 'gore', 'snuff', 'degen_larp', 'hub_room_sussy', 'bot_spam', 'cfm', 'jailbait', 'bot_porn', 'toddlercon', 'loli', 'csam', 'tfm', 'degen_meet', 'stylized_3d_loli', '3d_loli']
-
-WARNING! The following local users are current members of rooms tagged in rdlist: ['@***REDACTED***:perthchat.org']
-
-Do you want to generate a user report file for each of these users? y/n? n
-
-Skipping user report generation...
-
-
-Number of rdlist rooms being shutdown: 346
-
-Are you sure you want to block/shutdown these rooms? y/n? y
-
-
-Skipping already blocked room: !***REDACTED***:matrix.org
-
-
-Skipping already blocked room: !***REDACTED***:matrix.org
-
-
-Skipping already blocked room: !***REDACTED***:matrix.org
-
-
-Blocking unknown room: !***REDACTED***:matrix.org
-Successfully blocked room !***REDACTED***:matrix.org
-
-
-Blocking unknown room: !***REDACTED***:matrix.org
-Successfully blocked room !***REDACTED***:matrix.org
-
-
-Skipping already blocked room: !***REDACTED***:matrix.org
-
-
-Shutting down known room: !***REDACTED***:sibnsk.net
-Sleeping for 2 seconds...
-Sleeping for 4 seconds...
-Sleeping for 8 seconds...
-!***REDACTED***:sibnsk.net has been successfully shutdown!
-List of kicked users:
-@***REDACTED***:perthchat.org
-
-
-Skipping already blocked room: !***REDACTED***:anontier.nl
-
-
-Room shutdowns completed!
-
-User login details for your moderator account:
-
-Username: mod_team
-Password: ***REDACTED***
-
-Print rdlist statistics:
-
-Number of rooms blocked: 4
-Number of rooms purged: 2
-Number of local users located in rdlist rooms and kicked: 1
-
-The following users were current members of rooms tagged in rdlist: ['@***REDACTED***:perthchat.org']
-
-Do you want to also deactivate all these accounts that were kicked from rdlist rooms? y/n?
-...
-```
-
-Note that this script before shutting these rooms down will save the state events to the "./state_events" folder, please keep this data as it's important for law enforcement.
 
 
 ***
 ## One-touch Reporting
 
-WARNING: This section is under heavy development and shouldn't be used by anyone!!!
+CAUTION: This section is under heavy development and probably shouldn't be used by anyone!
 
 This script can automatically generate reports about user accounts for law enforcement.
 
@@ -219,15 +170,43 @@ Please enter a number from the above menu, or enter 'q' or 'e' to exit.
 
 150
 
-Please enter the username to automatically generate a report: michael
+Please enter the username to automatically generate a report: pedobear
 
 ...
 
-Report generated successfully on user: "michael"
+Report generated successfully on user: "pedobear"
 
 You can send this .zip file and password when reporting a user to law enforcement.
 
-Password: RwiFrw9zouhVO7Dy9kW7
+.zip file location: ./reports/pedobear_2023-07-23_02-21-56.zip
-Encrypted .zip file location: ./reports/michael_2023-07-23_02-21-56.zip.aes
+.zip file size: 0.503927 MB
-Encrypted .zip file size: 0.503927 MB
 ```
+
+
+***
+## Roadmap
+
+To do:
+1) Add the following functions:
+- https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#find-a-user-based-on-their-id-in-an-auth-provider
+- https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#find-a-user-based-on-their-third-party-id-threepid-or-3pid - DONE
+- https://github.com/matrix-org/synapse/blob/master/docs/admin_api/delete_group.md
+- https://matrix-org.github.io/synapse/v1.38/admin_api/rooms.html#make-room-admin-api
+- https://matrix-org.github.io/synapse/latest/admin_api/server_notices.html
+- https://matrix-org.github.io/synapse/latest/admin_api/event_reports.html
+- https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/federation.html#destination-rooms
+2) Add fully automated (should just return a web link and decryption password) reporting functions for users:
+- Description of why the report was made (what happened), include key information
+- Any other usernames associated with that IP
+- Timestamp for when illegal material was accessed
+- Description of report format and contents (to guide the reader)
+- Collect state event dumps of recently read rooms as well (as they may have looked at other suss rooms recently)
+3) Have recommended rdlist function:
+- return a list of offending accounts and the tags they accessed (for creating incident_dict's) - DONE
+- add the shadowban function to prevent members alerting others after mass shutdowns - DONE
+4) Only email reportID in incident report?
+5) Add a room report function to create a properly formatted report for rdlist
+6) Expand the incident reporting to also issue reports over Matrix
+7) Automated public room joining and reminder if reporting email is not available?
+8) Refine ipinfo module to also return extra details about the IP
+9) Make existing functions compatible with JSON formatted inputs

docs/rdlist_functions.md

@@ -0,0 +1,182 @@
+
+# rdlist Functions
+
+'rdlist' is a comprehensive list of child abuse related rooms on Matrix, it's a safety initiative led by the [Legion of Janitors](https://matrix.to/#/#janitors:glowers.club).
+
+This script can automatically load and block/purge abusive rooms from rdlist, making it **very easy** for inexperienced administrators to block this harmful content.
+
+If you are running a public server, please dm me at [@michael:perthchat.org](https://matrix.to/#/@michael:perthchat.org) and I can invite you to the 'Legion of Janitors' room.
+
+***
+## Collect User Reports on local users in rdlist rooms
+
+This script can automatically generate 'User Reports' for each one of your local users in rdlist rooms that have the 'recommended tags'.
+
+These user reports can be given to law enforcement or shared in [#janitor-dumps](https://matrix.to/#/#janitor-dumps:glowers.club) to help us locate more abusive users/rooms.
+
+```
+130
+
+rdlist repo already cloned...
+Fetching origin
+Pulling latest changes from rdlist repo...
+
+WARNING! The following local users are current members of rooms tagged in rdlist: ['@fatweeb23838:perthchat.org', '@somecreep29330:perthchat.org']
+
+Do you want to generate a user report file for each of these users? y/n? y
+
+Generating user report for fatweeb23838...
+Report generated successfully on user: "fatweeb23838"
+
+You can send this .zip file when reporting a user to law enforcement.
+.zip file location: /home/pcadmin/projects/matrix-moderation-tool/reports/fatweeb23838_2023-08-01_23-19-24.zip
+.zip file size: 0.00966 MB
+
+
+Generating user report for somecreep29330...
+Report generated successfully on user: "somecreep29330"
+
+You can send this .zip file when reporting a user to law enforcement.
+.zip file location: /home/pcadmin/projects/matrix-moderation-tool/reports/somecreep29330_2023-08-01_23-19-27.zip
+.zip file size: 0.29578 MB
+```
+
+'rdlist' is a comprehensive list of child abuse related rooms on Matrix, it's a safety initiative led by the [Legion of Janitors](https://matrix.to/#/#janitors:glowers.club).
+
+This script can automatically load and block/purge abusive rooms from rdlist, making it **very easy** for inexperienced administrators to block this harmful content.
+
+If you are running a public server, please dm me at [@michael:perthchat.org](https://matrix.to/#/@michael:perthchat.org) and I can invite you to the 'Legion of Janitors' room.
+
+Once you have read access to the [rdlist repository](https://code.glowers.club/loj/rdlist), this script can be used for multiple rdlist related functions.
+
+***
+## Send Incident Reports for remote users in rdlist rooms
+
+This script can automatically generate 'Incident Reports' for every remote homeserver admin with users in rdlist rooms that have the 'recommended tags'.
+
+It examines the homeserver involved to find a admin contact method via [MSC1929](https://github.com/matrix-org/matrix-spec-proposals/pull/1929). If an MXID is returned it will attempt to send the Incident Report over Matrix. If an email is provided it will send the Incident Report over email. If neither is found a whois lookup is performed and the Incident Report are sent to the domain registrar via email.
+
+```
+131
+
+rdlist repo already cloned...
+Fetching origin
+Pulling latest changes from rdlist repo...
+
+WARNING! The following remote users are current members of rooms tagged in rdlist: ['@pedobear:matrix.org', '@randomcreep:perthchat.org']
+
+Do you want to send out incident reports for these users to every homeserver admin involved? y/n? y
+
+Sending Incident Report for users from matrix.org to abuse@matrix.org
+
+Sending Incident Report for users from perthchat.org to @michael:perthchat.org
+
+```
+
+![A preview of an Incident Report being sent over Matrix.](https://github.com/PC-Admin/matrix-moderation-tool/assets/29645145/db5a4a56-fd66-413a-ac44-1216c7b2f1fd)
+
+
+## rdlist Block/Purge all rooms with recommended rdlist tags
+
+Finally this script can be used to shutdown rooms with the recommended rdlist tags.
+
+This function is much larger and will ask you if you also want to create user/incident reports before the shutdowns. (Recommended) It'll also ask you if you want to shadowban the users in these rooms to prevent them from alerting others. (Recommended) Finally it'll ask if you want to shutdown the local accounts located in these rooms.
+
+```
+$ python3 moderation_tool.py 
+
+Please select one of the following options:
+...
+Please enter a number from the above menu, or enter 'q' or 'e' to exit.
+
+132
+
+@mod_team:perthchat.org account already exists. Resetting account password.
+
+Ensuring @mod_team:perthchat.org account is a server admin.
+
+rdlist repo already cloned...
+Fetching origin
+rdlist repo is up-to-date, no need to pull changes.
+
+Using recommended rdlist tags. Rooms matching the following tags will be purged and/or blocked:
+['hub_room_links', 'hub_room_trade', 'preban', 'degen_misc', 'beastiality', 'degen_porn', 'gore', 'snuff', 'degen_larp', 'hub_room_sussy', 'bot_spam', 'cfm', 'jailbait', 'bot_porn', 'toddlercon', 'loli', 'csam', 'tfm', 'degen_meet', 'stylized_3d_loli', '3d_loli']
+
+WARNING! The following local users are current members of rooms tagged in rdlist: ['@***REDACTED***:perthchat.org']
+
+Do you want to generate a user report file for each of these users? y/n? n
+
+Skipping user report generation...
+
+WARNING! The following remote users are current members of rooms tagged in rdlist: ['@***REDACTED***:matrix.org']
+
+Do you want to send out incident reports for these users to every homeserver admin involved? y/n? n
+
+Skipping incident report generation...
+
+
+Number of rdlist rooms being shutdown: 346
+
+Are you sure you want to block/shutdown these rooms? y/n? y
+
+Do you want to also shadow ban all your local users in these rooms before performing these shutdowns? (This is recommended as it prevents them from alerting others about these mass shutdown.) y/n? y
+
+Shadow banning user: @***REDACTED***:perthchat.org
+
+
+Skipping already blocked room: !***REDACTED***:matrix.org
+
+
+Skipping already blocked room: !***REDACTED***:matrix.org
+
+
+Skipping already blocked room: !***REDACTED***:matrix.org
+
+
+Blocking unknown room: !***REDACTED***:matrix.org
+Successfully blocked room !***REDACTED***:matrix.org
+
+
+Blocking unknown room: !***REDACTED***:matrix.org
+Successfully blocked room !***REDACTED***:matrix.org
+
+
+Skipping already blocked room: !***REDACTED***:matrix.org
+
+
+Shutting down known room: !***REDACTED***:sibnsk.net
+Sleeping for 2 seconds...
+Sleeping for 4 seconds...
+Sleeping for 8 seconds...
+!***REDACTED***:sibnsk.net has been successfully shutdown!
+List of kicked users:
+@***REDACTED***:perthchat.org
+
+
+Skipping already blocked room: !***REDACTED***:anontier.nl
+
+
+Room shutdowns completed!
+
+User login details for your moderator account:
+
+Username: mod_team
+Password: ***REDACTED***
+
+Print rdlist statistics:
+
+Number of rooms blocked: 4
+Number of rooms purged: 2
+Number of local users located in rdlist rooms and kicked: 1
+
+The following users were current members of rooms tagged in rdlist: ['@***REDACTED***:perthchat.org']
+
+Do you want to also deactivate all these accounts that were kicked from rdlist rooms? y/n?
+
+https://matrix.perthchat.org/_synapse/admin/v1/deactivate/@***REDACTED***:perthchat.org
+
+Successfully deactivated account.
+
+These accounts have been deactivated.
+...
+```

docs/room_functions.md

@@ -0,0 +1,102 @@
+
+#### Room Commands ####
+
+50) **List details of a room.**
+
+> Shows comprehensive information about a specific room.
+
+https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#room-details-api
+
+51) **List the members of a room.**
+
+> Displays all the members who are part of a particular room.
+
+https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#room-members-api
+
+52) **Export the state events of a room.**
+
+> Retrieves and saves the state events (such as topic changes, member joins) of a specific room.
+
+https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#room-state-api
+
+53) **Export the state events of multiple rooms.**
+
+> Gathers and saves the state events from several rooms simultaneously.
+
+54) **List rooms in public directory.**
+
+> Displays all the rooms that are publicly available and listed in the directory.
+
+  https://spec.matrix.org/legacy/client_server/r0.6.0.html#get-matrix-client-r0-publicrooms
+
+55) **Remove a room from the public directory.**
+
+> Takes a specific room off the public directory, making it less accessible to users browsing the directory.
+
+56) **Remove multiple rooms from the public directory.**
+
+> Removes several rooms from the public directory at once.
+
+57) **Redact a room event.**
+
+> Conceals specific events in a room, such as messages or media, from being visible.
+
+58) **List/Download all media in a room.**
+
+> Displays and offers the option to download all media files shared in a specific room.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/media_admin_api.html#list-all-media-in-a-room
+
+59) **Download media from multiple rooms.**
+
+> Enables downloading of media files shared across several rooms.
+
+60) **Quarantine all media in a room.**
+
+> Isolates and restricts access to all media files in a specific room, usually due to concerns over inappropriate content.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/media_admin_api.html#quarantining-media-in-a-room
+
+61) **Shutdown a room.**
+
+> Terminates a room, preventing further activities or messages in it.
+
+https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#version-2-new-version
+
+62) **Shutdown multiple rooms.**
+
+> Closes several rooms simultaneously, halting any ongoing activity in them.
+
+63) **Delete a room.**
+
+> Permanently removes a room and all its content.
+
+64) **Delete multiple rooms.**
+
+> Erases several rooms and their content from the server at once.
+
+65) **Purge the event history of a room to a specific timestamp.**
+
+> Deletes all events in a room up to a certain point in time, preserving events after that timestamp.
+
+66) **Purge the event history of multiple rooms to a specific timestamp.**
+
+> Clears the events in several rooms up to a specified timestamp.
+
+67) **Get blocked status for room.**
+
+> Checks if a room is blocked and restricted from user access.
+
+https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#get-block-status
+
+68) **Block a room.**
+
+> Restricts access to a room, preventing users from joining or interacting in it.
+
+https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#block-or-unblock-a-room
+
+69) **Unblock a room.**
+
+> Restores access to a previously blocked room, allowing users to join and participate once more.
+
+**Note:** Managing rooms is a sensitive operation, especially when dealing with content and user privacy. Ensure you have the required permissions and handle operations with care.

docs/server_functions.md

@@ -0,0 +1,100 @@
+
+# Server Commands Guide
+
+This guide provides detailed steps for server-side operations that use the database and SSH. The commands and scripts are essential for handling specific server operations related to Matrix's Synapse server.
+
+## Table of Contents
+
+- [1. Delete and Block Specific Media](#1-delete-and-block-specific-media)
+- [2. Purge Remote Media Repository](#2-purge-remote-media-repository)
+- [3. Prepare Database for Copying Events of Multiple Rooms](#3-prepare-database-for-copying-events-of-multiple-rooms)
+
+---
+
+100) **Delete and Block Specific Media.**	
+
+> This command allows an admin to delete a specific media on their Matrix Synapse server and block it to prevent future accesses.
+
+#### Process Flow:
+
+1. Take `media_id` and remote server URL from the user.
+2. Use SSH to query the Synapse PostgreSQL database for the associated `filesystem_id`.
+3. Locate the target media files and thumbnails on the server's file system.
+4. Zero out (empty) each file and make them immutable, meaning they cannot be modified or deleted.
+
+#### Example:
+
+For a media with ID `eDmjusOjnHyFPOYGxlrOsULJ`, the process would involve:
+
+```bash
+$ ssh matrix.perthchat.org "... SQL query to get filesystem_id..."
+$ ssh matrix.perthchat.org "... command to locate files ..."
+$ ssh matrix.perthchat.org "true > ...path to file..."
+$ ssh matrix.perthchat.org "chattr +i ...path to file..."
+```
+
+101) **Purge Remote Media Repository**
+
+This command purges the remote media repository for a certain range of days.
+Process Flow:
+
+    Ask the user for the range of days to purge.
+    Calculate the epoch timestamp for each day in the range.
+    Send a request to the Synapse server to purge media for that day.
+    Repeat for each day in the range.
+
+Example:
+```bash
+$ date --date '149 days ago' +%s
+$ curl -X POST --header "Authorization: Bearer ACCESS_TOKEN" '... Matrix Synapse purge endpoint ...'
+```
+
+102) **Prepare Database for Copying Events of Multiple Rooms**
+
+This command prepares the PostgreSQL database to export events from multiple Matrix rooms.
+Process Flow:
+
+    Prompt for a list of room IDs.
+    Create a RAM disk on the server to store the export.
+    For each room ID:
+        Create a SQL query to extract room events.
+        Write the query to a file on the RAM disk.
+    Provide instructions for running the queries in the PostgreSQL container.
+
+Notes:
+
+    This function is compatible with Spantaleev's Matrix deploy script.
+    Ensure proper permissions and consider the impact on the server when copying a large amount of data.
+
+Example:
+```bash
+# As the root user on the target server:
+$ mkdir /matrix/postgres/data/ramdisk
+$ ... commands to set up RAM disk ...
+$ ... commands to generate SQL queries for each room ...
+$ docker exec -it matrix-postgres /bin/bash
+bash-5.0$ ... commands to execute SQL queries ...
+```
+
+After copying the data, ensure to clean up the RAM disk:
+
+```bash
+$ rm -r /matrix/postgres/data/ramdisk/*
+$ umount /matrix/postgres/data/ramdisk
+```
+
+103) **Show last 10 reported events.**
+
+> Gets the last 10 reported events using the event reports API and returns it.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/event_reports.html#show-reported-events
+
+104) **Paginate all reported events.**
+
+> Combines all the events into a large JSON and returns it.
+
+105) **Show details of a specific event report**
+
+> This API returns information about a specific event report.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/event_reports.html#show-details-of-a-specific-event-report

docs/user_functions.md

@@ -0,0 +1,118 @@
+
+#### User Account Commands ####
+
+1) **Deactivate a user account.**
+
+> This function disables a specific user's account, making it unusable for the owner.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#deactivate-account
+
+2) **Deactivate multiple user accounts.**
+
+> Allows you to disable several user accounts at once. Requires a room list file with room_ids separated by newlines, see the example [./examples/room_list.txt](./examples/room_list.txt) file.
+
+3) **Create a user account.**
+
+> Use this to generate a new user account.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#create-or-modify-account
+
+4) **Create multiple user accounts.**
+
+> This facilitates the creation of several user accounts simultaneously. Requires a user list file with user_ids separated by newlines, see the example [./examples/user_list.txt](./examples/user_list.txt) file.
+
+5) **Reset a user's password.**
+
+> If a user forgets their password, this function helps set a new one.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#reset-password
+
+6) **Whois user account.**	
+
+> This API returns information about the active sessions for a specific user.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#query-current-sessions-for-a-user
+
+7) **Whois multiple user accounts.**
+
+> Retrieves detailed information for multiple user accounts at once. Requires a user list file with user_ids separated by newlines, see the example [./examples/user_list.txt](./examples/user_list.txt) file.
+
+8) **Query user account.**	
+
+> Allows you to get specific details or attributes of a user account.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#query-user-account
+
+9) **Query multiple user accounts.**
+
+> Retrieve specific details for several user accounts simultaneously. Requires a user list file with user_ids separated by newlines, see the example [./examples/user_list.txt](./examples/user_list.txt) file.
+
+10) **List room memberships of user.**	
+
+> Displays the list of rooms that a user is a part of.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#list-room-memberships-of-a-user
+
+11) **Promote a user to server admin.**	
+
+> Elevates a user's privileges, making them an administrator on the server.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#change-whether-a-user-is-a-server-administrator-or-not
+
+12) **List all user accounts.**
+
+> Displays or prints to file a comprehensive list of all user accounts on the server.
+
+  https://matrix-org.github.io/synapse/v1.38/admin_api/user_admin_api.html#list-accounts
+
+13) **Quarantine all media a user uploaded.**
+
+> This API quarantines all local media that a local user has uploaded. That is to say, if you would like to quarantine media uploaded by a user on a remote homeserver, you should instead use one of the other APIs. Useful for potential harmful or inappropriate content.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/media_admin_api.html#quarantining-all-media-of-a-user
+
+14) **Collect account data.**
+
+> Retrieves all available data associated with a user's account.
+
+15) **List account pushers.**
+
+> Shows devices and services that have push access to a user's account.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#list-all-pushers
+
+16) **Get rate limit of a user account.**
+
+> Displays the frequency at which a user can make requests or actions.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#get-status-of-ratelimit
+
+17) **Set rate limit of a user account.**
+
+> Adjusts the frequency rate at which a user can make requests or actions.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#set-ratelimit
+
+18) **Delete rate limit of a user account.**
+
+> Removes any rate limits set on a user's account, granting them unrestricted action frequency.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#delete-ratelimit
+
+19) **Check if user account exists.**
+
+> Verifies the existence of a specific user_id, for example "@johndoe:example.org" on the homeserver.
+
+20) **Shadow ban a user.**	
+
+> Shadow-banning is a useful tool for moderating malicious or egregiously abusive users. A shadow-banned users receives successful responses to their client-server API requests, but the events are not propagated into rooms. This can be an effective tool as it (hopefully) takes longer for the user to realise they are being moderated before pivoting to another account.
+
+> Shadow-banning a user should be used as a tool of last resort and may lead to confusing or broken behaviour for the client. A shadow-banned user will not receive any notification and it is generally more appropriate to ban or kick abusive users. A shadow-banned user will be unable to contact anyone on the server.
+
+  https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#controlling-whether-a-user-is-shadow-banned
+
+21) **Find a user by their 3PID (Third-party ID).**
+  
+> Allows you to locate a user based on their third-party identifiers, such as email or phone number.
+
+**Note:** All these commands utilize the Synapse API. Always exercise caution and ensure you have the necessary permissions when accessing and modifying user accounts.

hardcoded_variables.py.sample

@@ -7,12 +7,13 @@ access_token = ""			# Your homeserver admin access token
 # ipinfo.io token
 ipinfo_token = ""			# Leave blank to disable ipinfo.io lookups
 # rdlist specific
+rdlist_dir = "~/rdlist"             	# The directory where rdlist is pulled to
 rdlist_bot_username = "mod_team"	# The username to perform automated room shutdowns
 rdlist_bot_username = "strong-password"	# The password for this user
 rdlist_recommended_tags = ['hub_room_links', 'hub_room_trade', 'preban', 'degen_misc', 'beastiality', 'degen_porn', 'gore', 'snuff', 'degen_larp', 'hub_room_sussy', 'bot_spam', 'cfm', 'jailbait', 'bot_porn', 'toddlercon', 'loli', 'csam', 'tfm', 'degen_meet', 'stylized_3d_loli', '3d_loli']
 # User report generator
 report_folder = "./reports"         # Reports folder name
-testing_mode = True                 # Prevents the incident report feature from messaging/emailing anyone besides you, also limits the number of room states are exported when generating user reports.
+testing_mode = False                # Prevents the incident report feature from messaging/emailing anyone besides you, also limits the number of room states are exported when generating user reports.
 # Incident report email settings
 smtp_user = "abuse@matrix.example.org"
 smtp_password = "strong-stmp-password"

moderation_tool.py

@@ -1,5 +1,14 @@
 
+#!/bin/env python3
+
+import os
 import json
+
+# Check if ./hardcoded_variables.py file exists
+if not os.path.exists("./hardcoded_variables.py"):
+	print("ERROR: The file './hardcoded_variables.py' does not exist. It must be configured before using this script.")
+	exit()
+
 import user_commands
 import room_commands
 import server_commands
@@ -7,24 +16,31 @@ import ipinfo_commands
 import rdlist_commands
 import report_commands
 import bot_commands
+
+# Importing the module only after verifying its existence
 import hardcoded_variables
 
-# check if homeserver url is hard coded, if not set it
+# If it does exist... check if the variables are configured:
+
+# check if homeserver url is the default
 
 if hardcoded_variables.homeserver_url == "matrix.example.org":
-	homeserver_url = input("What is the URL of your server? Eg: matrix.example.org ")
+	print("ERROR: homeserver_url not configured, please configure your './hardcoded_variables.py' file!")
+	exit()
 
-# check if base url is hard coded, if not set it
+# check if base url is the default
 
 if hardcoded_variables.base_url == "example.org":
-	base_url = input("What is the URL of your server? Eg: example.org ")
+	print("ERROR: base_url not configured, please configure your './hardcoded_variables.py' file!")
+	exit()
 
-# check if access token is hard coded, if not set it
+# check if access token is the default
 
 length_access_token = len(hardcoded_variables.access_token)
 
 if length_access_token == 0:
-	access_token = input("Please enter access token for server admin account: ")
+	print("ERROR: access_token not configured, please configure your './hardcoded_variables.py' file!")
+	exit()
 
 # loop menu for various moderation actions
 
@@ -56,18 +72,23 @@ while pass_token == False:
 	print("18) Delete rate limit of a user account.\t67) Get blocked status for room.")
 	print("19) Check if user account exists.\t\t68) Block a room.")
 	print("20) Shadow ban a user.\t\t\t\t69) Unblock a room.")
-	print("\n#### Server Commands ####\t\t\t\t\t#### Report Generation ####")
+	print("21) Find a user by their 3PID.")
-	print("100) Delete and block a specific media.\t\t\t\t150) Generate user report.")
+	print("\n#### Server Commands ####\t\t\t\t\t#### ipinfo.io ####")
-	print("101) Purge remote media repository up to a certain date.\t151) Lookup homeserver admin contact details.")
+	print("100) Delete and block a specific media.\t\t\t\t140) Analyse a users country of origin.")
-	print("102) Prepare database for copying events of multiple rooms.\t152) Send a test email (to yourself).")
+	print("101) Purge remote media repository up to a certain date.\t141) Analyse multiple users country of origin.")
-	print("\t\t\t\t\t\t\t\t153) Sent a test Matrix message (to yourself).")
+	print("102) Prepare database for copying events of multiple rooms.")
-	print("#### rdlist ####\t\t\t\t\t\t154) Send test incident reports (to yourself).")
+	print("103) Show last 10 reported events.\t\t\t\t#### Report Generation ####")
-	print("120) Block all rooms with specific rdlist tags.")
+	print("104) Get all reported events.\t\t\t\t\t150) Generate user report.")
-	print("121) Block all rooms with recommended rdlist tags.")
+	print("105) Get details of a reported event.\t\t\t\t151) Lookup homeserver admin contact details.")
-	print("122) Get rdlist tags for a room.")
+	print("\t\t\t\t\t\t\t\t152) Send a test email (to yourself).")
-	print("\n#### ipinfo.io ####")
+	print("#### rdlist - General ####\t\t\t\t\t153) Send a test Matrix message (to yourself).")
-	print("140) Analyse a users country of origin.")
+	print("120) Block all rooms with specific rdlist tags.\t\t\t154) Send test incident reports (to yourself).")
-	print("141) Analyse multiple users country of origin.")
+	print("121) Get rdlist tags for a room.")
+	print("\n#### rdlist - Recommended Tags ####")
+	print("For rdlist rooms with recommended tags, the following actions are available:")
+	print("130) Collect User Reports on local accounts in rdlist rooms.")
+	print("131) Send Incident Reports on remote accounts in rdlist rooms.")
+	print("132) Block/Purge all rdlist rooms.")
 	print("\nPlease enter a number from the above menu, or enter 'q' or 'e' to exit.\n")
 	menu_input = input()
 	if menu_input == "1":
@@ -92,7 +113,7 @@ while pass_token == False:
 	elif menu_input == "7":
 		user_commands.whois_multiple_accounts()
 	elif menu_input == "8":
-		query_account_dict = user_commands.query_account()
+		query_account_dict = user_commands.query_account('')
 		print(json.dumps(query_account_dict, indent=4, sort_keys=True))
 	elif menu_input == "9":
 		user_commands.query_multiple_accounts()
@@ -126,6 +147,9 @@ while pass_token == False:
 	elif menu_input == "20":
 		shadow_ban_dict = user_commands.shadow_ban_account('')
 		print(json.dumps(shadow_ban_dict, indent=4, sort_keys=True))
+	elif menu_input == "21":
+		user_dict = user_commands.find_account_with_threepid()
+		print(f"\n{json.dumps(user_dict, indent=4, sort_keys=True)}")
 	elif menu_input == "50":
 		room_details_dict = room_commands.get_room_details('')
 		print(json.dumps(room_details_dict, indent=4, sort_keys=True))
@@ -179,13 +203,26 @@ while pass_token == False:
 		server_commands.purge_remote_media_repo()
 	elif menu_input == "102":
 		server_commands.prepare_database_copy_of_multiple_rooms()
+	elif menu_input == "103":
+		reported_events = server_commands.get_reported_events(10)
+		print(json.dumps(reported_events, indent=4, sort_keys=True))
+	elif menu_input == "104":
+		all_reported_events = server_commands.paginate_reported_events()  # Again assuming default values are set
+		print(json.dumps(all_reported_events, indent=4, sort_keys=True))
+	elif menu_input == "105":
+		report_details = server_commands.get_event_report_details()
+		print(json.dumps(report_details, indent=4, sort_keys=True))
 	elif menu_input == "120":
 		rdlist_commands.block_all_rooms_with_rdlist_tags(False,'','','')
 	elif menu_input == "121":
-		rdlist_commands.block_recommended_rdlist_tags()
-	elif menu_input == "122":
 		rdlist_tags = rdlist_commands.get_rdlist_tags('')
 		print(json.dumps(rdlist_tags, indent=4, sort_keys=True))
+	elif menu_input == "130":
+		rdlist_commands.collect_user_reports_on_rdlist_accounts()
+	elif menu_input == "131":
+		rdlist_commands.send_incident_reports_on_rdlist_accounts()
+	elif menu_input == "132":
+		rdlist_commands.block_recommended_rdlist_tags()
 	elif menu_input == "140":
 		ipinfo_commands.analyse_account_ip('')
 	elif menu_input == "141":

rdlist_commands.py

@@ -3,37 +3,35 @@ import os
 import subprocess
 import json
 import time
+import asyncio
 import user_commands
 import room_commands
 import report_commands
 import hardcoded_variables
 
+def testing_mode_warning():
+	print("\nWARNING! Testing mode is enabled, this will:\n\n- Reduce the amount of data collected in user reports.\n- Slow down rdlist blocking/purging.\n- Prevent the deactivation of accounts.\n- Send incident reports to yourself instead of other homeserver admins.\n")
+
 def sync_rdlist():
-	rdlist_dir = "./rdlist"
+    rdlist_dir = os.path.expanduser(hardcoded_variables.rdlist_dir)
     os.makedirs(rdlist_dir, exist_ok=True)
     # Check if the rdlist repo has already been cloned
-	if os.path.isdir("./rdlist/.git"):
+    if os.path.isdir(os.path.join(rdlist_dir, ".git")):
         print("\nrdlist repo already cloned...")
-		os.chdir("./rdlist/")
         # Update git remote references and get status
-		subprocess.run(["git", "remote", "update"], check=True)
+        subprocess.run(["git", "-C", rdlist_dir, "remote", "update"], check=True)
-		status = subprocess.run(["git", "status", "-uno"], stdout=subprocess.PIPE, check=True)
+        status = subprocess.run(["git", "-C", rdlist_dir, "status", "-uno"], stdout=subprocess.PIPE, check=True)
-		os.chdir("..")
 
         # If "Your branch is up to date" is not in the status, then there are changes to pull
         if "Your branch is up to date" not in status.stdout.decode():
             print("Pulling latest changes from rdlist repo...")
-			os.chdir("./rdlist/")
+            subprocess.run(["git", "-C", rdlist_dir, "pull"], check=True)
-			# Avoid pulling changes if testing mode is enabled
-			if hardcoded_variables.testing_mode == False:
-				subprocess.run(["git", "pull"], check=True)
-			os.chdir("..")
         else:
             print("rdlist repo is up-to-date, no need to pull changes.")
 
     else:
         print("Cloning rdlist repo...")
-		subprocess.run(["git", "clone", "ssh://gitea@code.glowers.club:1488/loj/rdlist.git"], check=True)
+        subprocess.run(["git", "clone", "ssh://gitea@code.glowers.club:1488/loj/rdlist.git", rdlist_dir], check=True)
 
 # A function to return the rdlist tags associated with a room
 def get_rdlist_tags(preset_internal_ID):
@@ -42,11 +40,12 @@ def get_rdlist_tags(preset_internal_ID):
 	elif preset_internal_ID != '':
 		internal_ID = preset_internal_ID
 
-	# Git clone the rdlist repo to ./rdlist/
+	# Git clone the rdlist repo to specified directory
 	sync_rdlist()
 
-	# Load the summaries JSON file
+	# Expand the user in the path and load the summaries JSON file
-	summaries_path = os.path.join("rdlist", "dist", "summaries.json")
+	summaries_dir = os.path.expanduser(hardcoded_variables.rdlist_dir)
+	summaries_path = os.path.join(summaries_dir, "dist", "summaries.json")
 	with open(summaries_path, 'r') as file:
 		data = json.load(file)
 
@@ -58,32 +57,10 @@ def get_rdlist_tags(preset_internal_ID):
 
 	return None
 
-def block_all_rooms_with_rdlist_tags(rdlist_use_recommended,preset_user_ID,preset_new_room_name,preset_message):
+def get_key_rdlist_info(rdlist_tags):
-	# Git clone the rdlist repo to ./rdlist/
+	# Expand the user in the path and load the summaries JSON file
-	sync_rdlist()
+	summaries_dir = os.path.expanduser(hardcoded_variables.rdlist_dir)
-
+	summaries_path = os.path.join(summaries_dir, "dist", "summaries.json")
-	if rdlist_use_recommended == True:
-		# Use the hardcoded recommended tags
-		blocked_tags = hardcoded_variables.rdlist_recommended_tags
-		print(f"\nUsing recommended rdlist tags. Rooms matching the following tags will be purged and/or blocked:\n{hardcoded_variables.rdlist_recommended_tags}")
-
-	elif rdlist_use_recommended == False:
-		# After the git repo has been cloned/pulled, open the file and read it into a string
-		with open(os.path.join("rdlist", "lib", "docs", "tags.md"), 'r') as file:
-			data = file.readlines()
-
-		# Print ./rdlist/lib/docs/tags.md README file for the user
-		print("\nPrinting details about the current tags in rdlist:\n")
-		for line in data:
-			print(line, end='')  # Print the contents of the file
-
-		# Take input from the user and convert it to a list
-		print("\nPlease enter a space seperated list of tags you wish to block:\n")
-		blocked_tags = input().split()
-		print('')
-
-	# Load the summaries JSON file
-	summaries_path = os.path.join("rdlist", "dist", "summaries.json")
 	with open(summaries_path, 'r') as file:
 		data = json.load(file)
 
@@ -97,8 +74,9 @@ def block_all_rooms_with_rdlist_tags(rdlist_use_recommended,preset_user_ID,prese
 	# Create a dictionary to store the tags for each room
 	room_tags = dict()
 
-	# Iterate over blocked_tags
+	print("\nCalculating local and remote users in rdlist rooms... (This may take a while, please wait.)")
-	for tag in blocked_tags:
+	# Iterate over the provided rdlist_tags
+	for tag in rdlist_tags:
 		# Filter the data to keep only the entries where the tag appears in the "tags" list
 		filtered_data = [item for item in data if 'report_info' in item and 'tags' in item['report_info'] and tag in item['report_info']['tags']]
 
@@ -131,25 +109,97 @@ def block_all_rooms_with_rdlist_tags(rdlist_use_recommended,preset_user_ID,prese
 
 	all_room_ids = list(all_room_ids) # convert the set to a list
 
-	#print(f"all_local_users: {all_local_users}")
+	return all_room_ids, all_local_users, all_remote_users
-	#print(f"all_remote_users: {all_remote_users}")
+
-	#print(f"all_room_ids: {all_room_ids}")
+def collect_user_reports_on_rdlist_accounts(all_local_users=None, skip_input=False):
+	# Print warning if testing mode is enabled
+	if hardcoded_variables.testing_mode == True:
+		testing_mode_warning()
+
+	# If all_local_users is None, then we need to generate it
+	if all_local_users == None:
+		# Git clone the rdlist repo to specified directory
+		sync_rdlist()
+		all_room_ids, all_local_users, all_remote_users = get_key_rdlist_info(hardcoded_variables.rdlist_recommended_tags)
 
 	# If there's at least 1 local user detected, ask the admin if they want to generate a user report for every user found in rdlist rooms
 	if len(all_local_users) > 0:
 		print(f"\nWARNING! The following local users are current members of rooms tagged in rdlist: {list(all_local_users.keys())}")
+		if skip_input == False:
 			generate_user_report_confirmation = input("\nDo you want to generate a user report file for each of these users? y/n? ")
-		if generate_user_report_confirmation.lower() in ['y', 'yes', 'Y', 'Yes']:
+		if generate_user_report_confirmation.lower() in ['y', 'yes', 'Y', 'Yes'] or skip_input == True:
 			for user_id in all_local_users:
 				# Generate report_dict for each user
 				report_content = report_commands.generate_rdlist_report_summary(all_local_users[user_id], user_id)
 				report_commands.generate_user_report(user_id, report_content)
 		elif generate_user_report_confirmation.lower() in ['n', 'no', 'N', 'No']:
-			print("\nSkipping user report generation...\n")
+			print("\nSkipping user report generation...")
 	elif len(all_local_users) == 0:
 		print(f"\nNo local users were found in rdlist rooms.")
 
-	# Todo: Add Incident Report section
+def send_incident_reports_on_rdlist_accounts(all_remote_users=None, skip_input=False):
+	# Print warning if testing mode is enabled
+	if hardcoded_variables.testing_mode == True:
+		testing_mode_warning()
+
+	# If all_remote_users is None, then we need to generate it
+	if all_remote_users == None:
+		# Git clone the rdlist repo to specified directory
+		sync_rdlist()
+		all_room_ids, all_local_users, all_remote_users = get_key_rdlist_info(hardcoded_variables.rdlist_recommended_tags)
+
+	# If there's at least 1 remote user detected, ask the admin if they want to generate a user report for every user found in rdlist rooms
+	if len(all_remote_users) > 0:
+		print(f"\nWARNING! The following remote users are current members of rooms tagged in rdlist: {list(all_remote_users.keys())}")
+		if skip_input == False:
+			send_incident_report_confirmation = input("\nDo you want to send out incident reports for these users to every homeserver admin involved? y/n? ")
+		if send_incident_report_confirmation.lower() in ['y', 'yes', 'Y', 'Yes'] or skip_input == True:
+			loop = asyncio.get_event_loop()
+			loop.run_until_complete(report_commands.send_incident_reports(all_remote_users))
+		elif send_incident_report_confirmation.lower() in ['n', 'no', 'N', 'No']:
+			print("\nSkipping incident report generation...")
+	elif len(all_remote_users) == 0:
+		print(f"\nNo remote users were found in rdlist rooms.")
+
+def block_all_rooms_with_rdlist_tags(rdlist_use_recommended,preset_user_ID,preset_new_room_name,preset_message):
+	# Git clone the rdlist repo to specified directory
+	sync_rdlist()
+
+	if rdlist_use_recommended == True:
+		# Use the hardcoded recommended tags
+		rdlist_tags = hardcoded_variables.rdlist_recommended_tags
+		print(f"\nUsing recommended rdlist tags. Rooms matching the following tags will be purged and/or blocked:\n{hardcoded_variables.rdlist_recommended_tags}")
+
+	elif rdlist_use_recommended == False:
+		# Expand the user in the path and read the file into a string
+		rdlist_dir = os.path.expanduser(hardcoded_variables.rdlist_dir)
+		rdlist_path = os.path.join(rdlist_dir, "lib", "docs", "tags.md")
+		with open(rdlist_path, 'r') as file:
+			data = file.readlines()
+
+		# Print rdlist/lib/docs/tags.md README file for the user
+		print("\nPrinting details about the current tags in rdlist:\n")
+		for line in data:
+			print(line, end='')  # Print the contents of the file
+
+		# Take input from the user and convert it to a list
+		print("\nPlease enter a space seperated list of tags you wish to block:\n")
+		rdlist_tags = input().split()
+		print('')
+
+	all_room_ids, all_local_users, all_remote_users = get_key_rdlist_info(rdlist_tags)
+
+	#print(f"\nDEBUG all_local_users: {all_local_users}")
+	#print(f"DEBUG all_remote_users: {all_remote_users}")
+	#print(f"DEBUG all_room_ids: {all_room_ids}")
+
+	# If there's at least 1 local user detected, ask the admin if they want to generate a user report for every user found in rdlist rooms
+
+	collect_user_reports_on_rdlist_accounts(all_local_users, False)
+
+	# If there's at least 1 remote user detected, ask the admin if they want to generate a incident report for every user homeserver involved in rdlist rooms
+
+	send_incident_reports_on_rdlist_accounts(all_remote_users, False)
 
 	# Ask the user if they wish to block and purge all these rooms, then collect shutdown parameters
 	if preset_user_ID == '':
@@ -206,10 +256,10 @@ def block_all_rooms_with_rdlist_tags(rdlist_use_recommended,preset_user_ID,prese
 					time.sleep(5)
 	elif shutdown_confirmation.lower() in ['n', 'no', 'N', 'No']:
 		print("\nSkipping blocking/shutdown of rooms...\n")
-		return
+		return 0, 0, []
 	else:
 		print("\nInvalid input, skipping these files...\n")
-		return
+		return 0, 0, []
 
 	# Deduplicate the list of all kicked users
 	total_list_kicked_users = list(set(total_list_kicked_users))
@@ -220,7 +270,7 @@ def block_all_rooms_with_rdlist_tags(rdlist_use_recommended,preset_user_ID,prese
 def block_recommended_rdlist_tags():
 	# Print warning if testing mode is enabled
 	if hardcoded_variables.testing_mode == True:
-		print("\nWARNING! Testing mode is enabled, this will reduce the amount of data generated in reports and greatly slow down rdlist blocking!\n")
+		testing_mode_warning()
 
 	# Check if user account already exists
 	account_query = user_commands.query_account(hardcoded_variables.rdlist_bot_username)
@@ -230,7 +280,7 @@ def block_recommended_rdlist_tags():
 		# Create user account
 		user_commands.create_account(hardcoded_variables.rdlist_bot_username, hardcoded_variables.rdlist_bot_password)
 	else:
-		print(f"@{hardcoded_variables.rdlist_bot_username}:{hardcoded_variables.base_url} account already exists. Resetting account password.\n")
+		print(f"@{hardcoded_variables.rdlist_bot_username}:{hardcoded_variables.base_url} account already exists. Resetting account password.")
 		user_commands.reset_password(hardcoded_variables.rdlist_bot_username, hardcoded_variables.rdlist_bot_password)
 
 	# Define default valies for shutdown_room()
@@ -240,16 +290,18 @@ def block_recommended_rdlist_tags():
 	# Block all rooms with recommended tag set
 	num_rooms_blocked, num_rooms_purged, total_list_kicked_users = block_all_rooms_with_rdlist_tags(True, hardcoded_variables.rdlist_bot_username, preset_new_room_name, preset_message)
 
-	# Print user login details
+	# Print user login details if any rooms were shutdown
+	if total_list_kicked_users != []:
 		print("\n\nRoom shutdowns completed!\n\nUser login details for your moderator account:\n")
 		print("Username: " + hardcoded_variables.rdlist_bot_username)
 		print("Password: " + hardcoded_variables.rdlist_bot_password)
 
 	# Print statistics for the admin
-	print(f"\nPrint rdlist statistics:")
+	print(f"\nPrinting rdlist statistics:")
 	print(f"\nNumber of rooms blocked: {num_rooms_blocked}")
 	print(f"Number of rooms purged: {num_rooms_purged}")
 	print(f"Number of local users located in rdlist rooms and kicked: {len(total_list_kicked_users)}")
+	if total_list_kicked_users != []:
 		print(f"\nThe following users were current members of rooms tagged in rdlist: {total_list_kicked_users}")
 
 	# Ask admin if they want to deactivate all the accounts that were kicked from rdlist rooms

report_commands.py

@@ -59,6 +59,9 @@ remove it from your electronic mailbox.
 \n**********************************************************************
 """
 
+def testing_mode_warning():
+	print("\nWARNING! Testing mode is enabled, this will:\n\n- Reduce the amount of data collected in user reports.\n- Slow down rdlist blocking/purging.\n- Prevent the deactivation of accounts.\n- Send incident reports to yourself instead of other homeserver admins.\n")
+
 def get_report_folder():
 	# Get report_folder from hardcoded_variables
 	report_folder = hardcoded_variables.report_folder
@@ -85,6 +88,10 @@ def zip_report_folder(user_report_folder, username):
 	return zip_file_name
 
 def generate_user_report(preset_username, report_details):
+	# Print warning if testing mode is enabled
+	if hardcoded_variables.testing_mode == True:
+		testing_mode_warning()
+
 	if len(preset_username) == 0:
 		username = input("\nPlease enter the username to automatically generate a report: ")
 		username = user_commands.parse_username(username)
@@ -221,7 +228,7 @@ def generate_user_report(preset_username, report_details):
 	zip_file_size = os.path.getsize(zip_file_name) / 1000000
 
 	# Print the password and the encrypted .zip file name
-	print("Report generated successfully on user: \"" + username + "\"\n\nYou can send this .zip file when reporting a user to law enforcement.")
+	print("Report generated successfully on user: \"" + username + "\"\n\nYou can send this .zip file when reporting a user to law enforcement.\n")
 	print(".zip file location: " + zip_file_name)
 	print(".zip file size: " + str(zip_file_size) + " MB\n")
 
@@ -237,8 +244,8 @@ def lookup_homeserver_admin(preset_baseurl):
 
 	# If baseurl is matrix.org, return 'abuse@matrix.org' as a hardcoded response
 	if baseurl == "matrix.org":
-		print("\nAdmin contact email(s) for " + baseurl + " are: abuse@matrix.org")
+		#print("\nAdmin contact email(s) for " + baseurl + " are: abuse@matrix.org")
-		return {"admins": {"email_address": "abuse@matrix.org"}}, False
+		return {"admins": [{"email_address": "abuse@matrix.org"}]}, False
 
 	# Check target homserver for MSC1929 support email
 	url = f"https://{baseurl}/.well-known/matrix/support"
@@ -251,21 +258,28 @@ def lookup_homeserver_admin(preset_baseurl):
 	# If the request was successful, the status code will be 200
 	if response.status_code == 200 and ( "email_address" in response.text or "matrix_id" in response.text ):
 		# Parse the response as JSON
+		print(response.text)
 		data = json.loads(response.text)
 
 		#print("\nAdmin contact details for " + baseurl + " are: " + str(data))
 
 		return data, False
 	else:
-		print(f"Error: Unable to collect admin contact details from server {baseurl}")
+		print(f"\nError: Unable to collect admin contact details from server {baseurl}")
 		print("Attempting to collect admin email from WHOIS data...")
 
 		# Get WHOIS data
 		try:
 			w = whois.whois(baseurl)
 			if w.emails:
-				#print("\nAdmin contact email(s) for " + baseurl + " are: " + str(w.emails))
+				# Check if the emails field is a list
-				return {baseurl: list(w.emails)}, True
+				if isinstance(w.emails, list):
+					# Create a list of dictionaries, each containing one email address
+					emails_dict_list = [{"email_address": email} for email in w.emails]
+					return {"admins": emails_dict_list}, True
+				# If it's not a list, it must be a single string. So, we wrap it in a list
+				else:
+					return {"admins": [{"email_address": w.emails}]}, True
 			else:
 				print(f"Error: Unable to collect admin email from WHOIS data for {baseurl}")
 				return None, False
@@ -421,7 +435,7 @@ https://{hardcoded_variables.base_url}
 
 	return message_content
 
-async def send_incident_report(incidents_dict):
+async def send_incident_reports(incidents_dict):
 	success = True
 	homeserver_dict = {}
 
@@ -451,7 +465,7 @@ async def send_incident_report(incidents_dict):
 				message_content = prepare_message_content(user_dict, baseurl)
 
 				try:
-					print(f"Sending Incident Report message to {admin['matrix_id']}")
+					print(f"\nSending Incident Report for users from {baseurl} to {admin['matrix_id']}")
 					await bot_commands.send_message(admin["matrix_id"], message_content)
 				except Exception as e:
 					print(f"Failed to send message to {admin['matrix_id']}: {str(e)}")
@@ -464,7 +478,7 @@ async def send_incident_report(incidents_dict):
 					email_content = prepare_email_content(user_dict, from_whois, baseurl)
 
 					email_attachments = []
-					print(f"Sending Incident Report email to {email_address}")
+					print(f"Sending Incident Report for users from {baseurl} to {admin['email_address']}")
 					if not send_email(email_address, email_subject, email_content, email_attachments):
 						print(f"Failed to send email to {email_address}")
 						success = False
@@ -498,7 +512,7 @@ def test_send_incident_reports():
 	try:
 		if hardcoded_variables.testing_mode == True:
 			print("\nNOTE: Testing mode is enabled, sending Incident Reports to you! :)\n")
-		if asyncio.run(send_incident_report(incidents_dict)):
+		if asyncio.run(send_incident_reports(incidents_dict)):
 			print("\nIncident reports successfully sent.")
 		else:
 			print("\nFailed to send the incident reports.")

server_commands.py

@@ -136,3 +136,70 @@ def prepare_database_copy_of_multiple_rooms():
 	print(chown_command_process.stdout)
 
 	print("\nThe sql query files have been generated, as postgres user in container run:\n# docker exec -it matrix-postgres /bin/bash\nbash-5.0$  export PGPASSWORD=your-db-password\nbash-5.0$ for f in /var/lib/postgresql/data/ramdisk/*/dump_room_data.sql; do psql --host=127.0.0.1 --port=5432 --username=synapse -w -f $f; done\n\nAfter copying the data to a cloud location law enforcement can access, clean up the ramdisk like so:\n# rm -r /matrix/postgres/data/ramdisk/*\n# umount /matrix/postgres/data/ramdisk")
+
+def get_reported_events(limit=100, _from=0, dir='b', user_id=None, room_id=None):
+    url = f"https://{hardcoded_variables.homeserver_url}/_synapse/admin/v1/event_reports"
+
+    headers = {
+        "Content-Type": "application/json",
+        "Authorization": f"Bearer {hardcoded_variables.access_token}"
+    }
+
+    params = {
+        'limit': limit,
+        'from': _from,
+        'dir': dir
+    }
+
+    if user_id:
+        params['user_id'] = user_id
+
+    if room_id:
+        params['room_id'] = room_id
+
+    response = requests.get(url, headers=headers, params=params)
+
+    if response.status_code == 200:
+        return response.json()
+    else:
+        print(f"Error fetching reported events: {response.status_code}, {response.text}")
+        return None
+
+def paginate_reported_events(limit=100, dir='b', user_id=None, room_id=None):
+    _from = 0
+    all_reports = []
+
+    while True:
+        reports = get_reported_events(limit=limit, _from=_from, dir=dir, user_id=user_id, room_id=room_id)
+        if not reports or "event_reports" not in reports:
+            break
+
+        all_reports.extend(reports["event_reports"])
+
+        if "next_token" in reports:
+            _from = reports["next_token"]
+        else:
+            break
+
+    return all_reports
+
+def get_event_report_details(preset_report_id=''):
+    if preset_report_id == '':
+        report_id = input("\nEnter the report_id of the report you wish to query (Example: 56): ")
+    elif preset_report_id != '':
+        report_id = preset_report_id
+
+    url = f"https://{hardcoded_variables.homeserver_url}/_synapse/admin/v1/event_reports/{report_id}"
+
+    headers = {
+        "Content-Type": "application/json",
+        "Authorization": f"Bearer {hardcoded_variables.access_token}"
+    }
+
+    response = requests.get(url, headers=headers)
+
+    if response.status_code == 200:
+        return response.json()
+    else:
+        print(f"Error fetching event report details: {response.status_code}, {response.text}")
+        return None

user_commands.py

@@ -432,6 +432,7 @@ def list_account_pushers(preset_username):
 
 def get_rate_limit():
 	username = input("\nPlease enter the username to get its ratelimiting: ")
+	username = parse_username(username)
 
 	url = f"https://{hardcoded_variables.homeserver_url}/_synapse/admin/v1/users/@{username}:{hardcoded_variables.base_url}/override_ratelimit?access_token={hardcoded_variables.access_token}"
 
@@ -450,6 +451,7 @@ def get_rate_limit():
 
 def set_rate_limit():
 	username = input("\nPlease enter the username to adjust its ratelimiting: ")
+	username = parse_username(username)
 
 	messages_per_second = input("\nPlease enter the desired messages per second: ")
 	burst_count = input("\nPlease enter the desired burst count: ")
@@ -478,6 +480,7 @@ def set_rate_limit():
 
 def delete_rate_limit():
 	username = input("\nPlease enter the username to delete its ratelimiting: ")
+	username = parse_username(username)
 
 	url = f"https://{hardcoded_variables.homeserver_url}/_synapse/admin/v1/users/@{username}:{hardcoded_variables.base_url}/override_ratelimit?access_token={hardcoded_variables.access_token}"
 
@@ -537,3 +540,28 @@ def shadow_ban_account(preset_username):
 
 # Example:
 # curl -XPOST -H "Content-Type: application/json" 'https://matrix.perthchat.org/_synapse/admin/v1/users/@dogpoo:perthchat.org/shadow_ban?access_token=ACCESS_TOKEN'
+
+def find_account_with_threepid(medium="", address=""):
+    # prompt user to enter values if they're not provided
+    if medium == "":
+        print("\nPlease enter the medium (either 'email' or 'msisdn' for mobile number): ")
+        medium = input()
+    if address == "":
+        print("\nPlease enter the address (the email or mobile number): ")
+        address = input()
+
+    url = f"https://{hardcoded_variables.homeserver_url}/_synapse/admin/v1/threepid/{medium}/users/{address}?access_token={hardcoded_variables.access_token}"
+
+    response = requests.get(url, verify=True)
+
+    if response.status_code == 200:
+        # User exists
+        return response.json()
+    elif response.status_code == 404:
+        # User not found
+        return {"errcode":"M_NOT_FOUND", "error":"User not found"}
+    else:
+        print(f"Error querying account: {response.status_code}, {response.text}")
+
+# Example:
+# $ curl -X GET 'https://matrix.perthchat.org/_synapse/admin/v1/threepid/email/users/dogpoo@protonmail.com?access_token=ACCESS_TOKEN'