I think there are two separate things I want to address here:

First, agile isn’t a project management methodology, it’s just a set of 4 abstract priorities and 12 abstract principles. It’s very short, you can check it out here:

https://agilemanifesto.org/

Nothing here says that you’re not allowed to write documentation, write down requirements, etc. In fact, the principles encourage you yourself as a software team to create the exact processes and documentation that you need in order to meet your goals.

“Working software over comprehensive documentation” does not mean you aren’t allowed to have documentation, it just means that you should only write documentation if it helps you build working software, rather than writing documentation for the sake of bureaucracy.

“Individuals and interactions over processes and tools” does not mean that you should have no processes, it just means that the individuals in your team should be empowered to collaboratively create whatever processes you need to deliver good software.

Secondly, in terms of practical advice:

1. Talk about this problem with your team. Is it hard for others to figure out where requirements came from? Maybe they already have a good method and can share it with you. If it’s hard for everybody, then propose improvements to your process, for example, propose some type of design document process as part of building any new features
2. There are no perfect answers to the question of “how do I safely make non-trivial changes to systems”, but the general approach is to ensure that:

a. You have metrics about how your system is used.

b. You have automated tests covering any requirements, so that you can feel confident when making changes to one part of the system that it isn’t violating any unrelated requirements.

c. You actually document any confusing parts in the code itself using comments. The most important thing to cover in comments is “why is this logic necessary?” - whenever something is confusing, you need to answer this question with a comment. Otherwise, the system becomes very annoying to change later on.

If you are missing any of the above, then propose to your team that you start doing it ASAP

3. At the end of the day, somebody is responsible for making product decisions. Is it your team? Or maybe some separate product owner? Sometimes, you just need to communicate with whoever is responsible to figure out if any requirements are still relevant, or if they are now safe to change.

If I have several backends that more or less depend on each other anyway (for example: Lemmy + pict-rs), then I will create separate databases for them within a single postgres - reason being, if something bad happens to the database for one of them, then it affects the other one as well anyway, so there isn’t much to gain from isolating the databases.

Conversely, for completely unrelated services, I will always set up separate postgres instances, for full isolation.

That particular instance was very recently the source of a lot of CSAM and spam, so that’d be why. A lot of instances recently upped their security to combat that.

Just to add some more context, there was an attacker recently who created accounts on several Lemmy instances and used those accounts to spread CSAM. On lemm.ee, this attacker created 4 accounts over a 24h period, but was not able to upload any CSAM to our servers due to our stricter upload rules (we require 4 week old accounts to upload any images at all), and all of the 4 accounts were removed very shortly after creation (most of them within an hour of signing up). The attacker gave up trying to use lemm.ee very quickly, and moved on to other instances.

I just wanted to share this context to illustrate that while indeed the different measures we implement to protect the instance can have a negative impact on legitimate users, I really believe that overall, they have a net positive effect. In addition to Cloudflare DDoS protection and image upload restrictions, we also have a separate content-based alerting layer on top of Lemmy, which allows our admins to quickly notice when something suspicious is going on. As another example, this alerting has allowed us to extremely efficiently deal with a current ongoing spam attack on the Fediverse, and I bet many lemm.ee users aren’t even aware of this attack due to the quick content removal. We will continue to improve our defenses, and hopefully try to limit the impact on real users as much as possible, but some trade-offs are necessary here in order to protect the overall userbase.

The nice thing about Lemmy is that you can always host your own instance, even if it’s only for your own individual use. You can basically use your own instance as a proxy - other instances will not see how or from where you are connecting to your instance.

Large instances are being attacked almost constantly at this point in smaller and bigger ways. Almost all measures we implement to combat these attacks come with some trade-offs for the rest of the userbase.

As a test, I ran this on a very early backup of lemm.ee images from when we had very little federation and very little uploads, and unfortunately it is finding a whole bunch of false positives. Just some examples it flagged as CSAM:

• Calvin and Hobbes comic
• The default Lemmy logo
• Some random user’s avatar, which is just a digital drawing of a person’s face
• a Pikachu image

Do you think the parameters of the script should be tuned? I’m happy to test it further on my backup, as I am reasonably certain that it doesn’t contain any actual CSAM

Any thoughts about using this as a middleware between nginx and Lemmy for all image uploads?

Edit: I guess that wouldn’t work for external images - unless it also ran for all outgoing requests from pict-rs… I think the easiest way to integrate this with pict-rs would be through some upstream changes that would allow pict-rs itself to call this code on every image.