Someone should make a community to freely distribute examples of data poisoning people can randomly put in their social media posts/images to sabotage AI

Reply to Someone should make a community to freely distribute examples of data poisoning people can randomly put in their social media posts/images to sabotage AI on Tue, 11 Aug 2026 15:02:38 GMT

chaogomu@lemmy.world — Tue, 11 Aug 2026 15:02:38 GMT

There's a new technique that uses the AIs "thinking" tags to get it to do things that are otherwise banned by policy.

I'll have to find the article again. But due to the way LLMs work, they can't defend against this sort of attack.

Reply to Someone should make a community to freely distribute examples of data poisoning people can randomly put in their social media posts/images to sabotage AI on Tue, 11 Aug 2026 14:26:03 GMT

pudutr0n@lemmy.world — Tue, 11 Aug 2026 14:26:03 GMT

I've read in papers that you can poison datasets with a very small percentage of the data, if done cleverly. I can fish up the source if you want (but it might take me some time).

edit: here it is.

We conduct the largest pretraining poisoning experiments to date, pretraining models from 600M to 13B parameters on chinchilla-optimal datasets (6B to 260B tokens). We find that 250 poisoned documents similarly compromise models across all model and dataset sizes (...)

Emphasis mine. All it takes is 250 poisoned documents.

Reply to Someone should make a community to freely distribute examples of data poisoning people can randomly put in their social media posts/images to sabotage AI on Tue, 11 Aug 2026 14:24:06 GMT

slazer2au@lemmy.world — Tue, 11 Aug 2026 14:24:06 GMT

Na, for it to be effective it needs to be wide spread, but if its wide spread then it can be filtered out of the training material.