167

Media upload filter updates again (lemmy.world)

submitted 6 days ago by lwadmin@lemmy.world to c/lemmyworld@lemmy.world

16 comments fedilink hide all child comments

Hello World,

today, @db0@lemmy.dbzer0.com has provided an update to the media upload scanner we're using. This should reduce the amount of false positives blocked from being uploaded. We have deployed the updated version now.

While we do not have stats about false positives from before we implemented the scan when uploading, those changes did not change the overall data availability for us. Flagged images were still deleted, they were just still served by our cache in many cases. By moving this to the upload process, it has become much more effective, as previously images could persist in Cloudflare's cache for extended periods of time, while now they won't get cached in the first place.

Over the last week, we've seen a rate of roughly 6.7% uploads rejected out of around 3,000 total uploads. We'll be able to compare numbers in a week to confirm that this has indeed improved the false positive rate.

you are viewing a single comment's thread
view the rest of the comments

[-] MrKaplan@lemmy.world 4 points 5 days ago

a JSON parsing error is certainly unexpected. I wonder if you happen to be triggering some automatic Cloudflare WAF rule.

could you try uploading the same file from a computer? before uploading, please open your browser dev tools (F12) and visit the network tab. it should show some more details in there. especially the response tab should show an actual error message. in the headers tab, you could also share the value of the cf-ray header with us and we can take a look at our logs. please be careful not to share everything you see in that network tab, as there are values visible there that allow taking over your lemmy account. specifically cookies and the jwt value; in some cases also an authorization header. cf-ray is not sensitive, it's just an identifier associated with the individual request.

feel free to pm me the cf-ray value rather than posting it publicly.

[-] IMALlama@lemmy.world 2 points 5 days ago

Thanks for the reply and apologies for getting back to you. I typed that out right before going to bed :(

Looking in developer tools, it does look like Cloudflare is what's blocking me. Here's the cf-ray value: 8f4ffe74bffc22f1-ORD

Now that I'm on a PC, I can simply open the file in an image editor, resave it without making any intentional modifications, and the upload succeeds. It makes me think that it has something to do with the image metadata (maybe EXIF?).

I'll send you a pm you a link to the unmodified image momentarily.

[-] MrKaplan@lemmy.world 4 points 5 days ago

thanks for the image. for some reason i can't see the firewall event for your attempt, but when i tried to upload the image myself i found the event. it's a rule related to a php file upload vulnerability in software we're not even using, it's disabled now.