570

A new tool lets artists add invisible changes to the pixels in their art before they upload it online so that if it’s scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways.

The tool, called Nightshade, is intended as a way to fight back against AI companies that use artists’ work to train their models without the creator’s permission.
[...]
Zhao’s team also developed Glaze, a tool that allows artists to “mask” their own personal style to prevent it from being scraped by AI companies. It works in a similar way to Nightshade: by changing the pixels of images in subtle ways that are invisible to the human eye but manipulate machine-learning models to interpret the image as something different from what it actually shows.

you are viewing a single comment's thread
view the rest of the comments
[-] MargotRobbie@lemmy.world 332 points 1 year ago

It's made by Ben Zhao? You mean the "anti AI plagerism" UChicago professor who illegally stole GPLv3 code from an open source program called DiffusionBee for his proprietary Glaze software (reddit link), and when pressed, only released the code for the "front end" while still being in violation of GPL?

The Glaze tool that promised to be invisible to the naked eyes, but contained obvious AI generated artifacts? The same Glaze that reddit defeated in like a day after release?

Don't take anything this grifter says seriously, I'm surprised he hasn't been suspended for academic integrity violation yet.

[-] ElectroVagrant@lemmy.world 51 points 1 year ago

Thanks for added background! I haven't been monitoring this area very closely so wasn't aware, but I'd have thought a publication that has been would then be more skeptical and at least mention some of this, particularly highlighting disputes over the efficacy of the Glaze software. Not to mention the others they talked to for the article.

Figures that in a space rife with grifters you'd have ones for each side.

[-] Zeth0s@lemmy.world 29 points 1 year ago* (last edited 1 year ago)

Don't worry, it is normal.

People don't understand AI. Probably all articles I have read on it by mainstream media were somehow wrong. It often feels like reading a political journalist discussing about quantum mechanics.

My rule of thumb is: always assume that the articles on AI are wrong. I know it isn't nice, but that's the sad reality. Society is not ready for AI because too few people understand AI. Even AI creators don't fully understand AI (this is why you often hear about "emergent abilities" of models, it means "we really didn't expect it and we don't understand how this happened")

[-] ElectroVagrant@lemmy.world 4 points 1 year ago* (last edited 1 year ago)

Probably all articles I have read on it by mainstream media were somehow wrong. It often feels like reading a political journalist discussing about quantum mechanics.

Yeah, I view science/tech articles from sources without a tech background this way too. I expected more from this source given that it's literally MIT Tech Review, much as I'd expect more from other tech/science-focused sources, albeit I'm aware those require scrutiny just as well (e.g. Popular Science, Nature, etc. have spotty records from what I gather).

Also regarding your last point, I'm increasingly convinced AI creators' (or at least their business execs/spokespeople) are trying to have their cake and eat it too in terms of how much they claim to not know/understand how their creations work while also promoting how effective it is. On one hand, they genuinely don't understand some of the results, but on the other, they do know enough of how it works to have an idea of how/why those results came about, however it's to their advantage to pretend they don't insofar as it may mitigate their liability/responsibility should the results lead to collateral damage/legal issues.

[-] joel_feila@lemmy.world 0 points 1 year ago

By that logic humanity isnt ready for personal computers since few understand how they work.

[-] Zeth0s@lemmy.world 6 points 1 year ago* (last edited 1 year ago)

Kind of true. Check the law proposals on encryption around the world...

Technology is difficult, most people don't understand it, result is awful laws. AI is even more difficult, because even creators don't fully understand it (see emergent behaviors, i.e. capabilities that no one expected).

Computers luckily are much easier. A random teenager knows how to build one, and what it can do. But you are right, many are not yet ready even for computers

[-] joel_feila@lemmy.world 3 points 1 year ago

I read an article the other day about managers complaining about zoomers not even knowing how type on a keyboard.

[-] GenderNeutralBro@lemmy.sdf.org 4 points 1 year ago

That was certainly true in the 90s. Mainstream journalism on computers back then was absolutely awful. I'd say that only changed in the mid-2000 or 2010s. Even today, tech literacy in journalism is pretty low outside of specialist outlets like, say, Ars.

Today I see the same thing with new tech like AI.

[-] p03locke@lemmy.dbzer0.com 31 points 1 year ago

who illegally stole GPLv3 code from an open source program called DiffusionBee for his proprietary Glaze software (reddit link), and when pressed, only released the code for the “front end” while still being in violation of GPL?

Oh, how I wish the FSF had more of their act together nowadays and were more like the EFF or ACLU.

[-] MargotRobbie@lemmy.world 28 points 1 year ago

You should check out the decompilation they did on Glaze too, apparently it's hard coded to throw out a fake error upon detecting being ran on an A100 as some sort of anti-adversarial training measure.

[-] vidarh@lemmy.stad.social 11 points 1 year ago

That's hilarious, given that if these tools become remotely popular the users of the tools will provide enough adversarial data for the training to overcome them all by itself, so there's little reason to anyone with access to A100's to bother trying - they'll either be a minor nuisance used a by a tiny number of people, or be self-defeating.

[-] Dadifer@lemmy.world 13 points 1 year ago

Thank you, Margot Robbie! I'm a big fan!

[-] MargotRobbie@lemmy.world 18 points 1 year ago

You're welcome. Bet you didn't know that I'm pretty good at tech too.

Also, that's Academy Award nominated character actress Margot Robbie to you!

this post was submitted on 23 Oct 2023
570 points (86.4% liked)

Technology

60130 readers
2751 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 2 years ago
MODERATORS