108

New rule on Aggregators/Forwarders: (lemmy.world)

submitted 1 day ago by jordanlund@lemmy.world to c/world@lemmy.world

142 comments fedilink hide all child comments

We've had some trouble recently with posts from aggregator links like Google Amp, MSN, and Yahoo.

We're now requiring links go to the OG source, and not a conduit.

In an example like this, it can give the wrong attribution to the MBFC bot, and can give a more or less reliable rating than the original source, but it also makes it harder to run down duplicates.

So anything not linked to the original source, but is stuck on Google Amp, MSN, Yahoo, etc. will be removed.

you are viewing a single comment's thread
view the rest of the comments

[-] PhilipTheBucket@ponder.cat 8 points 17 hours ago

Here you go:

https://ponder.cat/wp/wp-sources.zip

It's in python, suitable for sticking directly into the bot if the bot is in python. There are docs. It's a first cut. How did you envision this working? I can make a real API, if for some reason that makes things easier, but it's not immediately obvious how it would get integrated into things.

Running it on the last 50 articles posted to /c/politics, we see:

https://lemmy.world/post/20739836: Source is unreliable since ownership change
https://lemmy.world/post/20736298: Source is unreliable for political topics since 2011
https://lemmy.world/post/20724155: Reliability consensus is mixed
https://lemmy.world/post/20723675: Source is unreliable
https://lemmy.world/post/20722912: Source is unreliable
https://lemmy.world/post/20722910: Reliability consensus is mixed
https://lemmy.world/post/20716118: Reliability consensus is mixed
https://slrpnk.net/post/14127964: Reliability consensus is mixed

It's more complex to use this than MBFC, because there's a lot more depth to the rankings, and sometimes human judgement is needed to assign scores. There's a category "needinfo," meaning it's necessary to know what topic is being discussed or when an article was written, because of an ownership change or similar factor. I've applied that judgement above. That, to me, is a good thing. It means the bot is grounded in something, and not just blithely spitting out arbitrary scores without bothering to ground them in any reality.

In practice, I think it would be realistic to assign a single reliability ranking to most of the "needinfo" sources. You can manually edit the .json data to do so. Almost all of the posts are going to fit into one of Wikipedia's categorizations or another. Newsweek is unreliable, The Guardian is reliable, and so on.

I think most of the mixed-consensus sources can be used without a second thought. Mostly, the questions about them boil down to open partisanship of the source, which for a political community is perfectly fine as long as they're trustable factually.

If you want me to boil this down further, so that it gives a single "yes" or "no" score to each source, I can do that and probably keep almost all of the accuracy of the rankings, now that I've looked at it for a little while.

When you talk about "adding" this to the bot, are you proposing to still have MBFC be the main source, with this as a footnote? A lot of the criticism of the bot is on the grounds that MBFC is a very bad source for judging reliability, so I would question the idea of keeping it on as the primary source.

[-] Rooki@lemmy.world 3 points 16 hours ago

By "adding" i mean adding it into the field higher than MBFC ( as i personally think wikipedia is a little bit better for that ).

new:

Wikipedia: Reliability consensus is mixed.....l ( whatever the scrapper scrapes ) MBFC: Right-Center - Credibility: High - Factual Reporting: Mostly Factual - United States of America
Search Wikipedia about this source

I would like to implement your code into the bot myself so i can learn how you would do it. If you are willing to share your code, please send me a github link ( or invite me if you want it to be private between you and me ) or if its super simple just send it in the dms.

[-] PhilipTheBucket@ponder.cat 4 points 16 hours ago* (last edited 16 hours ago)

I already sent it. It's here:

https://ponder.cat/wp/wp-sources.zip

Edit: You don't need to do the import initially, since there's already a sources file with some small modifications. The import is the only complicated part. Use categorize.py to categorize a source, or lookup.py to run a quick command-line test.

[-] Rooki@lemmy.world 2 points 16 hours ago

Ok i will look into it, thanks i thought it was just the sources not the code.

this post was submitted on 10 Oct 2024

108 points (85.1% liked)

World News

38797 readers

2075 users here now

A community for discussing events around the World

Rules:

Rule 1: posts have the following requirements:
- Post news articles only
- Video links are NOT articles and will be removed.
- Title must match the article headline
- Not United States Internal News
- Recent (Past 30 Days)
- Screenshots/links to other social media sites (Twitter/X/Facebook/Youtube/reddit, etc.) are explicitly forbidden, as are link shorteners.
Rule 2: Do not copy the entire article into your post. The key points in 1-2 paragraphs is allowed (even encouraged!), but large segments of articles posted in the body will result in the post being removed. If you have to stop and think "Is this fair use?", it probably isn't. Archive links, especially the ones created on link submission, are absolutely allowed but those that avoid paywalls are not.
Rule 3: Opinions articles, or Articles based on misinformation/propaganda may be removed. Sources that have a Low or Very Low factual reporting rating or MBFC Credibility Rating may be removed.
Rule 4: Posts or comments that are homophobic, transphobic, racist, sexist, anti-religious, or ableist will be removed. “Ironic” prejudice is just prejudiced.
Posts and comments must abide by the lemmy.world terms of service UPDATED AS OF 10/19
Rule 5: Keep it civil. It's OK to say the subject of an article is behaving like a (pejorative, pejorative). It's NOT OK to say another USER is (pejorative). Strong language is fine, just not directed at other members. Engage in good-faith and with respect! This includes accusing another user of being a bot or paid actor. Trolling is uncivil and is grounds for removal and/or a community ban.
Rule 6: Memes, spam, other low effort posting, reposts, misinformation, advocating violence, off-topic, trolling, offensive, regarding the moderators or meta in content may be removed at any time.
Rule 7: We didn't USED to need a rule about how many posts one could make in a day, then someone posted NINETEEN articles in a single day. Not comments, FULL ARTICLES. If you're posting more than say, 10 or so, consider going outside and touching grass. We reserve the right to limit over-posting so a single user does not dominate the front page.

We ask that the users report any comment or post that violate the rules, to use critical thinking when reading, posting or commenting. Users that post off-topic spam, advocate violence, have multiple comments or posts removed, weaponize reports or violate the code of conduct will be banned.

All posts and comments will be reviewed on a case-by-case basis. This means that some content that violates the rules may be allowed, while other content that does not violate the rules may be removed. The moderators retain the right to remove any content and ban users.