42

I've had real issues trying to search the fediverse. I've had bad luck with the search function of both Lemmy and Mbin, and while https://fedi-search.com/ exists the Whoogle server is down and either way the search just seems to be a list of various fediverse instances and nothing fancier (which also means that it's not a complete search?). Other than that it's quite the hassle to list all the instances you'd like to search for every search. What's the best way to search the fediverse? What works for you? And is it somehow possible to add a shortcut to e.g., DDG that searches specific sites without having to type for example site:lemmy.dbzer0.com and all the other instances all the time?

top 13 comments
sorted by: hot top controversial new old
[-] JupiterRowland@sh.itjust.works 1 points 14 hours ago

It's technologically impossible for any search to cover all of the Fediverse. Like, absolutely 100% of it.

That's because it's technologically impossible for anything in or outside the Fediverse to be aware of the full extent of the Fediverse and know all its instances, all its actors, all its (public) content in real-time.

It would only be possible if there was a fully centralised search engine. And that search engine had been hard-coded into all Fediverse server apps for years so that even instances that haven't been upgraded in two or three years know it.

If Joe Übergeek spun up his own personal CherryPick or (streams) or Forte instance or whatever on his own Raspi, that instance would immediately have to announce its existence to that centralised search engine. Otherwise, the search engine wouldn't have any way of knowing this new instance exists. If Joe Übergeek sent his first test post into the void because he has no connections yet, it would immediately have to be pushed to that search engine. And if Joe Übergeek decided to turn off ActivityPub on his (streams) channel, his instance would immediately have to notify the search engine which would immediately have to list that channel as formerly but no longer available.

Now imagine such a search being decentralised, e.g. built into Fediverse server apps like Mastodon or Lemmy. In this case, all server apps would have to know all instances out there with Fediverse-wide search. And immediately so.

Imagine Mastodon had such search built-in. Imagine Alice started up her own personal Mastodon instance with this search at 10:30. Imagine Bob installed his own personal (streams) instance from source at 10:31.

In order for the search on Alice's Mastodon instance to actually cover 100% of the Fediverse, it would require Bob's (streams) instance to push all necessary information to it. In order for this to work, Bob's (streams) instance would have to know of the existence of Alice's Mastodon instance from the moment it's installed.

This couldn't be done via any form of discovery, for where would (streams) go look for search instances?

So an automatically-generated list of search instances would have to be necessary. It would have to be delivered with the code upon installation.

This means that Alice's Mastodon instance would have to add itself to the list of search instances in the streams repository (https://codeberg.org/streams/streams) as a pull request and then immediately merge that PR into both dev and release, the latter past dev, both without Mike Macgirvin's permission, so that Bob's new (streams) instance knows about Alice's less-than-a-minute-old Mastodon instance with search the very moment that Bob installs it, so that Bob's (streams) instance knows that it will have to report everything that happens to it in public to Alice's Mastodon instance with built-in Fediverse search.

Whenever someone spins up a new instance that has Fediverse search built in, this would cause a PR in the code repositories of all Fediverse server applications that adds this instance to the initial list of search instances, and it'd cause that PR to immediately be merged into all active branches with no consent by the maintainers. And each shutdown of an instance with Fediverse search would cause a PR and an automated merge because that instance would have to be removed from the initial list of search instances.

I guess it should be obvious what an outlandish idea this is.

[-] Bishma@discuss.tchncs.de 11 points 2 days ago

Since I already pay for Kagi, I use their "Fediverse Forums" lens.

[-] Yingwu@lemmy.dbzer0.com 5 points 2 days ago

Kagi has definitely been on my radar, but it sounds a bit weird (for lack of a better word) to pay for search. I've never even thought about it. Maybe it's worth it...

[-] Danterious@lemmy.dbzer0.com 7 points 2 days ago

Most searxng instances have a similar lens for lemmy comments so you can do that too if you want an open source alternative.

~Anti~ ~Commercial-AI~ ~license~ ~(CC~ ~BY-NC-SA~ ~4.0)~

[-] Yingwu@lemmy.dbzer0.com 1 points 1 day ago

Thanks, I definitely prefer this alternative. How does searxng work in general?

[-] Danterious@lemmy.dbzer0.com 0 points 1 day ago

It is a meta-search engine so it takes results from other search engines and shows the results. Usually you can decide which search engines to use in preferences. You can host it yourself or find an online instance to use.

~Anti~ ~Commercial-AI~ ~license~ ~(CC~ ~BY-NC-SA~ ~4.0)~

[-] Yingwu@lemmy.dbzer0.com 1 points 1 day ago* (last edited 1 day ago)

It seems really great actually. Thanks for the rec. Might selfhost it. But I also get the argument of Kagi if one doesn't want to rely on the search results of Google/Bing/DDG or whatever. Even if I definitely prefer FOSS.

[-] Danterious@lemmy.dbzer0.com 2 points 1 day ago

Kagi doesn't really have its own index either. It mainly relies on other search engines as well and the indexes that are its own that focus on small web stuff is better done by marginalia.nu which is also open source.

~Anti~ ~Commercial-AI~ ~license~ ~(CC~ ~BY-NC-SA~ ~4.0)~

[-] Bishma@discuss.tchncs.de 1 points 2 days ago

Yeah, I was resistant to it for a while. I used the free version until I found I was running out of queries, then I got the cheap plan, and by the time they announced the $10 plan was going unlimited it seemed like a great deal. And I don't have to worry about Kagi trying to break my ad blocker because there are no ads to be blocked.

[-] mesamunefire@lemmy.world 1 points 2 days ago

Ive been coming around too. Can someone give good reasons to switch or any other killer features?

[-] maxenmajs@lemmy.world 2 points 2 days ago

A big one for me is that you can tweak the priority of specific domains. in my case I hate Fandom but I don't want to block it entirely, so I lower it and raise the other wikis I'd rather see first. I also have every Pinterest domain blocked, which would be very tedious to do with search engine operators.

[-] darkkite@lemmy.ml 2 points 2 days ago

neat! didn't know this was a thing

[-] Blaze@feddit.org 3 points 2 days ago

I’ve had bad luck with the search function of both Lemmy and Mbin

What were you searching for? I found a few results using that

site:lemmy.dbzer0.com and all the other instances all the time

You can probably just use one large instance after the "site:". If an instance is large and old enough (and I think dbzer0 definitely is), it would have all the valuable content available elsewhere.

this post was submitted on 15 Oct 2024
42 points (97.7% liked)

Fediverse

28051 readers
412 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!

Rules

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy

founded 1 year ago
MODERATORS