61
We tested ChatGPT in Bengali, Kurdish, and Tamil. It failed.
(restofworld.org)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 06 Sep 2023
61 points (100.0% liked)
Technology
37801 readers
272 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
Another thing worth keeping in mind is that LLMs are feedforward networks, meaning they do everything in a fixed amount of time, no looping or second turns allowed. So even if the network has enough information to translate Bengali to English, that still doesn't necessarily give it the power to answer question in Bengali directly, as by the time it has figured out what the Bengali means, it's already out of layers to solve the problem.
Proper prompting can sometimes overcome those limitations, e.g. translate the problem into English first, feed the English text back into it to solve the problem and translate that output back to Bengali. The LLM itself can't do all of that in one step, but if the human sends the proper prompts do to it step by step it can sometimes solve complex problems that it couldn't do in one turn.
That's an important point you raise. I feel like a big problem with the LLM projects we see today, including ChatGPT, Bard, etc., is that the developers have tunnel vision. Rather than using the LLM as one component of a system with many well-researched traditional algorithms doing what they do best, they want to do everything within the network.
This makes sense from a research perspective. It doesn't make sense from an end-product perspective.
The more I play with LLMs, the more I feel like their true value is as something like "regular expressions on crack".
How does it go?
"I know! I'll use an LLM!"
"Now you have three problems."
True. When you have a hammer, everything looks like a nail.
I haven't actually implemented this yet, but I've been thinking about making a local file search program using an LLM. It would enable me to search for things in ways that are absolutely impossible with language-naive tools.
Here are a few examples of tasks I have in the past wanted to do, but was not able to:
Find references to the board game 'go' but please for the love of god do not return every case of the verb 'go'. Also include passages that refer to the game but don't mention it by name.
Find all references to foods, eating, or meals.
Find all the dialog lines of a specific character.
You can't do any of that with simple search, or even with regular expressions. You need general language awareness.
I know that LLMs will not be perfect at these tasks, either (at least not the current ones), but I think they could be quite effective.