this post was submitted on 13 Jun 2023
94 points (100.0% liked)
Technology
37720 readers
36 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Feels like AI creators can only get away with using pre-2022 data for so long. At some point the information will be outdated and they'll have to train on newer data, and it'll be interesting to see if this is a problem that can be solved without harming the dataset's quality.
My guess is they'd need to have an AI that tries to find blatantly AI generated data and take it out of the dataset. It won't be 100% accurate, but it'll be better than nothing.
I'm surprised, these models don't have something like a "ground truth layer" by now.
Given that ChatGPT for example is completely unspecialized, I would have expected that relatively there's a way to hand encode axiomatic knowledge. Like specialized domain knowledge or even just basic math. Even tieried data (i.e. more/less trusted sources) seem not to be part of the design.
Because it's not designed to be a knowledge base, it's designed to imitate human communication. It's the same reason why ChatGPT can't do maths - it doesn't "know" anything, it just predicts the most likely word/bit-of-a-word to come next. ChatGPT being as good as it is at, say, writing code given a natural language prompt is sort of just a happy accident, but people now expect that to be it's primary function.