Imagine an AGI (Artificial General Intelligence) that could perform any task a human can do on a computer, but at a much faster pace. This AGI could create an operating system, produce a movie better than anything you've ever seen, and much more, all while being limited to SFW (Safe For Work) content. What are the first things you would ask it to do?
That last bit already happened. An AI (allegedly) told a guy to commit suicide and he did. A big part of the problem is while GPT4 for instance knows all about all the things you just said and can probably do what you're suggesting, nobody can guarantee it won't get something horribly wrong at some point. Sort of like how self driving cars can handle like 95% of things correctly but that 5% of unexpected stuff that maybe takes some extra context that a human has and the car was never trained on is very hard to get past.
Thanks for the link, that sounds like exactly what I was asking for but gone way wrong!
What do you think is missing to prevent these kinds of outcomes? Is AI simply incapable of categorizing topics as 'harmful to humans' on it's own without a human's explicit guidance? It seems like the philosophical nuances of things like consent or dependence or death would be difficult for a machine to learn if it isn't itself sensitive to them. How do you train empathy in something so inherently unlike us?