I think a lot of this is all backwards. People think AGI is taking something dumb, like an LLM, and sticking on learning, like a software upgrade.
I think it's the other way around: you build a system that first and foremost _learns_ as part of its fundamental function, _then_ you train it in the domain you want expertise.
You're not going to get expertise in all domains all the time, just like with people. And you're not going to get a perfect slave either, just like with humans. You'll probably get something more like in between a human and machine. If that's what you really want, great.
To put this another way, if you neglect your kids, they're still going to learn things, just probably not things you want them to learn. If you neglect your language model it's just not going to do anything.
I think it's the other way around: you build a system that first and foremost _learns_ as part of its fundamental function, _then_ you train it in the domain you want expertise.
You're not going to get expertise in all domains all the time, just like with people. And you're not going to get a perfect slave either, just like with humans. You'll probably get something more like in between a human and machine. If that's what you really want, great.
To put this another way, if you neglect your kids, they're still going to learn things, just probably not things you want them to learn. If you neglect your language model it's just not going to do anything.