LLMs are they are now, we are hitting what we are going to get, we are not going to get any more 100x quality jumps like we did, until another breakthrough that is transformer tier, is figured out
rawb
Wednesday, August 12, 2026
Replying to Phijkchu_Pikachu #klombak I think the vast majority of people think LLMs are either somwhat useful at niche topics, or maybe very useful in general, but nothing more than that
I think the general population is aligned mostly with Leg, probably skewing a bit towards "fun novelty"
Replying to Phijkchu_Pikachu #klombak LLMs are they are now, we are hitting what we are going to get, we are not going to get any more 100x quality jumps like we did, until another breakthrough that is transformer tier, is figured out
I firmly disagree with this now, but that was my opinion for a long time, that LLMs were probably a dead end to getting to AGI or beyond
full stop, its not what they are designed to do
A lot of the big leaps post transformers being implemented are upping compute power and harvesting more data for training
I think we will hit actual AGI in my lifetime, not by 2030, and it will not be a LLM
It will incorperate aspects of how LLMs work, but LLMs by themselves won't make it to that
artificial general intelligence
aka actual AI in the scifi sense
or moreso that than what we have now
I don't think that LLMs are as rigid a construct as I initially thought
the initial approach to training felt self-limiting, and I would have been the first person to tell you it was going to hit a wall and our fun markov chain toys were going to all be forgotten about
and I think for a while, the investment has literally been
I think they are not rigid, there is a lot you can do with prediction based reasoning that LLMs do
invest billions ---> build a big training set ---> magic---> super AI
and its crazy investors went for that
because I do think we ended up on this current course almost accidentally
I think the limit is, a LLM cannot freely think beyond its data set. I think you might be able to get higher results using multiple LLMs in parallel doing different things
but you are not overcoming the fact that LLMs are limited by their dataset
when some advanced network of them do, are you just going to stop calling them LLMs?
I'd call them a network of LLMs
I think you can approximate human thought with enough LLMs running in parralel, but actual AGI is beyond that
maybe a function without distinction, but I do not support the idea of claiming what we have now is more than it is, a tool that ranges from not to very useful depending on what you use it for
I'd also agree with that, nothing that exists currently that is generally available, is more than a productivity tool
And I think people who are claiming "Google Gemini is alive" are retarded
I think when a LLM can update its own training data in reasonable time, without human intervention, and without imploding itself, I will consider us closer to your view
If I start seeing things slowdown that suggest we're anywhere near the top of an S-curve, I'd also be more aligned with you
I'm not naiive, if it stops working suddenly, then this is all dead and gay
I think it can improve, but I think the cost to improve it is getting higher for lower returns, and the only reason you have not seen it slow down is instead of letting it slow down they are just burning trillions to get these diminishing returns
when they stop dumping actual trillions on this, you'll hit the top of the S
I think that's probably the fundamental difference in view, I see progress accelerating right now
burning the trillions is working
It'll work until the returns are diminished to the point that you have to burn quadrillions to get the same returns
then we print more money
line go up
Okay, I will put it this way, I do not think an LLM, which has the sum total of all human knowledge as it exists right now, and the entire planet is reformed to exist to be in service to (only the data ceneter, factories making more memory etc, power generation) will be AGI
and I think there will be something in the next ~2 years that I will consider AGI
sure, but it will not be an LLM
it'll be a Network of LLMs
mtcdood, it's worse
they are mostly convinced they are creating god
the actual AI lab people I work with
are unirronically convinced they are going to meet a god if they do a good enough job
I think the people funding it are doing it for what mtcdood is saying, I think the people working on it are delusional like spooky said
also most think their work will make them immortal
lol
RIP
Honestly spooky, I will go the other way with my take from yours
Replying to Spooky Phijkchu #spooky.131 also most think their work will make them immortal
so, they're weird fanatics?
I think this huge LLM push has caused the opposite, and delayed actual AGI by probably 10 years
because this will never hit that, but it will get close enough that people will say "yep, we did it"
Google-IBM is the only lab working on non-LLM AI right now
Replying to Dialga #god_of_time so, they're weird fanatics?
yes, but it's not in the way that you think
I spend a lot of time around them, they're all sad about it
folding@home died to crypto
Spooky, I would love to talk to your coworkers just to see what the fuck is going on in their head
they tried to make their own token, but it was on a dead chain
Actually, I would love to talk to the freezing light guy way more than your coworkers, but they are a close second
they're not my direct co-workers, we have Anthropic people on-loan from Amazon who sit with us for a few months, then rotate back to their labs
the freezing light guy is hard at work
Spooky, hear me out. They do not actually believe this, but because they work for Anthropic they really need to upsell it, so whenever you are around they have to act like they think they are causing the end of the world
no, they really believe it
these are the researchers, not the PR people
lol
the Amazon PR people are the ones who tell us non-stop about AI-bro talking points and why we need to give AWS more money
I'm roped into the space datacenter thing xAI and Anthropic are trying to do to beat OpenAI
since OpenAI is pulling ahead again
which I remain convinced is a bad idea
but it means I get to play with big lasers
which is fun
Yeah
I am interested when the LLM craze dies down and people who do ML can get funding for not LLM machine learning stuff again, if we get anything really cool