Archive details

Discord archive

Neon Genesis Phijkchu

Lifecycle
July 17, 2020 – Present
Snapshot
Through August 30, 2026
Preserved messages
June 19, 2020 – August 30, 2026
Members
712
Messages
1,098,506
Channels
43

Archive navigation

rawb

Jump to date
Pins
Spooky Phijkchu rawb
Replying to Phijkchu_Pikachu #klombak
LLMs are they are now, we are hitting what we are going to get, we are not going to get any more 100x quality jumps like we did, until another breakthrough that is transformer tier, is figured out
I firmly disagree with this now, but that was my opinion for a long time, that LLMs were probably a dead end to getting to AGI or beyond
Spooky Phijkchu rawb
I don't think that LLMs are as rigid a construct as I initially thought the initial approach to training felt self-limiting, and I would have been the first person to tell you it was going to hit a wall and our fun markov chain toys were going to all be forgotten about
Phijkchu_Pikachu rawb
I think the limit is, a LLM cannot freely think beyond its data set. I think you might be able to get higher results using multiple LLMs in parallel doing different things
Phijkchu_Pikachu rawb
maybe a function without distinction, but I do not support the idea of claiming what we have now is more than it is, a tool that ranges from not to very useful depending on what you use it for
Phijkchu_Pikachu rawb
I think when a LLM can update its own training data in reasonable time, without human intervention, and without imploding itself, I will consider us closer to your view
Phijkchu_Pikachu rawb
I think it can improve, but I think the cost to improve it is getting higher for lower returns, and the only reason you have not seen it slow down is instead of letting it slow down they are just burning trillions to get these diminishing returns
Phijkchu_Pikachu rawb
Okay, I will put it this way, I do not think an LLM, which has the sum total of all human knowledge as it exists right now, and the entire planet is reformed to exist to be in service to (only the data ceneter, factories making more memory etc, power generation) will be AGI
mtcdood rawb
I think a lot of the AI hype are people who know they can get huge loans and investments, then leave the company before it collapses.
mtcdood rawb
I don't know if I mentioned it, but the AI company I worked for did in fact fire me for pointing out my boss was incompetent and we had 10-15 people doing whatever they wanted all day.
Phijkchu_Pikachu rawb
Spooky, hear me out. They do not actually believe this, but because they work for Anthropic they really need to upsell it, so whenever you are around they have to act like they think they are causing the end of the world
legorhin rawb
a ton of the LLM breakthrough methods apply to general machine learning so I have to pay attention to that stuff even if I find LLMs generally pointless
legorhin rawb
At ICML this year there was a lecture about how optimization theory is kind of a waste of time in ML because the random breakthroughs rarely have any theoretical basis and kind of just work. ML research is weird.

This archive is read-only — snapshot through August 30, 2026

Channels:
Density:

Jump to date

Local bookmark

Choose a color

This marker stays in this browser. Your choice becomes the default for the next bookmark.