Rendered at 18:42:55 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
grey-area 20 hours ago [-]
How refreshing to read the thoughts of a human in a blog post instead of the garbage that LLMs produce. As the writer says about some of their proofs, the problem is not one of style or em-dashes, but something deeper about the writing that comes out of LLMs:
The problem is it doesn’t make any goddamned sense.
I'd probably insert a 'sometimes' in there, because many times in small doses it does make sense, it's only when you ask for long form writing/work that it becomes apparent there is no coordinating intelligence there.
Interesting just how much of this advice applies to programming as well as to maths:
--
Skills, agent.md and harnesses are probably a dead-end that will die off as the tools become more capable.
LLMs are not good at reducing complexity, however they're very good at adding it, and sometimes it is just unnecessary and he can find ways to cut it by interrogating the LLM.
Some people are responding to the new reality by gleefully using AI to generate lots of semi-OK-ish papers and puking them up on arXiv [this is like people producing mediocre one-shot clones of software].
On the other hand—with creative writing, my advice is not to use AI to write, at all. Ask questions if you want, but write every damned word yourself.
--
Definitely worth a read for the programmers here too.
IvanK_net 19 hours ago [-]
How do you know it was written by a human? All we can do is to believe the author.
grey-area 19 hours ago [-]
Fairly easy to tell at the moment, LLMs are not very coherent or interesting in long form writing as yet.
But if you want to believe otherwise that's up to you (or perhaps you just hope it could be otherwise?). If you prefer, you can read that sentence as 'How refreshing to read intelligent, coherent thoughts in a blog post instead of the garbage we so often see nowadays.'
The sentiment is the same, it's about quality and intelligent thought, not style or provenance. Maybe one day we'll have to say intelligence rather than human, which is a fun thought. I don't feel we're close to that yet though.
evdubs 18 hours ago [-]
If you believe pangram is a good AI text detector, it scores the first 2000 words of the article as 100% human written.
sdenton4 21 hours ago [-]
"If you just waited a year, would some other AI produce that same proof independently?"
I had a conversation about this very problem last November, with a visiting mathematician. It's essentially signaling a deflationary spiral in intellectual work: why bother doing the work now, when it will take half as much effort a year from now?
This is perhaps always a possible stance, but it's exacerbated when some proportion of your own agency is handed over to, well, agents.
augment_me 20 hours ago [-]
This might hold up for theoretical intellectual work without any economical value, but if you are researching anything that can be applied to the industry, you will instead have 1 year of market ownership and first-mover advantage.
I think it says more about what the value of math as intellectual work is, than about intellectual work in general.
Like if an Egyptian historian would come and say "let's wait one year until the methods are good enough to decrypt this language", no one would be bothered or surprised.
Loquebantur 19 hours ago [-]
You imply "worth" being equal to some known economic value.
You even imply, economic value was determined correctly in advance for arbitrarily long future intervals. And the applicability was known in advance to begin with.
Obviously, you're completely wrong.
Applying simplistic models to real life incurs a penalty. The affected scale multiplies with that penalty. When dealing with things affecting humanity as a whole, that makes careless over-generalizations a serious hazard.
doc_ick 20 hours ago [-]
“if you are researching anything that can be applied to the industry, you will instead have 1 year of market ownership and first-mover advantage.”
I would like to disagree with you on that point. Given the very popular rise of https://rea.tools/, it appears the time to reverse engineer anything software based (up to maybe 80% or more?) could be down to minutes. With a recent example, some indie video game developers are not going to show off their video game until release as a few have been flipped and made it to market before the original developers have. Point being, a few moths of market ownership could be possible if llms can’t reverse it in any way.
FeteCommuniste 18 hours ago [-]
What a waste. All the anticipation that used to be built up with previews and looks into the dev process, now flushed down the toilet so billionaires can flood the web with crap and make more billions.
ignoramous 20 hours ago [-]
Someone is going to make the case against using LLMs to reverse engineer software solely for commercial gain being as illegal as using guns for extortion.
doc_ick 18 hours ago [-]
Unless you’re able to get a patent on day one, once an idea is public potentially within the hour someone can make more money off of it and it’s not just your idea. Seems like a valid case to lock it down.
hn_throwaway_99 20 hours ago [-]
First movers often lose though in the market. Relevant to today's news about DoorDash acquiring Instacart for a few billion dollars: WebVan came out a decade earlier (i.e. at the wrong time) and went bankrupt after 3 years.
A perhaps more relevant personal anecdote: About a year ago I told a non-technical friend that I'd help him migrate his small business website that was originally set up in WordPress (by a developer who is since long gone) to a format that would be much easier for him to manage with no dev support. I planned to move him over to one of the usual suspects like Wix or Squarespace, but this is one instance where my procrastination was a virtue. In the ~9 months since he originally had an issue with his site, AI capabilities had exploded. Turned out it was a lot easier (and cheaper) to use Claude to fully migrate his site and get it up on Cloudflare workers, and I set up everything so he could manage all the content himself. Plus, it took me less than a day to do the migration off WordPress, and most of that time was just me wanting to learn the ins and outs of Claude Code because I was new to it at the time.
It might also mean we can take more time, or prep for it more, or think of the implications. The deflation can be good, it also means the future could be way better off. It could be that civilizations destroy themselves through multiple conflicting levels of power (political, genetic, energy, material consumption, mass media, malaise).
RealityVoid 21 hours ago [-]
I like this observation very much. Intellectual deflationary spiral is exactly how it feels at the moment and we might end up in a worse place because of it.
amelius 19 hours ago [-]
Or in a better place, because we're not feeding the beast, so to speak.
bananaflag 19 hours ago [-]
This is exactly how I feel. I tell everyone around me who is churning AI papers that the whole thing is pointless.
In 1-2 years the AI will give an instant answer to most conjectures anyway.
Loquebantur 19 hours ago [-]
Who will know to ask the right questions?
FeteCommuniste 18 hours ago [-]
Other AIs, of course. AIs all the way down.
cluckindan 18 hours ago [-]
42
latentsea 16 hours ago [-]
>But maybe that’s still just making the proof easier for humans to understand? No! For one, this usually makes the proof shorter, and sometimes dramatically shorter.
Hmm, so is the way to train this to take all the long complicated proofs it generates in Lean and then use autoresearch to minimize the length of the proof?
My priors have been so thoroughly updated by all the recent progress in AI that it really seems like there isn't anything that fundamentally cannot be learnt. So, is this how we train this kind of taste?
I'm not a mathematician, nor an AI engineer/researcher. Just watching from the sidelines and spitballing ideas.
What a time to be alive?
timbaboon 21 hours ago [-]
Point 19, right in the feels :(
an0malous 21 hours ago [-]
The overselling of AI abilities is doing such a disservice to society right now. These companies don’t want to acknowledge how much human collaboration is going into their AI’s achievements because it undermines their value proposition, but I imagine there’s still quite a fair amount going into it and that’s one reason why OpenAI still won’t provide session traces of their models solving open math problems. If they were more transparent about it I think we’d see that human guidance, vision, judgement, taste, and other traits are still highly valuable and complimentary to AI abilities. It should also be a huge red flag how little of any of this output has been formally verified by third parties. I do think AI will eventually do all the things people believe it’s already doing, but I suspect there’s a fair amount of mechanical turking going on behind the scenes as you’d expect any company trying to raise a trillion dollar public offering might do.
Jtarii 20 hours ago [-]
Are you suggesting the ~500 mathematical results they have released over the past year are primarily human discovered?
I think they would have employ just about every mathematician on earth for that to make sense.
nextaccountic 20 hours ago [-]
No, but it required non trivial human work
Jtarii 20 hours ago [-]
I agree, a colossal amount of human effort had to go into building the machine in the first place.
OAI is not simply prompting "Prove exponential time hypothesis false. Make no mistakes" for the same reasons you can't one-shot "Write a fully standards compliant internet browser from scratch"
jonlong 19 hours ago [-]
Terry's session is his attempt to understand Fable's counterexample to the Jacobian Conjecture, not an attempt to solve any open problem.
Levent Alpöge's prompting that did solve this open problem does not appear to be public, but Aaron Lou at OpenAI succeeded in reproducing the result the next day using an internal version of Codex and (reportedly) without web search, and he shared his prompt at https://aaronlou.com/jacobian_counterexample_prompt.pdf. It is a mathematically-sophisticated statement of the problem and of general search strategies, but contains no concrete ideas for the solution whatsoever (even the idea to look for a counterexample instead of a proof).
this is a great and wonderful contribution. best thing i have read on this.
jaynetics 12 hours ago [-]
I agree. Perhaps too many blogs go for a single, coherent punchline, and the text form of "somewhat scattered thoughts" is under-appreciated?
springtail5 4 days ago [-]
This is very similar to how I feel programming.
XTXinverseXTY 20 hours ago [-]
What's a Kofuni expansion?
rekcuts 20 hours ago [-]
I was similarly curious, but after a cursory search returned nothing, I assume it was a placeholder for an esoteric or tedious task.
latentsea 16 hours ago [-]
Is it like mathematician slang for 'yak shaving'?
idontknowmann 21 hours ago [-]
It’s funny how disconnected we all are from each other. This is exactly how LLMs felt a year or so to many programmers.
Useless toys, sometimes useful toys, ok good toys, ok.. we might have to look at this more carefully, oh god.
This loop however is quite tiring to watch happen over and over again. Is there really nothing we can do but just watch from the sidelines and watch our fellow men engage in their particular grief and copium filled mourning cycle?
hackinthebochs 18 hours ago [-]
It's not too late to do something about it. Acknowledging that we are not powerless to determine what our future looks like is the first step to taking control. We all seem resigned to the impending disempowerment and crisis of meaning for humanity as a whole. But this is just playing into their hands. We should collectively exercise our power of self-determination while we still have it.
noduerme 13 hours ago [-]
How, specifically? If I derive meaning in my life, economic prosperity, and social appreciation from creating games out of my imagination and bringing them to life for people, and they can now be reverse engineered and slop-copied instantly with no credit of any sort, what am I going to do about it? Make games and never show them to anyone? Stop doing what I love? Spend the rest of my life in pointless and wasteful battles over it?
No, I'm probably just going to say fuck 'em all, I feel sorry for everyone. Go to a bar and drink while the world burns.
There's nothing to do about it.
hackinthebochs 12 hours ago [-]
There's nothing any of us can do about it individually. But collectively, we still have power. It's a matter of organizing into collective action. This is a political issue just like any other. When the vast majority of people don't want something to happen, we have the power to stop it if we act. I'm not the one to make that happen; there are much better people for that. What I'm saying is how we speak about the issue should be one of agency, not of resignation. Agentic attitudes are contagious, just as defeatist attitudes are. Perhaps collective action on a small scale can spontaneously organize out of it. Or the right people might see enough people poised to act to be the organizing force. None of this happens if we admit defeat before shots are even fired.
noduerme 11 hours ago [-]
I don't see how. It's not an electoral matter; it's not a political system. It's not ICE coming for my neighbors, or a broken healthcare system. It's the epistemological apocalypse. It's the supernova where meaning collapses and the howling noise of madness overwhelms the signal of reason. Perhaps you could shut it down in one country, by law, but Pandora's box has been opened. The Tower of Babel. Open models will soon be good enough to create a phoney version of anything, anywhere, close enough to seeming true that by the time you discover it's not, a million more clones will be out there. Lies have always spread faster than truth, but never with such a force multiplier. The truth ain't dying in darkness, it's dying in blasting white noise, diffusion, ironic that turned out to be the approach. Or maybe not. What's this land rush to claim latent space but the attempt to expropriate, forever, the human ability to discover it?
The Leonard Cohen line keeps coming back to me: The blizzard, the blizzard of the world has crossed the threshold and it's overturned the order of the soul.
Sorry, but... I scoff at the idea that collective action can do a damn thing about it. There isn't even a collective idea of what to do instead of it. Collective action also has heavy connotations of communism, under which, people like me were also not permitted to write games and reap the economic and social benefits of doing what we enjoy. But at least one could say that, for the communist bloc, the collective action finally came against communism. So there's that, and maybe there's some sort of small-creative-capitalist-producer collective action that could be taken here...
but who are we kidding! I just had dinner with a friend who's a VP at [major motion picture studio] who described the situation in the movie business as "apocalyptic", and they have the ability to organize and a history of organizing far more than whichever random cross-section of white collar knowledge workers and engineers loses their livelihood in the next six months, or the six months after that.
No, my friend, I think this is a war where no shots needed to be fired, because we were all so dazzled by the bright lights that no one realized we were being steamrolled until it was too late. It's already over.
hackinthebochs 9 hours ago [-]
When you need a football field of GPU to host frontier models and advance the state of the art, it's not too late. The supply of GPUs can be monitored and controlled. Access can be limited. International agreements to stop further progress can be hashed out (the China bogeyman is a mirage). We could outlaw the commercialization of more capable models and airstrike datacenters that aren't in compliance. If the current models were as far as we got, we'd be mostly fine. The only thing stopping us from realizing this future is the inability to coordinate.
latentsea 16 hours ago [-]
The future is not set.
whoanoway 21 hours ago [-]
[dead]
lsvg 20 hours ago [-]
These mournings of programmers and now mathemathicians will as time goes on likely continue to become a part of other fields workers routines as well.
I wonder how we as a society will react to this, this crisis of meaning. Not to get into existential crisis if that is to happen as well, which of course is a whole other issue.
I assume we will all start to look for social ways of mourning with colleagues.
kypro 18 hours ago [-]
> I think the biggest reason I’m not more productive is: What’s the point? In the past, if you came up with an elegant proof, you’d tell yourself, "I made a thing! Other people will read and use my thing! I am advancing human progress."
This is so relatable.
I've always been very self-motivated and keen to dig deep into things and create difficult stuff, but these days I just see no point learning or doing anything...
Everything that used to give my life colour seems completely pointless now...
Learning new algorithms, creating open source projects, writing music, writing about tech, etc.
I know people say that it's great and now we all get to spend more time going for walks and watching TV, but it all feel kinda empty honestly. My fun was always doing difficult stuff and feeling that joy of working it out or giving the world something cool. But now, for the first time, I feel I have nothing to offer anyone.
thefourthchime 22 hours ago [-]
About two years ago, I was at a coffee shop and two maths students were trying to understand a concept in the book. I suggested they might ask a language model to help them understand it. They scoffed at me, called a stochastic parrot, and said they can't do math at all.
I responded with, "Well, you don't even need it to do math. It can explain the concept." They scoffed again. "Those things are useless toys"
georgemcbay 22 hours ago [-]
IMO they weren't exactly wrong... 2 years ago.
2 years ago LLMs were a great tech demo but were mostly awful for actual practical purposes and hallucinated almost as much as they got right.
They'd be wrong today, but that's a different thing.
preommr 20 hours ago [-]
Reading comments like this is so depressing because it was extremely obvious that better results were a matter of time/hardware. It evokes the seem feeling as that trump quote about "who knew healthcare could be so complicated".
skew-aberration 20 hours ago [-]
Also, even GPT-3 (6 years ago) was a very powerful search tool for exactly these kinds of problems. You needed some result that would give a bound on some obscure property, and it instantly gave you five different applicable theorems from three different domains of math. Usually, the theorems even existed.
I was taken aback by the progress in 2025/2026 because late 2024 did seem to be a plateau, but 5-10 years seemed inevitable.
Even when I first studied automated theorem provers in around 2013, the 10-20 years to automating proofs (not theory building) seemed inevitable. They were already superhumanly fast and generating real results, and neural networks were making massive progress on translation, image processing, and games. Someone just had to work out how to stick a neural net inside an ATP. The surprise was that it happened the other way around, and that it solved even more than just mathematical proofs.
AI has only just begun to expose the depth of willful human ignorance.
bananaflag 19 hours ago [-]
> late 2024 did seem to be a plateau
Why? o1 was released in late 2024
skew-aberration 18 hours ago [-]
Yes. Pre-o1, it didn't seem to be improving as rapidly as expected.
georgemcbay 19 hours ago [-]
> Reading comments like this is so depressing because it was extremely obvious that better results were a matter of time/hardware. It evokes the seem feeling as that trump quote about "who knew healthcare could be so complicated".
I honestly don't see how this has anything to do with my reply to the original comment.
Even if you were absolutely sure LLMs were going to get as good as they are today 2 years ago, that didn't make them any more of a practical solution to the issues you were trying to solve 2 years ago.
It is likely that the SpaceX Starship will eventually be a reliable rocket platform. That doesn't mean that given the chance I would hop on the next flight.
The problem is it doesn’t make any goddamned sense.
I'd probably insert a 'sometimes' in there, because many times in small doses it does make sense, it's only when you ask for long form writing/work that it becomes apparent there is no coordinating intelligence there.
Interesting just how much of this advice applies to programming as well as to maths:
--
Skills, agent.md and harnesses are probably a dead-end that will die off as the tools become more capable.
LLMs are not good at reducing complexity, however they're very good at adding it, and sometimes it is just unnecessary and he can find ways to cut it by interrogating the LLM.
Some people are responding to the new reality by gleefully using AI to generate lots of semi-OK-ish papers and puking them up on arXiv [this is like people producing mediocre one-shot clones of software].
On the other hand—with creative writing, my advice is not to use AI to write, at all. Ask questions if you want, but write every damned word yourself.
--
Definitely worth a read for the programmers here too.
But if you want to believe otherwise that's up to you (or perhaps you just hope it could be otherwise?). If you prefer, you can read that sentence as 'How refreshing to read intelligent, coherent thoughts in a blog post instead of the garbage we so often see nowadays.'
The sentiment is the same, it's about quality and intelligent thought, not style or provenance. Maybe one day we'll have to say intelligence rather than human, which is a fun thought. I don't feel we're close to that yet though.
I had a conversation about this very problem last November, with a visiting mathematician. It's essentially signaling a deflationary spiral in intellectual work: why bother doing the work now, when it will take half as much effort a year from now?
This is perhaps always a possible stance, but it's exacerbated when some proportion of your own agency is handed over to, well, agents.
I think it says more about what the value of math as intellectual work is, than about intellectual work in general.
Like if an Egyptian historian would come and say "let's wait one year until the methods are good enough to decrypt this language", no one would be bothered or surprised.
You even imply, economic value was determined correctly in advance for arbitrarily long future intervals. And the applicability was known in advance to begin with.
Obviously, you're completely wrong.
Applying simplistic models to real life incurs a penalty. The affected scale multiplies with that penalty. When dealing with things affecting humanity as a whole, that makes careless over-generalizations a serious hazard.
I would like to disagree with you on that point. Given the very popular rise of https://rea.tools/, it appears the time to reverse engineer anything software based (up to maybe 80% or more?) could be down to minutes. With a recent example, some indie video game developers are not going to show off their video game until release as a few have been flipped and made it to market before the original developers have. Point being, a few moths of market ownership could be possible if llms can’t reverse it in any way.
A perhaps more relevant personal anecdote: About a year ago I told a non-technical friend that I'd help him migrate his small business website that was originally set up in WordPress (by a developer who is since long gone) to a format that would be much easier for him to manage with no dev support. I planned to move him over to one of the usual suspects like Wix or Squarespace, but this is one instance where my procrastination was a virtue. In the ~9 months since he originally had an issue with his site, AI capabilities had exploded. Turned out it was a lot easier (and cheaper) to use Claude to fully migrate his site and get it up on Cloudflare workers, and I set up everything so he could manage all the content himself. Plus, it took me less than a day to do the migration off WordPress, and most of that time was just me wanting to learn the ins and outs of Claude Code because I was new to it at the time.
In 1-2 years the AI will give an instant answer to most conjectures anyway.
Hmm, so is the way to train this to take all the long complicated proofs it generates in Lean and then use autoresearch to minimize the length of the proof?
My priors have been so thoroughly updated by all the recent progress in AI that it really seems like there isn't anything that fundamentally cannot be learnt. So, is this how we train this kind of taste?
I'm not a mathematician, nor an AI engineer/researcher. Just watching from the sidelines and spitballing ideas.
What a time to be alive?
I think they would have employ just about every mathematician on earth for that to make sense.
OAI is not simply prompting "Prove exponential time hypothesis false. Make no mistakes" for the same reasons you can't one-shot "Write a fully standards compliant internet browser from scratch"
Levent Alpöge's prompting that did solve this open problem does not appear to be public, but Aaron Lou at OpenAI succeeded in reproducing the result the next day using an internal version of Codex and (reportedly) without web search, and he shared his prompt at https://aaronlou.com/jacobian_counterexample_prompt.pdf. It is a mathematically-sophisticated statement of the problem and of general search strategies, but contains no concrete ideas for the solution whatsoever (even the idea to look for a counterexample instead of a proof).
Aaron reported that this 2-page prompt found a counterexample in one shot. (https://x.com/aaron_lou/status/2079222629794189515)
Lex Fridman recent podcast on the AI crisis in math, https://youtu.be/ZAzWPE9CItg
Useless toys, sometimes useful toys, ok good toys, ok.. we might have to look at this more carefully, oh god.
This loop however is quite tiring to watch happen over and over again. Is there really nothing we can do but just watch from the sidelines and watch our fellow men engage in their particular grief and copium filled mourning cycle?
No, I'm probably just going to say fuck 'em all, I feel sorry for everyone. Go to a bar and drink while the world burns.
There's nothing to do about it.
The Leonard Cohen line keeps coming back to me: The blizzard, the blizzard of the world has crossed the threshold and it's overturned the order of the soul.
Sorry, but... I scoff at the idea that collective action can do a damn thing about it. There isn't even a collective idea of what to do instead of it. Collective action also has heavy connotations of communism, under which, people like me were also not permitted to write games and reap the economic and social benefits of doing what we enjoy. But at least one could say that, for the communist bloc, the collective action finally came against communism. So there's that, and maybe there's some sort of small-creative-capitalist-producer collective action that could be taken here...
but who are we kidding! I just had dinner with a friend who's a VP at [major motion picture studio] who described the situation in the movie business as "apocalyptic", and they have the ability to organize and a history of organizing far more than whichever random cross-section of white collar knowledge workers and engineers loses their livelihood in the next six months, or the six months after that.
No, my friend, I think this is a war where no shots needed to be fired, because we were all so dazzled by the bright lights that no one realized we were being steamrolled until it was too late. It's already over.
I wonder how we as a society will react to this, this crisis of meaning. Not to get into existential crisis if that is to happen as well, which of course is a whole other issue.
I assume we will all start to look for social ways of mourning with colleagues.
This is so relatable.
I've always been very self-motivated and keen to dig deep into things and create difficult stuff, but these days I just see no point learning or doing anything...
Everything that used to give my life colour seems completely pointless now...
Learning new algorithms, creating open source projects, writing music, writing about tech, etc.
I know people say that it's great and now we all get to spend more time going for walks and watching TV, but it all feel kinda empty honestly. My fun was always doing difficult stuff and feeling that joy of working it out or giving the world something cool. But now, for the first time, I feel I have nothing to offer anyone.
I responded with, "Well, you don't even need it to do math. It can explain the concept." They scoffed again. "Those things are useless toys"
2 years ago LLMs were a great tech demo but were mostly awful for actual practical purposes and hallucinated almost as much as they got right.
They'd be wrong today, but that's a different thing.
I was taken aback by the progress in 2025/2026 because late 2024 did seem to be a plateau, but 5-10 years seemed inevitable.
Even when I first studied automated theorem provers in around 2013, the 10-20 years to automating proofs (not theory building) seemed inevitable. They were already superhumanly fast and generating real results, and neural networks were making massive progress on translation, image processing, and games. Someone just had to work out how to stick a neural net inside an ATP. The surprise was that it happened the other way around, and that it solved even more than just mathematical proofs.
AI has only just begun to expose the depth of willful human ignorance.
Why? o1 was released in late 2024
I honestly don't see how this has anything to do with my reply to the original comment.
Even if you were absolutely sure LLMs were going to get as good as they are today 2 years ago, that didn't make them any more of a practical solution to the issues you were trying to solve 2 years ago.
It is likely that the SpaceX Starship will eventually be a reliable rocket platform. That doesn't mean that given the chance I would hop on the next flight.