A fairly obvious theory at that, unless I've critically misunderstood what's being described. As with so many obvious ideas I've always assumed that the reason I haven't come across it in the wild is because it doesn't work (or is comparatively inefficient, or tends to blow up during the training run, or etc).
> Realized reasoning gains, hardware efficiency, and RL scaling remain to be established.
Yeah so the first entry in that list is - if I may be so bold - typically what you'd start with at a small scale _before_ writing up and publishing your "genius" idea. This is the usual crank with delusions of grandeur presenting something straightforward that he hasn't tested as though it were a working breakthrough.
Ironically the cost of testing such theories has fallen to an all time low given the capabilities of coding models. I wouldn't be surprised if a frontier model could one shot a test of this.
I've been working on research related to this in the context of diferent LLMs, and I can tell you that similarity != executability.
While you can make embeddings across different LLMs similar (i.e. universally looking) the few percentage of R2 that you are missing in translating the universal representation into a native representation are precisely those that make the hidden states executable in the LLM (which makes them useful).
They do this with simple embedding models because there the purpose of the embeddings is to measure similarity, but if you would like to use this principle to turn latent representations of one LLM into latent representations that are understandable/executably by a different LLM, you will fail.
Naive question:
Why pile so much public money into quantum computing where there are (afaik) barely any useful algorithms, while Europe could instead invest in catching up on GPU infrastructure where there are plenty of known useful and capacity constrained use cases in academia and beyond?
GPUs are in a weird bubble captured by weird finance tricks in the private sector. They put money into that, taxpayers are going to freak out.
If they can make progress on QOA and HHL algos, that has value.
But the main thing they're trying to avoid - and this can't be understated - is dependency on US tech firms when quantum gets good enough to be practical for those sorts of problems.
Since WW2, Europe has happily been a consumer of US produced tech. The last 10 years has caused a revisiting of that view. It's an exercise for the reader to figure out the factors leading to that, and whether they think it's sensible or not.
> Why pile so much public money into quantum computing where there are (afaik) barely any useful algorithms, while Europe could instead invest in catching up on GPU infrastructure...
Because if you do not become competitive on the frontier you are doomed to perpetually play a game of catch-up. This is what happened to the EU with battery technology, renewable technology, AI, digital services, and semiconductors.
GPUs require sub-7nm design, fabrication, and packaging - all of which is nonexistent in Europe today, and would take 10-20 years at which point the EU-27 would have fallen behind the frontier again.
Europe cannot be competitive in bleeding edge and legacy packaging in 2026-36 given how heavily subsidized and tightly integrated the Asian and American cluster is (eg. I can get 0-1% interest rate loan terms with tax holidays and subsidizes in the 9 figure range in much of Asia and the US). Europe CAN reinvest in 2D packaging, materials, and metrology (which is critical for quantum).
Europe cannot be competitive in bleeding edge fabrication in 2026-36 given how heavily subsidized and tightly integrated the Asian and American cluster is (eg. I can get 0-1% interest rate loan terms with tax holidays and subsidizes in the 9 figure range in much of Asia and the US). Europe CAN be competitive in legacy (28nm and above), power, and compound semiconductor fabrication (this also had a downstream impact on quantum).
Europe cannot be competitive in chip design in 2026-36 given how heavily subsidized and tightly integrated the Asian and American cluster is. It CAN work on building the next generation of tooling needed to design quantum circuits and conduct metrology.
Quantum applications of Sensing, Optics, Communication, Key Distribution, Computing, Packaging, and Hardware Design are all greenfield segments that everyone is at a roughly equal starting point at. All these technologies have directly relevant applications in telecommunications, defense, semiconductor packaging, and computing.
And this is what can help some EU states regain relevance in the competitive frontier. There's a reason France has been working heavily on this for a decade.
"This is what happened to the EU with battery technology, renewable technology, AI, digital services, and semiconductors."
I don't agree with the "semiconductor" take of that. ASML, a European company funded by a lot of private and non-European capital, is basically producing the machines that are making all of the high-end semi conductors of the past few years. They have received grants from the EU, arguably small in compared to the private funding though.
IMEC, and their NanoIC research on the other hand receives a lot of funding from the EU.
Hmm, I think you provide a compelling narrative. However I'm not very convinced.
I think AI is a good example. EU wide there are plenty of public HPC centers and in 2020 we had (relative to the world) plenty of compute. In other words, we had the tech, we had the compute, and we had the know how. And yet we don't really have a competitive AI scene in Europe. There were some very small projects launched to try to train LLMs. All of them were no more than learning exercises.
Because individual EU states now have no choice. The alternative is to accept being a contested beach ball squeezed between the US and China, and with other regional powers taking advantage.
The EU had already lost the initial race in HPC and AI by 2011-16 when EU institutions and state-level institutions failed to support the kind of public-private collaboration that was happening contemporarily in the US and China. I remember noticing on visits how EU nationals were often working on or closely with the Guangzhou Supercomputing Center in large part because capital that could monetize fundamental research dried up due to the Eurozone crisis.
Similarly in Quantum, barely 15 years ago, a large portion of China's current leading quantum researchers like Pan Jianwei were still working closely with European researchers like Anton Zellinger at institutions like Wien, Geneva, INRIA, and the Max Planck Institutes to close knowledge gaps that remained. While the gap has now been reversed, it can be managed because having some sovereign capabilities at least helps reduce the risk of being at the mercy of export controls.
Basically, the EU and individual EU states have one last chance to recapture the frontier lest it's gone.
It's part of the EU's strategy to catch up by investing in future technology instead of building out present capacity.
I think it's a mistaken strategy but it's hard to suggest anything better when you look at the measly amounts they're investing that would get them very little with existing technology. So, they're forced to gamble on vaporwave tech.
Edit: I'm specifically talking about their semiconductor and computing strategy.
True, but what they do is they invest in fundamental research and then expect magic to happen and that research to lead to economic growth. Unfortunately the wizards that cast the spells live in the US, not Europe.
Citation needed.
And the problem isn't that Europe doesn't have its own wizards, it's that they don't have the mana market to turn those spell ideas into reality.
Catching up in GPU infrastructure in what way? It is not reasonable to expect to catch up on full GPU design to fabrication. If you mean datacenters, I believe they are being built wherever feasible, nordic regions and then spain/portugal.
Quantum today is actually in a good place investment wise because two fields have matured enough that there is a defense usecase to rally funding around - communication and sensing.
Computing is less than 1-3rd of the story. We could end up failing to make a fault tolerant computer and be totally fine and happy with the other benefits.
The long term hope that we throw out there is that these two usecases subsidise the development of quantum computers in some way, and _that_ helps take it over the line. But that's just a bonus.
Quantum computing is a far less mature technology than GPUs, and investments in it will yield a lot of interesting research that nobody else would be willing to pay for. The bet is that solving the problems one needs to solve to make advancements towards practical quantum computing will also solve other problems in other fields that might be much closer to a commercial application.
No word about taxes and the paper describes workers as being “protected on the downside.” while the model has removed the downside risk that workers actually face. I could write a long essay with all the issues this "paper' has.
Truly dismal science of an Economics professor at MIT.
This isn't really on you but the problem I have with comments like this is that I think most people write poorly so I can't tell if those are LLM artifacts or LinkedIn-speak artifacts. I need better heuristics for these things.
It was flagged for being clearly botted, and I don't just mean LLM-generated (although it is also that). When I posted on and flagged the old submission, I noted that it had over two dozen upvotes in 15 minutes, despite the essay having a helpful "54 min read" indicator at the top. I truly do not believe the upvotes on these submissions are organic, and it really destroys my faith in HN that this was restored when the first one went down successfully.
(If by some chance I am wrong and this monster of an LLM-generated essay really got dozens of people instantly upvoting it from the title alone, that fact would also not give me much faith in HN, I have to add.)
I upvoted this without reading. For me sometimes the article is just the spark for a far more interesting set of comments that overshadow it. And it is really that I am voting for.
I upvoted after reading a large portion. I had commented previously that the use of Gaussian math was reductive, but I think the contrasting mathematics of the two types of inheritance is interesting to consider.
I've done the same. Discussions on some subjects are worth having and so I will sometimes upvote because I really want to hear what the HN crowd - who are by and large pretty smart and have interesting perspectives - have to say.
Does the first 1/5th or 1/4th of the article provide such a compelling case that one wants to stop reading it and return to upvote it? I certainly didn't think so.
The essay, if taken seriously, has a logical conclusion that I'm sure a decent number on this website find uncomfortable, but perhaps more importantly, which at least some people who might have disproportionate power to make such decisions prefer not be discussed too heavily. This is a microcosm of the larger phenomenon
Well, all of the comments on the old submissions were complaining about the essay being AI slop. I haven't read the essay so I couldn't say, but that's clearly the reason why.
I read the whole thing; it's definitely not AI slop. A few sentences taken out of an hour long read that are commonly used by AI doesn't mean the article was composed by AI. You'll find similar artifacts in any longer published work, including those published prior to AI.
While there are more excessive examples of moderation than Hacker News, "Rate Your Music" moderation is hilariously indulgent and arbitrary from my 2nd hand vantage, but moderation here and elsewhere tends to error on the side of removal and exercise of power. There is no BDSM thrill behind staying one's moderation ability. Many are attracted to moderation just as many police are attracted to their work to get the dopamine that is gifted to them by dominating others. But it might be reductive to think this moderation is tied to an individual's brain treat, and it might instead be useful to consider the host organization -- they may not feel enriched by posts that critique status quo capitalism.
This is not the only paper that scales reasoning complexity / difficulty.
The CogniLoad benchmark does this as well (in addition to scaling reasoning length and distractor ratio). Requiring the LLM to purely reason based on what is in the context (i.e. not based on the information its pretrained on), it finds that reasoning performance decreases significantly as problems get harder (i.e. require the LLM to hold more information in its hidden state simultaneously), but the bigger challenge for them is length.
reply