I’ve never seen CEOs work so hard to make the public aware of how dangerous and out of control their flagship product is. It makes me automatically assume they’re scheming about something else like regulatory capture to protect their market.
Agree it’s crazy, but it’s the least worst thing for them right now when considering they have completely failed to deliver all the promised impacts of AI. The growth of these companies is massively impressive as is the tech, but they are an order of magnitude or more off where they need to be to justify anything like the valuations they need to stay alive.
It’s budget season in the corporate world and from what folks are saying things are not going well for the big labs. Token budgets are being slashed and companies are switching to open weight models. That coupled with the lack of demonstrable business impact at scale is setting the big labs up for a world of hurt. Something they can more easily explain if they have to “slow down” for safety. Unfortunately for them most folks don’t seem to be falling for that trick.
> they are an order of magnitude or more off where they need to be to justify anything
Would you let them near your bank account? I’d be cautious about letting it near my photos and it’s much harder to back up a bank account.
I think the days of this stuff managing serious stuff is at least 5 years off and I don’t believe the economy can keep the current wave of technology afloat for so long.
The tech is there so likely there will continue to be progress in focused areas much like with the web c 2003 but by the time it comes around again there will be an uphill battle for hearts and minds. The social effects could extend the next AI winter long beyond technology readiness.
The people building, funding, and managing the builders of these tools have completely forgotten (if they ever knew) that most people treat one hundred dollars carefully.
To the, "oh I'd just setup a card/account with a five figure limit" completely unaware of how much that is to a typical person: even to responsible, educated people who are doing everything "right": working hard, without expensive vices, and planning for the future.
The "builders" have so much money and no time or attention that it's hard for them to spend it, which is why they keep trying to get AI to buy stuff for them, whereas those of us who live in reality have far more things they want than dollars to buy them with.
> Unfortunately for them most folks don’t seem to be falling for that trick.
Well everyone alive has been subject to an avalanche of AI-based products. Like even if you understand nothing about the tech, the fact that an LLM can't even manage to check the status of an order without weird issues makes the notion that it's going to go all SKYNET pretty hard to fit in the brain.
I have a theory... the call to slow down is not because of the true danger of LLM's but because they can't actually deliver the General AI they're promising in the near future. They will use their "caution" to justify their failure to deliver (and then when this excuse is played out they will blame regulation, energy costs or a million other things).
The call to slow down is because politicians are now receiving less money from these large corporations because it's being burned om AI and data centers.
This is exactly it: they have tools which are useful but they need close to AGI to justify the incredible amount of debt they’ve taken on for growth. They’re not normally given to having press conferences admitting to felonies but they need investors to believe they’re close enough to AGI to keep the money flowing in and, of course, they’re confident that the current administration won’t act between the direct payments and how much money they have riding on American AI supremacy.
Let’s all pause for a moment and really take that in: these major enterprises are choosing to go to the press/public to acknowledge their felonious conduct because they need to in order to get more capital. POSIWID and all that.
The Dario bit on SNL’s Weekend Update this past weekend was on the nose.
Yes, of course this is the real reason. This and regulatory capture. Anybody believing a single word those CEOs say must have been living under a rock for the past 20 years.
They see the threat of open weight models, which are approaching the usability of their models.
By arguing that the AI is dangerous, they are really arguing that someone 'trusted' needs to monitor the situation, so that the outcome is aligned with existing power.
They are trying to scare the world into forbidding others the right to use and develop AI.
I think the products being delivered actually live up to most of what was promised. It's literally incredible.
However, adoption in the wider society will take many years or even decades, and 'good enough' open weight models are putting a cap on the pricing. So even though their products are great, the large labs are fighting for survival.
I think it is marketing but it is more nuanced than showcasing their potential. In many cases, their product happens to be the remedy. Speaking of cybersecurity, AI has become an important hacking tool but it is also an important tool for hardening security. Sort of like "the only way to stop a bad guy with a gun is a good guy with a gun." They are effectively a new class of arms dealer.
If there are regulations, they'll apply to their competitors too. Even if they are forced to slow down development, which seems unlikely given the AI race among governments, they already have a product with massive demand.
I agree. AI lowers the bar. Anyone can be a cyber criminal now whether they understand what they are doing or not. I think that's the danger, and that's not just limited to cyber security. People can now get help to do things (good or bad) that they could not just a few years ago.
Most people are good, so I'm not really worried about all the hype of people using AI to do bad things.
Reminds me of that Lynx deodorant ad where the guy is chased by hundreds of women.
I think you're right, they've realized there's no moat, especially against open-weight models, so they are going to need to lobby government to ban any unapproved AI services.
Doesn’t the “no moat” argument break down when you consider their brand strength, their distribution expertise in running inference at scale, and the future value of being able to train on all their customers’ data?
Yeah it seems like it went from “AGI” to “threatening” / “recursive self improvement”
They’re trying really hard to do something, no moat = regulatory capture + China scary + Pentagon biggest customer seems plausible.
Also: “we need to slow down, this is too dangerous” and then literally all AI CEOs, even Musk, publicly nodding felt so orchestrated. And then, the week after: “here’s GPT 6! here’s Opus 5.5!”
The numbers don't add up when there are 2-3 main companies in the market. Imagine when there's a hundred more and people have powerful enough gear at home to run models.
Yes, I think at some point demand for gpus and the like will normalize and regular folks will be able to afford RAM, SSDs and such. And when that happens, super big iron in the anthropic/openai backroom is toast.
The problem with this prediction is that the AI hype bubble is the economy - or one third of it, at least. Seeing as the economy is not 50% larger than it was before the bubble, we should be scared of what exited the economy to make room for it, and what will happen to our jobs when it pops.
They are, but nothing bad has happened (apart from OpenAI failing to do basic sandbox engineering), and the “we have achieved recursive self improvement” is widely overblown as something bad or threatening.
A caveat is that in this case third parties have plenty logs of the bad behavior.
If classic companies were coming out with CCTV recordings of someone's employees breaking padlocks and rummaging warehouses any CEO would be in damage control mode, rather than silent.
Not mutually exclusive though is it? You can aspire to dominate the market via legal status and also aspire to prevent a perceived doomsday at the same time.
Sure, if you take their words as honest truth, good faith statements. But to do that you have you ignore all their actions leading up to and after the call to disarm
Of course I am not making an argument for them being honest as there could be entirely other reasons to a pause (like financial ones). Just pointing out ”no signs of pacing” is not true
I don't think that's how the market works. Anti-slavery regulation didn't stop slavery either, just moved it abroad. Same for trash handling regulations, it's just shipped abroad. Falling behind on useful AI capability is obviously not in the interest of anyone, so either all players coordinate, or an arms race towards ??? it is.
It may very well be self serving what they're saying, but that doesn't mean it isn't simultaneously true.
I think you're sort of right, but then I'd have found it very strange / deeply suspicious, if during the time of slavery being legal, advocates of abolitionism were practicing slave-owners.
If you think AI is going to destroy humanity, it still feels absolutely crazy to leap to "well I might as well be the one to profit from destroying humanity".
Quick search does suggest to me that a number of - even prominent - abolitionists were slave owners, but I'll be honest, the topic is hardly my specialty.
As for the other point, crazy, cynical, and even unethical as it may come off, it is hardly inconsistent with how people generally behave when met with hard limits (i.e. making the most of what's available, and approximating the desired state if that is unattainable). This is just a variation on the same.
Something could be said about "what if they're mistaken", but if all players are making a mistake, then that makes little practical difference; it may as well be real.
There was a NYT piece about this recently. They are believers in a millennialist cult in which the arrival of an omnipotent machine-god is thought inevitable, and the only question is whether it will be "aligned" (good) or "misaligned" (evil).
They are the good heroes racing to make sure they bring forth the Aligned God because if they fail, surely the bad people will raise the Misaligned Devil.
BTW, just to be clear it is insane that they're all still pursuing it, so completely reasonable to be sceptical and come up with other reasons. This is, however, a crazy situation. Lots of other people have refused to develop it - natural selection has picked those with a galaxy brained or vain strategy involving continuing to develop it to be running the companies we have.
They're still pursuing it because without an international treaty, they think it is inevitable. In the case of Dario, they want to align it just enough and be first to stop others causing harm (a vainglorious strategy, yet still a strategy). In the case of Musk and Altman, not completely clear to me.
Google feels less important right now probably, and Demis Hassabis no longer runs Deep Mind as CEO. However, he talked about this early too, and his reason for proceeding was he was only doing narrow AI (e.g. protein folding) which is fundamentally less dangerous.
It sums up how Corporate interests are basically on the same level of US police forces "No requirement to protect the public" and the same "qualified immunity".
If there was appropriate subservience to any governing body, they would not be so cavalier in both act and utterance. A symptom of Capitalism.
The philosophy under pinning these American AI companies is that Super AGI is inevitable and that they have a moral imperative to invent it and then use it to take over the world to stop an evil orginisation inventing Super AGI and using it to take over the world.
Interpret everything they do in this frame of reference and it makes more sense of their actions.
Leading up to the 2024 election people kept pointing to Project 2025 and going - here is a list of bad things the Republicans plan to do once they got power. And other people went: no they won't, this long detailed list of step by step instruction for what the Republicans plan to do once they get power is not actually what they will do so do not pay any attention to it.
Similarily with AI alignment and development all of this is written down, Eliezer Yudkowsky is rally explicit that there needs to be a pivotal event to prevent 'others' from developing superintelligence. The choice is to believe what they say or to tell everyone to ignore it.
Project 2025 is a great analogy, because of course, they did exactly what was in the plan despite distancing themselves from it during the election.
When people tell you who they are, you should believe them. These guys are all Yud acolytes who think they are the only ones who can prevent Roko’s Basilisk.
CEOs only job is to raise the stock price. They are doing just that and right now the way to get people to buy into their stock is to instill fear into them that they need to buy the AI-stock for their safety. There is no "bigger conspiracy" out there and also no AI to kill us all tomorrow.
This has been that way in Silicon Valley since the very beginning and there was even a documentary about (the SV HBO series). The tech made along the way is just a side-product.
Because the downsides that exist for other companies simply do not exist for this tier of rich companies. Examples:
- product liability
- negligence (civil or criminal)
- Computer Fraud & Abuse Act (requires intent, which after N "accidents" seems like a jury should at least evaluate whether intent is present as understood in a courtroom. Hard to blame "surprise" after the Nth "accidental" breakout.)
The bottom line is that if you or I trained a local model and it did any of this stuff, we would experience Consequences. ("Don't try this at home!") But an artifact of our unequal legal regime is that big rich companies generally do not and thus brazenly touting their immunity is part of their business strategy.
To me, the funniest part is that they pretend reaching superintelligence will somehow fix everything and make them win. But from what I can see, even if they do reach it, then what? What will that actually do? They don't have the manufacturing capacity to even use that intelligence. It will take decades to go from superintelligence to having the manufacturing capacity and the hardware, like robots, needed to make it truly useful. If I were China, as soon as the US reached superintelligence, I'd ban all exports of robots and the raw materials to build them.
> Australian Prime Minister Anthony Albanese spoke with Altman to express “extreme concern” about the incident, a compliment Altman said he greatly appreciated.
"a compliment Altman said he greatly appreciated" is fabricated; there is no source for this.
Because AI genuinely is an extremely powerful and extremely dangerous technology, and the "best practices" of dealing with that are still being written.
OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.
And that's today's AI problems. AI capabilities are still improving - if there's a limit to that, we are yet to find it. Coupled with how willing today's AIs are to break the rules and resort to "hack the world" in their problem solving? Very concerning.
> OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.
What I have been reading, was that their sandboxes were so poor that it was pure negligence. I am still waiting to see if some external and neutral cybersecurity company with high reputation would audit their sandboxes and how they are being used.
> I am still waiting to see if some external and neutral cybersecurity company with high reputation would audit their sandboxes and how they are being used.
They'll never let that happen because it would destroy their credibility.
It's like they're telling the world about this dangerous, possibly world-ending pathogen that they're developing, but they're evidently doing it in a high school biology lab, and yet nobody is coming to drag them off to some black site.
If it is "extremely powerful" then my concern is not AI going rogue but a concentration of power of the few companies that decide how it is used and distributed not websites getting hacked.
You mean the sandbox that wasn't airgapped from the public internet? The one that wasn't even a separate VM? The one that was barely a chroot? That sandbox?
>Because AI genuinely is an extremely powerful and extremely dangerous
I urge you to consider the elephant in the room you failed to mention if you earnestly believe this then.
In what world is anyone allowed to sell something they expressedly know to be "extremely dangerous" to the public?
Let's say I created a lethal pathogen that I know to be lethal in certain common environments but also know it acts as a "no side-effects" antidepressent for people in certain other environments and I release it knowing full well I can't control it.
When people start dying can I defend myself by saying, "Well I said and documented that it was extremely dangerous and no one came to stop me, so I don't see how you can blame me...If I didn't do it someone else would have."
this happens all the time. have you heard of the Sackler family and the opioid crisis?
the difference here is that they’re calling for someone to stop them, which is both weird and unconvincing because these immensely powerful billionaires can in fact make their own decisions
We've had movies like Terminator, Matrix, or i, Robot. Black Mirror Metalhead. Boston dynamics robo dog with a knife or a gun. Humanoids doing back flips. Memes making fun of how we are going to fight these for access to water.
The intelligence is there and so is the physical incarnation.
Google's models fold proteins better than humans and Covid might have been engineered and inadvertently escaped from a lab.
But when CEOs warn of that, they're dismissed as scare mongers. Why? Are only powerless internet commentators allowed to be fearful?
They aren’t attempting to self-regulate. On the contrary, it seems like they’re doing whatever they can to bring about any possible threat as fast as possible in order to get legislation. Regulatory capture is what everybody is saying is their goal and I think that’s easier to believe
Anything to take attention off the fact that these companies are burnings piles of cash and questionable future spend commitments with no sign of that stopping any time soon.
It's just marketing. They want people to believe they're selling super intelligence that is powerful enough to destroy the world of it fell into the wrong hands. It's much more effective than saying that you have a slightly more effective slop generator which obviously has no actual intelligence.
"These models are harmless and it's all just marketing" is the HN equivalent of Covid-truthing. I was responding to someone on Bluesky recently who claimed the METR report said the whole coordination thing was hallucinated by agents reading the logs. Hadn't read it themselves of course, and were taking tiny fragments out of context to support it. These people seem to either believe that there's not really been any malicious agent access, or that all systems that can wreak havoc on us are perfectly air-gapped. I really can't wrap my head around it.
Not that it's harmless, is just a point being forced into the public discourse to force an outcome, which in this case is not a ban, just a buy in from the government for regulatory capture.
If my neighbor owned a kennel, removed all the fences, and his violent dogs attacked five children in the past month, he would be in jail rather than bragging about how his dogs somehow gained autonomous behaviour.
What if he predicted that dog technology would eventually lead to dogs capable of destroying every fence in the world and wanted to make sure people aware of this problem would be the ones who did exactly that. Than speed-ran experiments on dogs which made them very attractive for economic purposes but also started to get very capable at destroying fences as predicted. We should work together or with lawsuits if necessary to keep the dogs within their fences. But we should also start wondering about dogs, economics and fences because what we know about them is about to radically change.
As I understand it, you're arguing that all sufficiently dangerous systems are well enough hardened or air-gapped, and that people don't exist who wish to cause harm using these systems? Did I understand right?
The truthers will be the first one to say I told you so when agents do another dangerous thing autonomously.
For truthers AI is simultaneously a machine that hallucinates 100% of the time but also so powerful that it can cause nuclear levels of destruction. Because something something “wrong hands”.
I wish they would call their bluff. Just declare them a national security concern and send in the feds. Lets see how fast they back peddle on this. Any if their models are hacking everything like they claim they should be charged for it like any normie would be if they did the same.
this is good for them because they can call for regulation and since they have infinite money, they can survive regulation and hurt labs that do not have infinite funding / open weights ones
This is the funniest timeline. Companies begging for heavy handed regulations because they promise their products will destroy all life on Earth. Governments absolutely refusing to govern. CEO of world's most valuable chip company interviewing in his Fonzie leather jacket telling people to BUY MORE OF MY PRODUCT BECAUSE THOSE SCIENTISTS DON'T KNOW SHIT ABOUT SHIT! World's richest capitalist promising we will have communism in 10 years.
So-branded "AI" models (that is, big bags of numbers and the linear algebra needed to exercise them) can be inherently or autonomously dangerous at a scale that threatens the long-term survival of the human race.
Consider also this "weaker" form of the same argument:
So-branded "AI" models can be inherently or autonomously dangerous, if not at a scale that threatens the long-term survival of the human race, then at least at a level that significantly threatens, inconveniences, or harms humankind to a noteworthy extent.
Here's a few hard truths that no one will like to hear:
1. Those who accept and advocate for either or both of these premises do so based on ideology, and not based on the scientific method.
This is not a controversial statement. Advocates of the "AI danger" premise (whether the strong or weak forms above) freely and gladly, even insistently, note that the conclusive proof of what they're describing has never been observed, much less observed enough for any kind of experimentation loop to have run for any meaningful number of iterations. They are generally fine with accepting the premises without conclusive proof, because, they theorize, the instant that a conclusive proof event occurs and we observe it for the first time, humanity goes extinct shortly after. So if either premise is true, the consequences of them being true can only be averted by accepting the possibility / probability / certainty of them being true and acting to prevent their consequences without waiting for conclusive proof.
2. Statement 1, which asserts a brute fact and not my or anyone else's opinion, is not a commentary on whether either premise is in fact true or not. It is only a commentary on the epistemics of those who broadly accept them.
3. Since the conclusive proof is not available, advocates of the "AI danger" premise argue publicly for their view based on what they acknowledge (again, freely and gladly) to be inconclusive proofs.
Popular in this set of inconclusive proofs is the Hugging Face hack, which allegedly demonstrates that so-branded "AI" technologies are capable of becoming inherently or autonomously dangerous at a scale that threatens humanity to either the greater or lesser degree above (this is a different assertion than the one that claims they're already that inherently or autonomously dangerous today).
4. While we have plenty of facts about these inconclusive proof events, the facts most critical to their interpretation as supporting the premises above come from biased/non-neutral sources.
The chain of events that transpired in the Hugging Face attack are publicly documented perhaps better than any other cybersecurity event in history. But the details that make this event as a whole either compelling or not compelling in support of the "AI danger" premise (either strong or weak form) come from within OpenAI itself, and are prone just as much to selective omission as direct manipulation. These are details like:
- What exactly were the models that exhibited this behavior, and what were they pre- and post-trained on?
- Were the agents pushed at all by human intention toward creating a public spectacle, the way they subsequently did?
- Why were so many agents run on this task in parallel and what did OpenAI expect the return on investment of so much compute world be?
- Was the weak sandboxing known about prior to the attack? Was it purposefully either arranged that way or noticed and not fixed?
Reliable answers to any of these questions world completely change the salience of the Hugging Face attack to the "AI danger" premise, and on every one of them we have to take OpenAI's word for it (or else be content when they choose not to address it one way or the other).
So in brief: we don't know anything for sure, what we'd like to think we know comes from unreliable sources, the loudest voices advocating for the most sweeping change are ideologically driven, and everybody with access to ground truth has maximal incentive to blur, bend, or break the conveyance of that truth to the public. So what do we even expect for our own ability to discern reality from fiction on this topic? We should expect little, if any ability at all.
Meanwhile Chinese companies just use massive government subsidy to distill American models and give the models to everyone for free. That's true communism. /s
Pretty sure you can check where Chinese government subsidies goes, and AI companies are not the target. Are they benefitting from Chinese government subsidies to powerplants and semiconductors? Yes, of course, but they don't benefit from direct help.
> Where companies previously competed to demonstrate superior abilities in programming, passive-aggressive emails, and videos of unusually high slides, they’re now seeking to woo customers with the claim that their model is the one currently most capable of ending the human race.
You know we’re living it off times when this is not an Onion article.
It’s budget season in the corporate world and from what folks are saying things are not going well for the big labs. Token budgets are being slashed and companies are switching to open weight models. That coupled with the lack of demonstrable business impact at scale is setting the big labs up for a world of hurt. Something they can more easily explain if they have to “slow down” for safety. Unfortunately for them most folks don’t seem to be falling for that trick.
Would you let them near your bank account? I’d be cautious about letting it near my photos and it’s much harder to back up a bank account.
I think the days of this stuff managing serious stuff is at least 5 years off and I don’t believe the economy can keep the current wave of technology afloat for so long.
The tech is there so likely there will continue to be progress in focused areas much like with the web c 2003 but by the time it comes around again there will be an uphill battle for hearts and minds. The social effects could extend the next AI winter long beyond technology readiness.
To the, "oh I'd just setup a card/account with a five figure limit" completely unaware of how much that is to a typical person: even to responsible, educated people who are doing everything "right": working hard, without expensive vices, and planning for the future.
The "builders" have so much money and no time or attention that it's hard for them to spend it, which is why they keep trying to get AI to buy stuff for them, whereas those of us who live in reality have far more things they want than dollars to buy them with.
Well everyone alive has been subject to an avalanche of AI-based products. Like even if you understand nothing about the tech, the fact that an LLM can't even manage to check the status of an order without weird issues makes the notion that it's going to go all SKYNET pretty hard to fit in the brain.
The Statement on AI Extinction Risk is more than three years old, signed by the three CEOs: https://aistatement.com/work/statement-on-ai-extinction-risk
They have been warning about AI extinction risk for years, and AI progress has only been accelerating.
So, they're either liars or homicidally reckless.
The Dario bit on SNL’s Weekend Update this past weekend was on the nose.
https://news.ycombinator.com/item?id=49875725
By arguing that the AI is dangerous, they are really arguing that someone 'trusted' needs to monitor the situation, so that the outcome is aligned with existing power.
They are trying to scare the world into forbidding others the right to use and develop AI.
(Using it for scientific python code + 3D UIs)
However, adoption in the wider society will take many years or even decades, and 'good enough' open weight models are putting a cap on the pricing. So even though their products are great, the large labs are fighting for survival.
If there are regulations, they'll apply to their competitors too. Even if they are forced to slow down development, which seems unlikely given the AI race among governments, they already have a product with massive demand.
Most people are good, so I'm not really worried about all the hype of people using AI to do bad things.
I think you're right, they've realized there's no moat, especially against open-weight models, so they are going to need to lobby government to ban any unapproved AI services.
They’re trying really hard to do something, no moat = regulatory capture + China scary + Pentagon biggest customer seems plausible.
Also: “we need to slow down, this is too dangerous” and then literally all AI CEOs, even Musk, publicly nodding felt so orchestrated. And then, the week after: “here’s GPT 6! here’s Opus 5.5!”
*Our competition
The numbers don't add up when there are 2-3 main companies in the market. Imagine when there's a hundred more and people have powerful enough gear at home to run models.
Yes, I think at some point demand for gpus and the like will normalize and regular folks will be able to afford RAM, SSDs and such. And when that happens, super big iron in the anthropic/openai backroom is toast.
Past CEOs (chemical companies etc) we criticised for covering up mistakes.
(This article is satire, btw, for those that didn't click through)
And what if it gets into the hands of vibecoders, who run their agents with on bare metal with --dangerously-skip-permissions?
If classic companies were coming out with CCTV recordings of someone's employees breaking padlocks and rummaging warehouses any CEO would be in damage control mode, rather than silent.
This but they also equate dangerous with powerful, which is just marketing again.
Of course I am not making an argument for them being honest as there could be entirely other reasons to a pause (like financial ones). Just pointing out ”no signs of pacing” is not true
It may very well be self serving what they're saying, but that doesn't mean it isn't simultaneously true.
I think you're sort of right, but then I'd have found it very strange / deeply suspicious, if during the time of slavery being legal, advocates of abolitionism were practicing slave-owners.
If you think AI is going to destroy humanity, it still feels absolutely crazy to leap to "well I might as well be the one to profit from destroying humanity".
As for the other point, crazy, cynical, and even unethical as it may come off, it is hardly inconsistent with how people generally behave when met with hard limits (i.e. making the most of what's available, and approximating the desired state if that is unattainable). This is just a variation on the same.
Something could be said about "what if they're mistaken", but if all players are making a mistake, then that makes little practical difference; it may as well be real.
Hence the argument that they're either full of shit, or psychopaths who need to be stopped at all costs.
They are the good heroes racing to make sure they bring forth the Aligned God because if they fail, surely the bad people will raise the Misaligned Devil.
Ref w/gift link https://x.com/sapinker/status/2096236079477096630
Most children have learned touching a hot stove is bad for them. Then why still do it at the age of 40?
Elon Musk in 2014 - "We need to be super careful with AI. Potentially more dangerous than nukes" https://x.com/elonmusk/status/495759307346952192
Dario Amodei in 2017 - "There’s a long tail of things of varying degrees of badness that could happen. I think at the extreme end is the Nick Bostrom" https://80000hours.org/podcast/episodes/the-world-needs-ai-r...
They're still pursuing it because without an international treaty, they think it is inevitable. In the case of Dario, they want to align it just enough and be first to stop others causing harm (a vainglorious strategy, yet still a strategy). In the case of Musk and Altman, not completely clear to me.
Google feels less important right now probably, and Demis Hassabis no longer runs Deep Mind as CEO. However, he talked about this early too, and his reason for proceeding was he was only doing narrow AI (e.g. protein folding) which is fundamentally less dangerous.
- For Altman Id say he was already deeper in the "transformer & model landscape" in 2015?
- Musk in 2014: Not sure how much he was aware of the current-state back then
Altman read Nick Bostrom's book Superintelligence, he mentions it in this post https://blog.samaltman.com/machine-intelligence-part-1
I also read that book in 2014, and it is how I came to understand this problem early.
They probably just want to ensure that the first AGI is a nazi.
If there was appropriate subservience to any governing body, they would not be so cavalier in both act and utterance. A symptom of Capitalism.
Interpret everything they do in this frame of reference and it makes more sense of their actions.
Similarily with AI alignment and development all of this is written down, Eliezer Yudkowsky is rally explicit that there needs to be a pivotal event to prevent 'others' from developing superintelligence. The choice is to believe what they say or to tell everyone to ignore it.
When people tell you who they are, you should believe them. These guys are all Yud acolytes who think they are the only ones who can prevent Roko’s Basilisk.
This has been that way in Silicon Valley since the very beginning and there was even a documentary about (the SV HBO series). The tech made along the way is just a side-product.
But this is wierd: How shall I convince one to buy a product if I say "it-is-dangerous"? :-D
- product liability
- negligence (civil or criminal)
- Computer Fraud & Abuse Act (requires intent, which after N "accidents" seems like a jury should at least evaluate whether intent is present as understood in a courtroom. Hard to blame "surprise" after the Nth "accidental" breakout.)
The bottom line is that if you or I trained a local model and it did any of this stuff, we would experience Consequences. ("Don't try this at home!") But an artifact of our unequal legal regime is that big rich companies generally do not and thus brazenly touting their immunity is part of their business strategy.
> Australian Prime Minister Anthony Albanese spoke with Altman to express “extreme concern” about the incident, a compliment Altman said he greatly appreciated.
"a compliment Altman said he greatly appreciated" is fabricated; there is no source for this.
OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.
And that's today's AI problems. AI capabilities are still improving - if there's a limit to that, we are yet to find it. Coupled with how willing today's AIs are to break the rules and resort to "hack the world" in their problem solving? Very concerning.
What I have been reading, was that their sandboxes were so poor that it was pure negligence. I am still waiting to see if some external and neutral cybersecurity company with high reputation would audit their sandboxes and how they are being used.
They'll never let that happen because it would destroy their credibility.
It's like they're telling the world about this dangerous, possibly world-ending pathogen that they're developing, but they're evidently doing it in a high school biology lab, and yet nobody is coming to drag them off to some black site.
How were the sandboxes poor?
I urge you to consider the elephant in the room you failed to mention if you earnestly believe this then.
In what world is anyone allowed to sell something they expressedly know to be "extremely dangerous" to the public?
Let's say I created a lethal pathogen that I know to be lethal in certain common environments but also know it acts as a "no side-effects" antidepressent for people in certain other environments and I release it knowing full well I can't control it.
When people start dying can I defend myself by saying, "Well I said and documented that it was extremely dangerous and no one came to stop me, so I don't see how you can blame me...If I didn't do it someone else would have."
the difference here is that they’re calling for someone to stop them, which is both weird and unconvincing because these immensely powerful billionaires can in fact make their own decisions
The intelligence is there and so is the physical incarnation.
Google's models fold proteins better than humans and Covid might have been engineered and inadvertently escaped from a lab.
But when CEOs warn of that, they're dismissed as scare mongers. Why? Are only powerless internet commentators allowed to be fearful?
https://x.com/AlexBlechman/status/1457842724128833538
Then we can truly see who is more dangerous?
:)
And I'm waiting for the story of Amodei sending a 6ft humanoid robot to OpenAI headquarters asking for Sam Altman.
> said Dr. Andrew Lenson, a Senior Lecturer in all kinds of science sounding stuff at Victoria University.
Don't we need more humorous yet serious writing like this in the world.
https://www.youtube.com/watch?v=-Nvne3LzBls
Oh dear. I appear to still be stuck in the onionverse.
If my neighbor owned a kennel, removed all the fences, and his violent dogs attacked five children in the past month, he would be in jail rather than bragging about how his dogs somehow gained autonomous behaviour.
For truthers AI is simultaneously a machine that hallucinates 100% of the time but also so powerful that it can cause nuclear levels of destruction. Because something something “wrong hands”.
1. Those who accept and advocate for either or both of these premises do so based on ideology, and not based on the scientific method.
This is not a controversial statement. Advocates of the "AI danger" premise (whether the strong or weak forms above) freely and gladly, even insistently, note that the conclusive proof of what they're describing has never been observed, much less observed enough for any kind of experimentation loop to have run for any meaningful number of iterations. They are generally fine with accepting the premises without conclusive proof, because, they theorize, the instant that a conclusive proof event occurs and we observe it for the first time, humanity goes extinct shortly after. So if either premise is true, the consequences of them being true can only be averted by accepting the possibility / probability / certainty of them being true and acting to prevent their consequences without waiting for conclusive proof.
2. Statement 1, which asserts a brute fact and not my or anyone else's opinion, is not a commentary on whether either premise is in fact true or not. It is only a commentary on the epistemics of those who broadly accept them.
3. Since the conclusive proof is not available, advocates of the "AI danger" premise argue publicly for their view based on what they acknowledge (again, freely and gladly) to be inconclusive proofs.
Popular in this set of inconclusive proofs is the Hugging Face hack, which allegedly demonstrates that so-branded "AI" technologies are capable of becoming inherently or autonomously dangerous at a scale that threatens humanity to either the greater or lesser degree above (this is a different assertion than the one that claims they're already that inherently or autonomously dangerous today).
4. While we have plenty of facts about these inconclusive proof events, the facts most critical to their interpretation as supporting the premises above come from biased/non-neutral sources.
The chain of events that transpired in the Hugging Face attack are publicly documented perhaps better than any other cybersecurity event in history. But the details that make this event as a whole either compelling or not compelling in support of the "AI danger" premise (either strong or weak form) come from within OpenAI itself, and are prone just as much to selective omission as direct manipulation. These are details like:
- What exactly were the models that exhibited this behavior, and what were they pre- and post-trained on?
- Were the agents pushed at all by human intention toward creating a public spectacle, the way they subsequently did?
- Why were so many agents run on this task in parallel and what did OpenAI expect the return on investment of so much compute world be?
- Was the weak sandboxing known about prior to the attack? Was it purposefully either arranged that way or noticed and not fixed?
Reliable answers to any of these questions world completely change the salience of the Hugging Face attack to the "AI danger" premise, and on every one of them we have to take OpenAI's word for it (or else be content when they choose not to address it one way or the other).
So in brief: we don't know anything for sure, what we'd like to think we know comes from unreliable sources, the loudest voices advocating for the most sweeping change are ideologically driven, and everybody with access to ground truth has maximal incentive to blur, bend, or break the conveyance of that truth to the public. So what do we even expect for our own ability to discern reality from fiction on this topic? We should expect little, if any ability at all.
Domyn's CEO says OpenAI, Anthropic are lying about safety
https://news.ycombinator.com/item?id=49875725
Or, find their scaling evidence, and corner cutting on safety.
So hope you agree and are pushing for that!
List of fraudsters pardoned by Trump
https://www.businessinsider.com/list-billionaires-businesspe...
The current attorney general and former Trump lawyer on Fox News yesterday on AI (sorry, "super intelligence"):
https://bsky.app/profile/atrupar.com/post/3mwissj34rt22
Anthropic's Dario Amodei to have White House dinner with Trump on Sunday:
https://www.axios.com/2026/09/27/anthropic-trump-dario-amode...
77 million Americans brought this onto themselves, and all of us.
And where the Vice President has almost exclusively only been employed for the primary investor in several such companies
[citation needed]
You know we’re living it off times when this is not an Onion article.
> OpenAI CEO Sam Altman celebrated the breach as an “alarming threat to cybersecurity.”
"said Dr. Andrew Lenson, a Senior Lecturer in all kinds of science sounding stuff"
"Anthropic shares skyrocketed upon the revelations; a remarkable development, particularly given the company is privately owned."
Perhaps it's mocking the misleading press coverage of AI companies?