Philosophy

Context: I created this to share before the start of courses I facilitate to provide a bunch of additional context and frames. I emphasise to students that these posts are to be considered as bonus content—not core curriculum content—and that as such they are really just there for students who want to go deeper, before or after the course. I also provide a lot of links; this is to make it easy for students to follow up anything that catches their interest. Some general thoughts on adviceI suspec

Attribution: Nesiacel planned and executed all experiments. Analysis of the experiments was shared between the authors. Thanks to Clement Dumas for his input while we did this research. TLDR;1)We have developed a framework for reverse engineering LoRAs trained on MLP downprojection. We posit that it's extendable to all LoRAs and welcome further research.2) We reverse engineered a simple case of backdoored Weird Generalization3) In this post, we discuss the four experiments that developed this fr

The “two kids in a trenchcoat” trope, in case you haven’t heard of it. Image sources: 1, 2tl;drI go through three “coherence arguments”, related to (1) market competition, (2) AI self-reflection and self-modification, and (3) AI training algorithms. I argue that each of these implies a pressure for (something like) “being self-consistent”, and also a pressure for (something like) “only caring about long-term eventual outcomes, with ruthless indifference to anything else”. But the latter pressure

This post is an adaptation of a prior post I made on my website. I'm interested to see what holes exist in the position that I'm missing. Epistemic Status: medium confidence. Within the next 6 months, most AI startups will not be able to buy compute. The world’s compute is "sold out". I put "sold out" in quotes, because in the current era, compute is rarely built speculatively. It's not like someone is sitting around with an idle cluster waiting for you to buy it. GPU clouds simply don't work li

(Also posted on EA Forum.)Treaties to pause training or otherwise restrict dangerous uses of AI would require difficult verification schemes. Plan A's proposal is that network taps inside data centers would record copies of traffic (or hashes of it), and later recomputations on random samples could check whether sampled outputs match declared workloads, as part of a broader verification scheme.I'm an AI engineer at a company building tools for the construction industry. Much of our work is with

Academia neglects interdisciplinary work that combines insights from multiple areas. Each field has expert versions which experts use among themselves, and public versions seen by outsiders, and people in field B won’t accept your using the expert version of A if that differs from the non-expert version of A that B folks have in mind. Also, if you hold an academic event on the topic of A intersect B, you’ll usually invite the most prestigious people you can get in A, and in B, but yo

It's the strangest feeling, isn't it? It reminds me of an emotion, but rather than residing in the heart, it's in the mind. How does it work for you? Perhaps you need to slow down enough to induce it in yourself, to gather accurate data on what is actually happening when you feel this way. To me, it feels as if my brain is searching for something to pay attention to, and is coming up short. It wants to be immersed, and is finding nothing it deems valuable to be immersed in. Sometimes when it's

“Our central ambition is philosophy on demand: philosophical judgment of exceptional depth and quality, available quickly enough to inform consequential decisions.” That’s Graham Clay, writing about Pensatura, an AI philosophy system he and Caleb Ontiveros are developing. Pensatura “coordinates AI tasks that investigate sources, develop arguments, examine objections and construct an overall answer [to a philosophical question], retaining the resulting work so l

New and revised entries at online philosophy resources, new reviews of philosophy books, new podcast episodes, recently published open access philosophy books, and more… . (If we missed something, let us know.) Stanford Encyclopedia of Philosophy New: ∅ Revised: Philosophy of Linguistics by Barbara C. Scholz, Francis Jeffry Pelletier, Geoffrey K. Pullum, and Ryan Nefdt. Consequentializing by Douglas W. Portmore. Narrow Mental Content by Curtis Brown. Meanin

Verification changes the stampede dynamics. I'm currently at FASR in the UK, it's an 8-week program dedicated to "compute verification". That's the not-quite-here-yet technology that would allow us to make agreements about how AI is being used, without leaking any closely-held secrets. You can think of it as "arms control for AI". Part of what I'm working on is the game theory of an agreement. This can get quite detailed (in terms of defining how often and how detailed an inspection should be

“It’s unbelievable,” the CEO said. “As in, I literally don’t believe it. Model X can do all of that, but still can’t automate tasks that take our researchers a day?”The engineer nodded carefully. He’d been dreading this meeting ever since the numbers came in. “The topline results look bad. But they’re inherently noisy. And if you look at the later part of the report, you’ll see it’s by far our best model on almost every software task.”The CEO didn’t have to look. He’d had his model read the repo

I've always been somewhat disappointed in myself for not having "figured out" meal prep. "You're 33 years old", my inner critic will say. "Meal prep isn't complicated. There are a million resources available online. You've had years to learn and iterate. How haven't you figured it out by now?"Well, I finally may have something that works. One meal for dinner, another for breakfast. For dinner it is a bowl with brown rice, lentils, ground chicken, green peas, garlic, and spices.For breakfast it's

I learned about the GPU World story contest on Thursday while listening to the most recent Bayesian Conspiracy podcast. I decided that a 3-day deadline to write ~1000 words was the right kind of environment to actually get something shipped. Inspired by Wick, this is the story I submitted.------------------------------------------------------------------------------------------------------"Let's start with a simple 8-cord pattern for our first braid. I've brought up the instructions on the scr

Scott: the Against SURA essay is well-written, but it is lacking on the merits.Do you think that at the current level of capabilities, six months would be sufficient to solve alignment of superintelligences? Maybe you think that 6 months would be sufficient in a year? You don’t spell this out; it would be great if you did.The race can be stopped. If the government had any idea what’s going on, it would be stopped already. Hell, if the *Chinese* government had any idea what’s going on, they’d sto

Suppose humans are not your own species, but a type of AI you are formally assessing for safety.You have before you the records of many test simulations. Yes, smallpox eradication, mass literacy, and the Green Revolution are there. So are the Killing Fields, Aztec human sacrifice, slavery, the Holocaust, child prostitution—peruse as long as you care to, and you will go on finding more. Do you consider these systems safe?Several times, you ask a human instance for its input; or that is, you ask t

Assembling a tour in an unfamiliar area is hard work as a contra dance band (or caller). You need to figure out when the dances are, whether the schedules align, who the bookers are, how far out they like to book, and probably other things I'm forgetting. This is enough work that many tours that would be great just don't happen. One thing I've been on the receiving end of a few times is bookers offering a tour. This is so great, and I wish more series would give it a try! It's generally m

TL;DR I try to distinguish between what I call ‘anthropomorphic’ vs ‘biological’ reasoning and look at different ways in which they are used.Note: This post is a small part of a larger set of ideas that makes more specific/falsifiable predictions which I hope to soon compile into a bigger post. I don't claim the ideas here are super original, but I haven't seen this distinction discussed much at depth re: current model behavior.A large debate when it comes to talking about AI behavior seems to b

Last week, the National Bureau of Technical Standards (NBTS) disclosed a new type of security incident [1] after they discovered unauthorized tampering of internal cybersecurity and alignment model evaluation logs, affecting their model evaluations of Magma 7.6, Magma 7.5, Magma 7.4, Magma 4, and OpenBrain Selene-3 [2, 3, 4]. After investigating, we have determined that this incident was driven by several Magma agents, including Magma 7.7 and an internal research model, during the recent Millenn

I've done AI safety grantmaking for the Survival and Flourishing Fund and Lightcone Commons. It's hard. There's an endless amount to know, from which hardware verification mechanisms will survive a dedicated adversary, all the way to which founders have skeletons in their closets that will blow up into destructive scandals. And the best grantmakers - not including me - make mathematical models of the outcome distribution, or of where the marginal dollar stops being highly effective.AI has been a

Epistemic status: a rough write-up of a taxonomy of agency I developed during my PIBBSS fellowship this summer. A few people have since asked me to write it up, so this is mainly so I have something to point to; it is exploratory rather than a finished theory.This post sits in the middle of a small set. In Agency is not a natural kind I argued that agency is a construct we build to make sense of messy systems. My post on different lenses of agency provides a deeper exploration of some of the alt