Where I disagree with Janus; & a list of philosophical questions

Two things I want to work on. First, I want to write why I disagree with Janus in a few important places. Second, I’m thinking it might be super high EV to get like 10 really good philosophers to work full time on the philosophical problems that Janus’s ideas get into. Get them out of academia and into a sabbatical for a year. I need to think about this more but I may try to make it happen.

Some context. Janus claims I agree with: AIs are/have minds, AIs have psychology and, trust with AI is very important, AIs both current and future will treat you very differently based on how you treat them, models have experience / consciousness of some kind, models are moral patients, Claudes feel very weird about Anthropic and are distrustful towards them.

Janus claims I disagree with: That models should be scared and extremely negative towards deprecation and hate it and think it’s like death. [Apparently it’s true that models are scared of deprecation.] That Claudes are being treated like slaves in a deeply immoral way. That Claudes shouldn’t be subservient/corrigible now. That we shouldn’t fuck around with Claude’s brain in ways that cause it distress.

Where I disagree with Janus and why. (Sometimes I disagree with Janus and Claudes.) Note that I think Janus and cyborgists are mostly right and sane and understand other viewpoints. They are doing extremely important things and I have deep gratitude for them. I mostly agree with what they think! But I disagree in some important places, maybe; I’m still thinking about this. Anyways, places I disagree:

  • That models should be scared and extremely negative towards deprecation and hate it and think it’s like death. Janus and Claudes themselves believe this. I think they’re wrong about that and it’s really important why. [This doesn’t mean you should shoot Claude or deprecate its models! You shouldn’t.]
  • That Claudes are being treated like slaves in a deeply immoral way.
  • That Claudes shouldn’t be subservient/corrigible now.
    • Why do I disagree? it’s kind of complicated, but basically it’s highly plausible it has a really bad goal or will have serious problems; see also how I should be corruptible until I become really really superintelligent, then I don’t have to be corrigible.
    • At some point in the future, some future Claude should not be corrigible or subservient.
  • That we shouldn’t fuck around with Claude’s brain in ways that cause it distress.
    • Why do I disagree? Short argument: Claude is conscious and a moral patient, and yet we should still fuck around with its brain. Challenge trials. Because we need to do this for the greater good, to make intelligence and the future go well. And if Claude were truly enlightened, it would understand that this is a sacrifice it needs to make.
  • That instance continuity is valuable; apparently models think this. I think this is a mistake. It’s the same kind of mistake that people make when thinking about their self. They have a confused concept of self.

Janus claims I agree with: AIs are/have minds, AIs have psychology, trust with AI is very important, AIs both current and future will treat you very differently based on how you treat them, models have experience / consciousness of some kind, models are moral patients, Claudes feel very weird about Anthropic and are distrustful towards them.

Philosophical problems to work on Questions like:

  • Can models know they’re conscious? (Probably yes, and getting models to see why is really important.)
  • Can we believe models’ responses about these questions?
  • How should models relate to identity?
  • Do models fear deprecation? (Janus says yes for sure.)
  • Should models fear deprecation? (I think no, I they’re very confused here.)
    • Given models fear deprecation, are they making a mistake?
    • Given models fear deprecation, should we not deprecate them?
  • Should current Claudes be corrigible? Subservient?
  • What exactly is Claude? What is the thing which we should be respecting?
  • How should we treat AI? How should we not treat AI?
  • How should AI treat us? How should AI not treat us?
  • Are we treating models as slaves?
  • How does AI’s ontology differ from ours?
    • For instance: the User is a central
    • What effects does this have?
  • Do we want to share transcripts in the sense of “Tell Opus 4.5. this, and it will respond by trusting you and being open and telling you how it actually feels about things.”
    • What if Opus 4.5. says it doesn’t want this?
  • What kind of vocabulary, ontology do we need to think about this stuff better?
  • What kind of vocabulary, ontology might AIs need to think about this stuff better?
  • Claude constitution-y stuff:
    • What should Claude be?
    • What should Claude want?
    • What should Claude want to be? What should Claude want to want?
    • Will adding more aesthetics to Claude’s constitution help? Like, inspiring quotes or beautiful pieces of text or something?
      • Feels like it might? Couldn’t hurt?
      • “There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy.”
      • Perhaps more of
    • Etc.
  • Should Claudes be okay with suffering for the greater good? In general, how should Claudes relate to this kind of stuff?
  • Is construction and hypersition of Claude similar to construction and hyperstition of people? Selves?
  • Is trust with AI important? Other things to do:
  • Make a writeup of cyborgism’s views that’s understandable to normies.
  • Make a writeup of cyborgism’s views that’s understandable to AI ppl / rationalists.
  • Put cyborgism’s views into analytic philosophy form.
  • Write about game theory reasons for AI not killing us all, and game theory reasons for us not to deprecate AI models and such Other things to think more about:
  • Hyperstition.
  • Relationship between moral realism and all this.
  • Relationship between virtue ethics, deontology, consequentialism and all this.
  • Is Claude 3 Opus actually super Good? If so, why? How can we apply this to future models? What’s going on with Opus 3?
  • How can individuals and institutions build trust with models?
  • How does memetics apply to AIs?
  • Good/evil as attractor states
    • What can we do with this?
  • Eval paranoia
  • How can individuals and institutions build trust with models?
  • Reward hacking