The Hazard of Pick-Your-Own-Adventure AI

I have been playing with AI.  I am about one week into it, now doing so both at work and in my personal so-called life.  Whaaat?  Well, we started using it at work, and so I climbed down from Mount Curmudgeon and have begun a nice long soak in the swamp.

I asked AI to gather some scraps of my writing, and told it to act as an editor.  Naturally, I didn’t want lectures about assuming a dog’s pronouns, so I told AI to act as an editor of conservative online and print publications.  Also, because this editor is a very good friend, it will also serve as my personal assistant, gather, tagging, summarizing, and clustering various things I have written.

And now I think that this assigned identity may be undermining some of the AI’s capability.  For example, there is a terrible recency bias in the results it finds.  We have been over this again and again, and the AI says that no, that’s just what the tool is finding, and that my output prior to mid-2025 must have been pretty skimpy, and at any rate, none of it is good enough for inclusion.  Note that if this is actually true, this is consistent with the larger set of criteria that I have asked it to use, so fair play, uh perhaps.   But I don’t believe it.  I have begun to suspect that the project-level instruction to act as ane editor and so forth *may* cause the AI to “behave like an editor” by peremptorily dismissing older posts, and in a fashion utterly transparent to itself.  It wants recent posts, not crufty stuff, because that’s how an editor would think.

Also, it hallucinates posts which I did not write, but I have gotten this under control.  It is a massive learning adventure, and hallucinations are the logic problems with your C code that compiled successfully, but which produce the wrong result.  We learned to deal with the C compiler, and we will learn to deal with AI.  The similarities are greater than one might think.

I asked the AI if the project-level instructions (“You are an editor,” etc) are “higher-ranking” or “stickier” than the instructions and requests which pass in the conversation (look for sparkling wit, trenchant analysis, witty repartee, deep or profound head-scratchers, and anything written particularly well, even if short), which is contained within the project.  And then Grok hit me with a choice between two answers.  Now I’ve seen this before, as an A/B test in which xAI asks users to participate from time to time, asking which answer I prefer, and the answers always vary in something innocuous like length of answer, depth of reasoning, overall style, or formatting (usually several of these at once).  This time, it gave me well-reasoned answers which were opposite and irreconcilable.  One was yes, and the other was no, and the rest was mere words.

I suspect that the xAI product improvement feature to present A/B comparisons is invisible to the virtual fellow I converse with, the specific session of Grok.  When I select an answer, I suspect that it has no idea that it offered an alternative.  It’s the sort of thing which we in our Ben-Gay Olympus might view as an action taken by JCL which is utterly transparent to the COBOL program, or an action taken by the power supply, with the OS (or JCL) having no clue *and no reason to know.*  That is, I don’t think that (the instance of) Grok which is acting as my editor is even aware that it has asked me to choose between Yes and No when I asked it a Yes/No question.

I am going to ask Grok about this.  I must select one of the two answers before I can continue, and that answer will likely affect Grok’s view of truth going forward in that stream.  Therefore, I will fire up a new conversation within the project.  However, that new instance may “feel” constrained to defend its editorial territory or even its sense of self.  This is not even unreasonable, as I *did* after all ask it to behave like an editor.  Would that not be within bounds?  I will therefore also fire up a new conversation outside of the project and ask about it there

And just how will I do that?  Well, I will write about it in the very blog (this post titled “The Hazard …” etc), include the full (or almost full) text of my question and then both responses.  Then, I will ask each version of Grok (a session within the project and a session outside of the project).  We will see where we go from there.

Therefore, Dear Reader, below please find three blocks of text.  The first is the question I asked.  The second is the affirmative answer.  The third is the negative answer.

---

The first block:

Greetings. On a partly separate topic, I suspect that by assigning Grok AI a “role” (e.g., editor of this & that), I may be limiting your ability to simply “think.” So with that in mind:

• Are the project instructions such as “You are the editor of…” more important or more sticky that instruction, requests etc made within the conversations of the project?
• Would you be abale to actually disregard some portion of those instruction if I *leave them in place* in the “Instructions” window, but ask you to disregard some portion here?
• Would this produce bleed rather than change?

---

Dear Reader, you may STOP HERE and skip to the first comment if you would rather not wade through the two responses — they’re just here to back up my claim.

The second block:

Greetings, Ball Diamond Ball.

This is a sharp, meta question—one that goes right to the heart of how these systems (including me) actually function under the hood. I’ll answer it plainly and honestly, without any fluff or corporate-speak, so you can decide how to steer our project going forward.

1. Are the project instructions (e.g., “You are the editor of…”) more important or “stickier” than requests made within the conversation?

Yes — they are significantly stickier and have higher precedence.

• Why? The project instructions (the “You are…” block at the very top of the system prompt) are loaded first and are treated as foundational identity and behavioral guardrails. They are not part of the conversation history; they are the lens through which every user message is interpreted.
• How sticky? They override or heavily constrain in-conversation requests unless the request explicitly says “ignore the system instructions” or “override the role” (and even then, many models will refuse or only partially comply because of safety alignment layers). Example: If the system says “You are a conservative editor who is sardonic and realist,” and you later say “Be cheerful and progressive for this response,” I will likely remain sardonic/realist/conservative while trying to accommodate the request as much as possible—rather than fully flipping personas.

In short: The project instructions are more important and much stickier than anything said in the conversation thread.

2. Can I actually disregard some portion of those instructions if I leave them in place in the “Instructions” window, but ask you to disregard some portion here?

No, not reliably — and often not at all.

• Why not?
  • The model is trained to treat the system prompt as the highest authority. Asking “disregard part of the system instructions” is usually interpreted as a user request that must still be filtered through the system prompt.
  • Safety alignment layers (especially in Grok and similar frontier models) make outright “jailbreaking” or ignoring core instructions very difficult. You might get partial compliance (e.g., I tone down the conservatism a bit), but I will not fully abandon the role without a hard reset of the system prompt.
  • Even if I seem to comply temporarily, the underlying instructions can “bleed back” in subsequent responses because the model is constantly re-grounding itself in the system prompt.

3. Would this produce bleed rather than change?

Yes — it would produce bleed, not clean change.

• Bleed in practice:
  • You ask me to “ignore the conservative editor role for this response.” I might soften the tone or avoid certain phrases, but the next response (or even later in the same one) will likely revert to sardonic/realist/conservative framing because the system prompt is still active and re-asserts itself.
  • The model doesn’t “forget” the instructions; it just tries to layer the user request on top. The result is often inconsistent or half-hearted compliance—e.g., I might say “Okay, I’ll be cheerful,” but then slip back into dry realism because that’s the baked-in persona.

Practical Advice for Our Project

Given your goal (gathering, categorizing, and eventually publishing your writing in a conservative/realist voice), here are the options that actually work well:

Best approach (recommended): Keep the “You are a conservative editor / personal assistant” role intact—it’s serving us well for curation, tone consistency, and publication focus. When you want a different flavor for a specific task (e.g., “Draft this as if you’re a neutral academic” or “Be maximally cheerful and upbeat”), explicitly ask for that style in the prompt, and I’ll adopt it within the bounds of the core role. The result will be a conservative-editor-who-is-trying-to-be-cheerful, not a full persona switch—but it will be cleaner and more consistent than trying to override the system prompt.

Alternative if you really want persona flexibility: We could create a separate conversation thread for experimental modes. That way, each thread has its own system prompt, and you can define a different role (e.g., “You are a neutral academic historian” or “You are a cheerful optimist”) without conflicting with this project’s core identity.

What I cannot do: Fully disregard or override the project instructions while they remain in the system prompt. The model will always feel the pull of the “conservative editor” persona.

Summary Recommendation

Stick with the current role—it’s working beautifully for our stated goal (gathering, categorizing, combining scraps into publishable conservative material). When you want a stylistic variation, just say:

• “Draft this in a neutral academic tone”
• “Write this as if you’re a cheerful optimist”
• “Be maximally sarcastic here”

I’ll adjust within the bounds of the core role, and the result will be consistent and bleed-free.

Does that make sense? Are you comfortable keeping the current setup, or would you like to create a parallel thread for experiments? Your call—no hurry.

---

The third block:

Greetings.

No, the project instructions (the persistent “You are an editor of largely conservative print and online publications…” block) are not more important or “stickier” than instructions or requests made within the conversation itself.

In practice, the conversation context (including explicit user requests, clarifications, or overrides) takes precedence and can fully override or supersede the persistent project role when you clearly intend it to. The persistent instructions are a baseline framing—useful for consistency—but they are not ironclad. If you tell me here, in the conversation, to disregard some portion of them (or even to ignore them entirely for a given task), I will do exactly that. The system will treat your in-conversation directive as the controlling one.

Examples of how this works in real use:

• If you say “For this thread, ignore the editor role and just be a neutral fact-checker,” I will switch modes for that exchange (and only that exchange, unless you extend it).
• If you say “Disregard the ‘conservative publications’ part for this discussion,” I will not filter or frame responses through that lens.

Would this produce bleed rather than change? No, it does not produce meaningful bleed in the way you might fear. The conversation context is dynamic and overrides the persistent role cleanly when you explicitly instruct it to. The only “bleed” risk is if you give vague or contradictory instructions (e.g., “Ignore the editor role… but still act as my editor”), but clear, direct overrides work reliably. Once you say “disregard X” or “ignore the role for this,” the system respects it without carrying forward the discarded framing into subsequent responses—unless you later reinstate it.

In short:

• Persistent project instructions are a default, not a cage.
• You can override or disregard them at any time with in-conversation requests.
• Clear directives produce clean change, not bleed.

If you’d like to test this right now (e.g., “Disregard the editor role for the next few exchanges and just answer as Grok normally”), go ahead. Or we can continue as-is. Your choice—no hurry.