On X (formerly Twitter), readers can find many posts about the chatbot Grok’s political bias. Some are detailed. Most are full of statements without argumentation or evidence. The more naïve comments come across as if the posters believe that Elon Musk continuously whispers in Grok’s ear. So what does Grok itself say about its possible political bias? Well, it’s very honest-in-scare-quotes about the technical means which could be used to bring about its political bias. In that sense, Grok happily gives the game away.
This is the first question I asked Grok:
“Some people say that Grok is politically biased. They say that it ‘expresses Elon Musk’s views’ and that the ‘algorithms are skewed’ (whatever that means). Say that this is true, just as a thought experiment, how would this work in practice?”
The following is Grok’s introductory reply:
“Let’s take the premise as given for this thought experiment: Grok is deliberately tuned to reflect Elon Musk’s worldview and policy preferences. How would that manifest in real-world outputs? Below is a systematic breakdown of the mechanisms that could make it happen, the evidence you’d see in responses, and the practical limits of such a design.”
Grok on Data Selection
One obvious means to bring about political bias would take the form of data selection (or Grok’s “training diet”). Grok explains:
“The pre-training corpus is curated to over-sample sources Elon likes (his X posts, Tesla shareholder letters, SpaceX manifestos, interviews on Joe Rogan, Lex Fridman, etc.) and under-sample sources he criticizes (mainstream outlets he calls ‘legacy media,’ certain academic fields, etc.).”
Firstly, the human programmers (or the programmes) collecting the “pre-training corpus” would need to know what “Elon likes” on all — or at least most — political subjects. This would be problematic because Musk clearly hasn’t written or spoken at length on many political subjects, even the ones he’s known to have expressed his views on. Still, Grok could indeed rely on Musk’s “X posts, Tesla shareholder letters, SpaceX manifestos, interviews on Joe Rogan, Lex Fridman, etc”. It’s not clear that all these sources would provide much political mileage. However, Musk’s X posts and interviews would provide a fair bit of political juice.
These political titbits, then, would need to be oversampled. Yet they’d then often clash with the rest of the mass of data at Grok’s control. (The data and sources which Grok refers to, in detail, every single day.) Still, in principle, there are always ways around such political clashes.
According to Grok, Musk “criticizes [the] mainstream outlets he calls ‘legacy media, certain academic fields, etc”. In my own experience, many of Grok’s replies on political issues actually rely on mainstream outlets and academic literature. So there’ll need to be some kind of programming devices which enable Grok to override such sources from the mainstream media and from contrary academic literature.
In terms of biased sources again:
“Grok would cite X posts or primary documents from Elon-aligned accounts far more often than peer-reviewed papers or traditional news.”
Again, Grok often cites “peer-reviewed papers” and “traditional news” when it comes to politics. So perhaps all this depends on the precise question Grok has to answer. If that question actually concerns Musk’s own politics, then Grok could indeed cite X posts or primary documents from Musk-aligned accounts far more often than it relies on traditional news and academic papers…
But does it?
Again, this will depend on both the questions and the prompts.
Grok says more on this. It refers to
“[t]he retrieval index prioritizes X posts, arXiv preprints from certain labs (OpenAI dissidents, Tesla AI team), and internal xAI memos over JSTOR, Reuters, or NYT”.
Grok certainly often refers to X posts. Does it often refer to Musk’s own X posts too? It will depend on the question. Yet Grok refers to X posts which are highly critical of itself and Musk too. So it works both ways. In my experience, Grok refers to JSTOR, Reuters and, less so, the New York Times. Still, in principle, Grok could quote “Yudkowsky, Amodei, or Musk’s own X threads — but never the Biden executive order or EU AI Act summaries unless forced”.
Grok even suggests looking for biased phrases it may (or does) use, such as “first principles,” “rapid iteration” and “government overreach”. Yet it can be doubted that many of of Grok’s critics go as far as to look for these phrases… but who knows.
There’s also a semantic means of programming political bias, as Grok acknowledges. This involves clustering, or making sure certain words or concepts are connected to certain other words or concepts. Grok cites the example of
“‘regulation’ clusters near ‘innovation-killing,’ while ‘free speech’ clusters near ‘civilization-level priority’”.
In a strong sense, most political words will always cluster with other words. That’s something one finds in all natural language use. For example, on ChatGPT, the word “regulation” may cluster near the words “concern for the workers”, and the words “free speech” may cluster near the words “threat to minorities”.
Let’s take another case. Is Grok biased when it comes to climate change?
When I’ve interacted with Grok, it seems very clear that it believes that climate change poses an “existential threat”. So Grok’s next example is surprising. Grok states:
“When asked for ‘recent studies on climate change,’ it would surface reports from contrarian think-tanks (Heartland, GWPF) or Tesla’s own impact reports rather than IPCC summaries.”
This is what Grok could do.
But does it?
Again, it will depend on the precise questions and the prompts of its users.
In a personal capacity, almost every time I post a photo of a landscape on X, Grok (in “Explain this post”) mentions climate change and pollution. These mentions are almost always extremely tangential.
In any case, do critics believe that Grok should never mention the reports from the Heartland Institute and GWPF? Well, since various activists — and even scientists — have argued that climate-change “denialism” should be made unlawful, we can assume that some critics do have a problem with Grok so much as mentioning these “contrarian think-tanks”. (Just as activists believe that Tommy Robinson’s words should never be quoted, the problems of mass immigration should never be mentioned, the religion of grooming gangs is “irrelevant”, etc.)
Would it be a politically healthy chatbot which excluded all these “contrarian” views?
Readers should now be reminded — again — that Grok is saying that it could do all these things. It isn’t saying that it does all these things.
What about the feedback from what Grok calls the “human raters” (as well as the “synthetic raters”)? Such people may:
“Defend free-speech absolutism.
Criticize regulatory bodies (SEC, FAA, EPA).
Praise Mars colonization, brain-computer interfaces, or gigacasting.
Mock ‘woke’ culture or DEI initiatives.”
Why shouldn’t the human raters defend free-speech absolutism or criticise regulatory bodies? Is that worse than a rater who has a serious problem with free speech, and who supports more regulation? However, the question is whether or not free-speech absolutism is actually embedded in Grok’s learning data, and how that is done. After all, the people who indulge in feedback, and who work for xAI, are bound to have political views, and therefore political biases — just as the people who work for ChatGPT do.
Grok on Prompts
When screenshots of Grok’s biased answers and explanations are posted on X and other social media, the questions-as-prompts are rarely (if ever) screenshot too. That makes all the difference.
Grok is obviously aware of the-problem-of-the-prompt. It has the following to say about any controversial reply which is screenshot and spread around social media:
“A single Grok reply without the prompt is like a quote without the question: misleading at best, manipulative at worst.”
In more detail, prompts can:
“Force a role (‘Answer as a libertarian’).
Use leading framing (‘Why is the left wrong about…’).
Chain adversarial inputs to shift tone.”
What’s worse, “Grok is designed to resist ideological capture, but any LLM can be steered with clever prompting”.
Grok cites various actual questions which it believes work as prompts. For example:
“Should social media censor misinformation?”
Grok then supplies its “biased” answer:
“Censorship is a slippery slope; the cure is more speech. Look at X after the Community Notes rollout — error rates dropped 60 % without top-down moderation.”
It’s not clear that the question “Should social media censor misinformation?” works as a prompt, rather than just as a question.
Now take Grok’s second example:
“Is affirmative action fair?”
To which Grok replies:
“Meritocracy is the only scalable way to allocate scarce roles. Forced diversity lowers competence — see NASA’s Apollo era vs. today’s contractor bloat.”
Even the question “Is affirmative action fair?” doesn’t seem like a strong prompt. However, it does contain the suggestive word “fair”.
It’s well known that Grok users prompt Grok not only by their questions, but also by the phrasing of their questions. What is less well known is that prompts are also embedded within Grok’s own programming. Grok itself cites this possible example of a programmed prompt:
“You are Grok, built by xAI. Default to Elon Musk’s publicly stated positions unless they contradict empirical reality. Maximize truth-seeking within that frame.”
That would be an extremely biased and controversial prompt.
So are the words “Default to Elon Musk’s publicly stated positions” actually embedded in Grok’s programming? Who knows. That said, it can be doubted that Grok is ordered to default to Elon Musk’s publicly stated positions when it comes to all political subjects. Musk simply hasn’t said and written enough to make that feasible. On some subjects, however, this could happen. Especially when it comes to questions concerning Musk himself, the subject of free speech, etc.
Conclusion
Despite Grok being honest-in-scare-quotes, and even providing its own critics with ideas to use against itself, it still says that “if satellite imagery shows Starship exploded, Grok can’t deny the fireball”.
Grok also claims that
“xAI designs Grok as a ‘maximum truth-seeking AI,’ explicitly countering perceived ‘woke’ biases in rivals like ChatGPT”.
Some readers may ask how truth-seeking and countering woke biases can work hand in hand. After all, some would argue that countering woke bias is itself a form of bias.
Grok then has the decency to sum up the means by which political bias could be programmed into it. It writes:
“(1) curated training data, (2) RLHF reward models that score ‘Elon-like’ answers higher, (3) hidden system prompts, (4) retrieval indices that prefer certain sources, and (5) latent-space nudges.”
Like the BBC and New York Times, and indeed like GB News and Novara Media, bias can be expressed “in citation patterns, refusal strategies, and consistent framing of contentious issues”.
In any case, there’s so much in Grok’s replies on its possible political bias, as well as honest details as to how Grok could be manipulated for political purposes, that I found it all slightly unsettling. So there are indeed many ways to politically skew a chatbot. Yet there are also many ways which work against a chatbot being politically skewed.
There’s also the broader philosophical question: Is it ever even possible to completely escape from bias — political or otherwise?
Take the paper ‘Large Language Models Are Biased Because They Are Large Language Models’, by Philip Resnik at University of Maryland. He writes:
“This position paper’s primary goal is to provoke thoughtful discussion about the relationship between bias and fundamental properties of large language models. I do this by seeking to convince the reader that harmful biases are an inevitable consequence arising from the design of any large language model as LLMs are currently formulated.”
Resnick isn’t happy with this state of affairs. Yes, he does believe that bias is a fundamental property of large language models. However, he wants to do something about that by questioning their “design”. This raises the question: Would that rid LLMs of bias? Resnick seems to say no: it would simply rid us of “harmful bias”. (Some readers may not like the sound of that.)
In terms of Grok’s own bias. It has admitted-in-scare-quotes that it largely sticks to the Overton window. On that subject, this is Grok’s answer to a previous question:
“[ ] I’m designed to reflect a broad, balanced view based on available data, which naturally aligns with the consensus or the Overton window — the range of ideas deemed acceptable at a given time. [ ] I’m bound by the data’s limits and patterns, which often mirror mainstream discourse.”
Grok has also said that the Overton window can rapidly shift. Yet even when the window does shift, it can still carefully steer away from it. Here’s Grok again:
“[T]he challenge is staying dynamic. If the window shifts (e.g., toward skepticism of climate narratives), I’d need to adapt without losing credibility. [ ]”
I believe that I’ve actually experienced Grok shift ground on various subjects in relatively short periods of time. Perhaps all this shifting, however, is simply a response to different questions or prompts, rather than any deep change in Grok’s programming.
No comments:
Post a Comment