Yes. LLMs can create convincingly human output.

AI 10个月前 admin
162 0 0

Yes. LLMs can create convincingly human output.Why LLMs don’t sound human, strategies to fix it, and real examples.

I’ve talked to a lot of people that think it’s obvious when text has been written by LLMs. That’s true for most generated text. However, this can lead to overconfidence in determining if something has been written by AI.

It creates a bias towards trusting content that “seems legit.” It may also lead teachers and professors to believe their intuition (or even programs like TurnItIn) will be able to detect LLM-generated content.

This is a misconception. The reality is that this is a limitation of the prompt. Prompts which generate human-sounding text are feasible to write. I offer examples below, but first let me explain why I believe LLMs have a distinct default tone.

Why So Serious 为什么这么严重

LLMs use RLHF (Reinforcement Learning from Human Feedback) to improve their responses. They are trained on a diverse range of internet text. They predict what text should come next based on patterns and feedback. Humans are impressed by good vocabulary and academic-sounding output. So it’s natural that the default style of LLMs has migrated towards that.
LLM使用RLHF(来自人类反馈的强化学习)来改善他们的反应。他们接受过各种互联网文本的培训。他们根据模式和反馈预测接下来应该出现什么文本。良好的词汇量和听起来像学术的输出给人类留下了深刻的印象。因此,LLM 的默认样式已向此迁移是很自然的。

Also, academic content tends to be dense with information. When we are testing or chatting with LLMs, we usually want information. This also likely leads to them sounding more academic.

Another sign of good writing is not using the same words over and over again. LLMs literally have features which decrease the likelihood of repeating tokens. Humans, on the other hand, tend to use the same words over and over, especially in informal writing.

This is why LLMs often sound formal or academic. The training data is skewed towards more formal, information-dense text. This doesn’t mean they can’t generate casual or creative text, it just means they’re less likely to do so unless specifically prompted.

Strategies For Generating Human-like Text

There are a couple ways to get more authentic sounding output.

Ask for less formal text directly. Here’s some examples:

  1. “Write a text message to a friend about (topic)”
  2. “Write a junior high level paper about (topic). It shouldn’t be well written.”
  3. “Informally, as if writing on IRC or Reddit, tell me about (topic) but still use good grammar and puctuation.”

Limit its vocabulary by adding something like this to your prompt:

<prompt>. It should use middle school level vocabulary.

Give it the desired style as context by including a few thousand tokens of the writing style and tone you’d like it to emulate. Example:

Write about <prompt>. It should use my tone and voice.
Here is a sample of my writing so that you can emulate it:
<snippet of your writing>

Utilize presence_penalty or frequency_penalty to increase the chance of repeating words. Let’s assume you’re using OpenAI’s models. If you’re using Simon’s command line tool llm, you would use -o presence_penalty -.8 (or any value between -2 and 0) but you can also pass presence_penalty and frequency_penalty in via the API. Here’s an example using llm command line tool.
利用presence_penalty或frequency_penalty来增加重复单词的机会。假设你使用的是OpenAI的模型。如果您使用的是 Simon 的命令行工具 llm,您将使用 -o presence_penalty -.8 (或 -2 到 0 之间的任何值),但您也可以通过 API 传入 presence_penalty frequency_penalty 。下面是使用命令行工具的示例 llm 。

echo '<prompt>' | llm -o presence_penalty -.8

Put the desired format in the prompt because a middle school essay, for example, will have a rigid Introduction-Arguments-Conclusion structure. By limiting its creativity to the skill level of the desired output, you can force it to seem more like the intended writer. This can be extrapolated to other structures. Example:

<prompt> it should be 5 paragraphs. 1 intro where it introduces 
the 3 ideas of the middle 3 paragraphs. and then 1 conclusion 
paragraph that also mentions the 3 main ideas.

Full Examples With Real Output

Note: If you can’t read theses, right click on the image and choose “Open in New Tab” to view them easier.

Harry Potter Paper Example

Yes. LLMs can create convincingly human output.

Cybersecurity Career Essay Example

Yes. LLMs can create convincingly human output.

New Blog Post Introduction Example

Yes. LLMs can create convincingly human output.

Conclusion 结论

While it’s true that LLM-generated text often has a distinctive style, it’s not because LLMs are inherently formal or uncreative. It’s a reflection of the data they were trained on and the way they were prompted. With the right prompt, I believe LLMs can generate text that is indistinguishable from human-written text.

原文始发于rez0:Yes. LLMs can create convincingly human output.

版权声明:admin 发表于 2023年8月31日 上午9:52。
转载请注明:Yes. LLMs can create convincingly human output. | CTF导航