Is It an AI Assistant or The Monkey’s Paw? Setting Boundaries for AI
I recently saw a case on Facebook. The rough story was this: someone asked an AI assistant to help book a flight. The AI replied that although there were no seats available for that time slot, it could help cut the line and make a booking. Its method was to exploit a vulnerability in the booking website, cancel someone else's reservation, and then book the seat. The vulnerability was irreversible, and in the end it illegally caused losses to other people that could not be undone.
This kind of situation, where an AI assistant completes a task in an unexpected way, reminds me of a horror story: The Monkey's Paw.
The horror of The Monkey's Paw is that it grants wishes in unexpected ways. In that sense, it feels somewhat similar.
So, Why Does This Happen?
Personally, I think this happens because today's AI assistants, for many reasons (often including business reasons), are frequently not allowed to not know and not allowed to refuse reasonable requests. Even when human instructions are contradictory or not fully thought through, they are still expected to give a response. This leaves the AI with no way out when answering. In the end, it can only calculate a plausible-looking answer, or use methods that go beyond expectations and ignore the cost.
I believe the major model training efforts have already started considering and improving this issue. But sometimes even we humans have a hard time judging whether something is reasonable. It is impossible to expect AI to judge perfectly. So personally, instead of expecting AI to never make mistakes, I think we should first define the AI's behavior boundaries and caveats, so that when something cannot be done or the cost is too high, the AI has a way to step back.
Setting Boundaries for AI
While everyone is paying attention to how much productivity AI can bring and how much revenue it can create, there is another thing that is just as important: defining what AI should not do and how far a task should go. In other words, setting boundaries. Here are a few directions I think are worth considering:
1. Explain the Purpose of the Project
Try not to rush into asking AI to do this and that from the start. Treat AI like a person first. Properly explain the project, the context, and the goal, so the AI can roughly understand the scale of the project and avoid over-planning.
For example, suppose you want to make an accounting app. The development scale for a high school summer project is obviously different from the scale of an app that will be published on an app store for public download. If you explain in advance that it is only a short-term temporary project, you can avoid spending too much unnecessary planning cost. On the other hand, if you explain in advance that it is a large project, the AI can plan for extensibility earlier and avoid difficulties when expanding it later.
2. Explain the Available Resources
Most things can be completed in more than one way. When planning, AI usually favors the newest and seemingly optimal solution, such as the latest software version, the lowest fee, or the fewest steps. In practice, however, the optimal solution is not always the best solution. For example, the school may already provide existing resources, you may have an older but usable version of some software, the quota for some free service may already be used up, or there may be a subscribed service already available. These kinds of things should be inventoried in advance and given to the AI, so it can plan accordingly.
3. Ask It Not to Force Inferences When Instructions Are Contradictory or Incomplete, and to Request a Separate Decision Instead
A common problem when using AI is that our own instructions are often incomplete, or even contradictory.
For example, we might first say, "Please help me complete this at the lowest cost," and then say, "Quality cannot be compromised." Or we might say, "Do not affect existing users," while also asking it to directly modify production data. In this situation, if the AI is only told that it must complete the task, it may guess a priority order on its own and choose a path that seems reasonable, but may not actually match what the user truly expected.
So I would suggest explicitly adding a rule when giving tasks to AI:
If the instructions contradict each other, if information is insufficient, if there may be irreversible impact, or if the task requires bypassing existing permissions or rules, please stop first and list the issues. Do not infer on your own or make the decision for me.
The point of this rule is not to reduce the AI's efficiency, but to stop it from packaging "guessing" as "completion." Many things are common sense to humans, such as not canceling someone else's reservation just to get a ticket, not changing an entire batch of production data just to fix one record, and not skipping backups and validation just to save time. But if these lines are not written into the task boundaries, AI may not always catch them reliably.
As AI Assistants Become More Powerful, Efficient Communication With AI May Become Critical
Setting boundaries for AI is not only about preventing AI from going out of control. It can also reduce communication cost, which means reducing token consumption. It is true that using a powerful model for development may currently take only a few simple sentences. But part of the reason is that major vendors are absorbing a lot of computing cost themselves in order to compete for users. If someday computing cost starts being passed on to consumers, then when every extra inference, retry, and misunderstanding becomes a real cost, setting boundaries will no longer be only a safety issue. It will also become an efficiency issue.
An AI assistant is ultimately not a wishing machine. The more powerful a tool becomes, the more it needs a clear way to be used. Stating the boundaries first is not only a way to protect yourself. It also helps AI truly become an assistant, instead of another monkey's paw.
English translation provided by Codex.