Is it true that "AI doesn't hallucinate"? - The Concept of "Not Letting ChatGPT Speak": The True Nature of the Ultra-Fast AI "Jev" That Enthralls Developers

Is it true that "AI doesn't hallucinate"? - The Concept of "Not Letting ChatGPT Speak": The True Nature of the Ultra-Fast AI "Jev" That Enthralls Developers

Is the True Value of AI Really in "Speaking Well"?

The progress of generative AI has long been discussed in terms of "how naturally it can write" and "how well it can answer difficult questions." However, when companies integrate AI deep into software to automate processes, fluency is not necessarily the greatest value.

What is needed is the ability to distinguish whether an inquiry is a cancellation consultation or a refund request, to determine whether a command being executed is dangerous, to select the appropriate AI model from multiple options for the task, and to return the level of confidence in a form that the program can handle.

Gaining attention in this context is "Jev," released for early access by the U.S. company TypeSafe AI. Led by Diogo Almeida, a former OpenAI researcher involved in RLHF research using human feedback and the development of technologies leading to ChatGPT, Jev is a notable development.

The most distinctive feature of Jev is that it does not engage in chat. It does not generate stories or explanations but instead returns structured values and probabilities for predefined options set by developers. TypeSafe calls this the "System One Model," likening it to "System 1," which represents fast, intuitive human thinking, and optimizing the AI towards processing a large number of instantaneous judgments rather than deep, lengthy reasoning.


Speed Achieved by "Not Creating Sentences"

Typical large language models generate the next token one by one based on the previous words or tokens. While they are highly versatile and can be used for conversation, summarization, and program creation, longer outputs increase time and cost. Moreover, when used from a program, the response must be parsed and checked to see if it is in the expected format.

Jev intentionally discards this degree of freedom. Even if the input is unstructured data like natural language, the output candidates and data types are predetermined. For example, customer emails can be classified as "billing," "cancellation," "product defect," or "other," with probabilities returned for each. By not writing sentences word by word and outputting multiple judgments in parallel, it is easier to accelerate the process.

According to TypeSafe's published figures, response times range from 70 to 500 milliseconds. Input costs are $0.042 per million tokens, and outputs are treated as free, as they are "not costly enough to measure." The company explains that in the targeted System One-type tasks, it can be tens to up to 200 times faster while maintaining intelligence close to existing LLMs. Their proprietary workflow evaluation shows it to be 193.6 times faster and 444.6 times cheaper.

However, these evaluations are primarily designed and published by the developers. TypeSafe itself notes the potential for bias by workflow creators, cases where comparison conditions favor Jev, and that the long-term sustainability of prices cannot be proven. The flashy multipliers should not be taken as general performance reproducible for all purposes.


Developers Reacted More to Ease of Integration Than "Smartness"

Immediately after its release, interest was so high that the ability to provide the API temporarily could not keep up. On social media, the focus was not on enjoying casual conversation with the model, but on posts by developers measuring its replacement in actual business workflows.

 

Pranit Sharma, a software engineer at Vercel, reported that replacing an existing OpenAI model with Jev for command safety classification resulted in increased accuracy and a speed increase of 5 to 18 times.

Nikhil Mudholkar, CTO of Bryo AI, compared it with Gemini for business email classification. In his tests, Gemini had slightly better accuracy, but Jev's cost was about 1/10 to 1/20, and he particularly valued the practical probabilities attached to each decision, making it easier to create a system where only cases with low confidence are referred to humans.

On the other hand, Armin Ronacher, involved in open-source model execution platforms, pointed out that how probabilities are handled is the responsibility of the user. Should processing stop at 50%, or is it okay to automate at 95%? The boundary is not determined by the model. Jev shows uncertainty, but how to convert that uncertainty into business rules must be designed by the implementing company.

These reports on social media are interesting as initial examples showing Jev's potential. However, they are not large-scale independent benchmarks conducted under uniform conditions, but results from specific uses tested by individual developers. Favorable reactions and general performance proof need to be distinguished.


What Does "Not Hallucinating" Mean?

The expression "not hallucinating" regarding Jev requires careful attention.

If output candidates and types are fixed from the start, it is easier to fundamentally prevent problems such as creating non-existent items or breaking format in the middle of an explanation. Since what should be returned is limited to true/false values, classification labels, numbers, probabilities, etc., there is little room to generate unreadable sentences or fictitious function calls.

However, correct format does not guarantee correct judgment content. Misclassifying a "refund request" as a "general inquiry" still results in a perfectly correct output format. What TypeSafe guarantees is mainly staying within schemas and types, not always being correct about reality.

Probability calibration is also important. When a model answers "90%" and a large number of such cases are collected, if about 90% are indeed correct, the probability is well-calibrated. Conversely, if it shows 90% but the accuracy rate is 60%, it cannot be used as a safety valve for automation. It is necessary to continuously verify for each industry, language, input length, and data changes over time.

When used by Japanese companies, it is also necessary to check the handling of Japanese honorifics, abbreviated expressions, company-specific terms, and ambiguous requests. Even if it performs well with English emails, it may not achieve the same accuracy with Japanese call center records or approval documents.


Not Replacing Generative AI, But Monitoring Alongside

A promising use of Jev is not only to replace LLMs themselves but also to act as a "monitor" placed before and after generative AI.

For example, it can determine whether a user's request contains commands aimed at obtaining confidential information, check whether operations an AI agent is attempting to execute are dangerous, and evaluate whether generated responses contradict the conversation content or violate prohibitions. Calling a high-performance LLM as a monitor each time increases costs and wait times, but a low-cost, fast decision model makes it easier to insert multiple stages of checks.

Another use is "model routing." Simple requests are directed to small, inexpensive models, difficult questions to high-performance models, and requests involving images to multimodal models. Using an expensive LLM for routing itself diminishes the savings effect, but a model like Jev may allow for detailed selection at the entry point.

In corporate systems, it is often more realistic in terms of cost, speed, and auditing to combine small components with different specialties rather than entrusting all processes to a single all-purpose AI. Jev aims to be an "invisible common component of judgment" called upon within countless applications.


The "Jevons Paradox" Behind the Name

The name Jev is derived from the 19th-century economist William Stanley Jevons. Jevons Paradox is the idea that as resource utilization efficiency increases, instead of saving, the scope of use expands, and total consumption may increase.

If the cost of AI judgment becomes 1/100, it does not necessarily mean that companies will reduce AI spending to 1/100. There is a possibility of adding hundreds or thousands of judgment processes to scenarios that were previously too expensive to implement. It allows for inserting "slightly smarter judgments" into every inquiry, transaction, click, and agent operation.

While this enhances convenience, it also means that monitoring and automatic selection spread to unseen areas. If used for classifying job candidates, prioritizing customers, or determining fraudulent use, the impact of errors and biases is significant. The advantage of being inexpensive and thus usable in large quantities is closely linked to the risk of amplifying the same misjudgments on a large scale.


Practical Implementation Steps for Japanese Companies

In Japan, the introduction of generative AI often starts with internal chat and document creation support. Models like Jev are suitable for the next stage. They can be used for inquiry routing, prioritizing monitoring alerts, classifying sales cases, content review, and AI agent operation approval—tasks currently done by humans looking at screens and branching.

During implementation, it is safer not to automate immediately but to first run alongside human judgment. Measure accuracy and probability calibration with business data, and set higher automation thresholds for items where errors would result in significant losses. Return cases with low confidence to humans and continuously monitor changes in models and input data. Additionally, it is necessary to record who defined the output candidates and who approved the thresholds.

It is important not to obscure responsibility with "because AI judged," but to be able to explain the rules that converted AI probabilities into corporate decision-making. Jev's strength lies in returning structured values rather than free-form sentences. Whether that strength can be linked to auditability makes the difference between a mere fast classifier and a reliable business foundation.


The Main Battlefield for AI Moves Outside the Screen

Jev is not an entity that will end conversational AI like ChatGPT. For text creation, dialogue, complex reasoning, and creative tasks, the freedom of LLMs is indispensable. On the other hand, for small judgments repeated millions of times within software, freedom can be a hindrance.

The essential question posed by Jev is not "which is the smartest AI," but "is text generation really necessary for that task?" If the necessary answer is a few options and probabilities, there is little reason to call a massive conversational model each time.

In the future, similar models and competing products will increase, and independent third-party evaluations will progress. The secretive architecture of Jev, claims of training with synthetic data alone, and the sustainability of pricing also need verification. Nonetheless, the view that the future of AI is not only in flashy chat screens but also in mundane processes like conditional branching, monitoring, classification, and distribution is compelling.

The next AI revolution may not necessarily speak to humans. It will decide "what to do next" so quickly that it goes unnoticed, deep within the software. Jev aims to be at the forefront of that quiet change.


Regarding Reactions on Social Media

The social media reactions introduced in the text were organized based on individual tests posted by developers immediately after the release and interviews and quotes by TechCrunch. While there is strong evaluation of speed, cost, and probability output, it is not a large-scale independent verification under the same conditions, so it needs to be re-evaluated with each company's production data.


Source URL

Related Articles