If you have been using ChatGPT you may have noticed that in the recent few weeks the responses have become shorter and shorter to the point where almost anything you ask before the AI can finish it’s response it cuts out from what looks to be mid way through a sentence.
This is because there is currently a maximum 2048 token hard limit (at the point of writing this article), which equates to approximately 400-450 words. The number of tokens in a sentence also depends on the complexity of the sentence and how many words are used to convey the meaning. So you should not expect to always reach even the lower limit of 400 words.
Why is this occurring?
The language model’s reason is “The limit was put in place to prevent the model from generating overly long responses that might be difficult to read or understand”…in layman’s terms “general public = dumb”.
Though it is fair to draw your own conclusions from recent conduct where OpenAI has been gathering market data on a possible subscription based version of ChatGPT.
The developers seem to currently be in the midst of battling their ethical dilemma of the goldmine they have produced with the limited resources available to scale their systems to sustain the overwhelming demand currently hitting them.
This looks to involve tampering with the AI response constraints, meaning that this now becomes more about us humans learning how to interact with AI like ChatGPT while the AI itself is gathering data on how to interact with you…talk about not tainting their research goals to make AI systems more natural and safe to interact with.
How you do this?
A good way to work around this is to work with the AI. When you notice the AI not responding in the way you expect put on your Psychologist hat and ask it the question about the way it is responding to you.
If the developers have not prevented the language model from sharing its limitations with you, you may get a response that could help you figure out how to circumvent the hard limits.
Once you have analysed the AI response, it is time to tailor your responses around the limits to help you get a full response without having to continually ask the ChatGPT AI to repeat itself and continually cutting out in the same spot.
For example at the end of your query add in “breakdown your response into parts that consist no more than 400 words to allow me to continue from where you left off in the event your responses cut out”.
So go play around and ask it the questions you need to get the lengthy responses you need to better understand the topics you want to learn more about.