AI Prompting techniques to get the right responses: Insights from Wharton's Latest Study

Artificial Intelligence (AI) has become an integral part of our daily lives, from virtual assistants to advanced data analysis. However, prompting techniques for getting the right responses from AI can be challenging. A recent study by Wharton's Generative AI Labs, led by AI expert Ethan Mollick, sheds light on the complexities of AI interactions and offers valuable insights on how to improve AI performance.

Why is it so hard to get consistent AI responses?

One of the key findings from the Wharton study is that AI models often give different answers to the same question. This inconsistency can be problematic, especially for critical tasks where accuracy is paramount. The study tested different prompting approaches on AI models GPT-4o and GPT-4o mini, revealing some surprising results.

The Politeness Paradox

The study found that saying "please" or using polite language doesn't always help. In fact, politeness improved accuracy for some questions but worsened it for others. Similarly, commanding language had mixed effects. This indicates that the context and nature of the prompt play significant roles in determining the AI's performance.

The Importance of Formatting

Another crucial insight from the study is the importance of formatting. Clearly instructing the AI on how to structure its responses consistently improved performance. Precise and clear prompts lead to better results, highlighting the need for well-structured queries when interacting with AI.

The Quest for Perfection

The study also highlights that if an AI needs to be right 100% of the time, it might not be reliable. Both GPT-4o and GPT-4o mini performed only slightly better than random guessing on PhD-level questions when required to be perfect. This underscores the limitations of current AI models in achieving absolute accuracy.

Real-World Implications

These findings have significant implications for how we use and evaluate AI tools. Companies relying solely on benchmarks might miss the real-world inconsistencies of AI models. For mission-critical work, it might be beneficial to ask the same question multiple times and select the best answer.

How Sensisize Counters Inconsistencies

At Sensisize, we tackle these challenges head-on by implementing AI-based checks on model outputs. This approach decreases the probability of errors. Moreover we encourage a human-in-the-loop methodology to ensure accuracy and reliability. By having AI models check each other's work and involving human review before making decisions, we can achieve more reliable and accurate AI interactions.

AI prompting techniques must account for inconsistencies

Understanding the nuances of AI prompting and leveraging innovative solutions like those offered by Sensisize can transform how we interact with AI. By addressing the inconsistencies and tailoring our prompting strategies, we can unlock the true potential of AI and achieve more reliable outcomes.