Welcome to the AI Ready Blog, where we explore the evolving world of Artificial Intelligence and Generative AI in education, fostering dialogue, experimentation, and research to enhance teaching, learning, and collaboration across disciplines.
Note: Features may change with future updates.
Chat GPT? Gemini? Co-Pilot? Claude? Which is the best generative AI tool?
Well, these models are always being refined by their respective parent companies, so what’s “best” is constantly shifting. Additionally, while one tool might be good at something, another might be better at something else… and then that might shift after a particular update!
That being said, there are plenty of people out there tracking the developments in generative AI. AI leaderboards – scoreboards that help track and compare AI systems – have become the main way to track them, providing a standardized way to benchmark the performance of different AI models on specific tasks, fostering innovation and accelerating scientific discovery.
How AI Leaderboards Work
AI leaderboards typically function by establishing a standardized benchmark dataset and evaluation metric. Researchers submit their AI models to the leaderboard, and their performance is measured against the metric. The models are then ranked based on their scores, allowing for easy comparison and identification of top-performing approaches.
Popular Ways to Compare AI Systems
Different platforms take various approaches to evaluating AI:
- Traditional Benchmarks: Some leaderboards test AI systems on specific tasks, like reading comprehension or mathematical problem-solving.
- User Experience Testing: Platforms like Chatbot Arena take a more practical approach – they show users responses from different AI systems and let them vote on which is more helpful, similar to a blind taste test.
Top AI Leaderboards for Higher Ed
Several prominent AI leaderboards are relevant to higher education research:
- Livebench: A relatively new and innovative platform that offers real-time benchmarking of AI models. Unlike traditional leaderboards, LiveBench provides a dynamic environment where researchers can continuously evaluate and improve their models.
- MixEval: Another new platform that combines real-world user queries and ground-truth benchmarks to provide a more comprehensive and accurate evaluation of large language models.
- ChatbotArena: LMSYS ORG’s platform that lets users compare AI chatbots head-to-head by having two models respond to the same prompt and voting on which response is better, with the results aggregated into public rankings using an Elo rating system.
- ZebraLogicBench: Benchmarks the logical reasoning ability of language models
There are many others out there and the field is constantly evolving. That said, it is interesting to see what people see as important.
Related links
- Infographic: The Most Popular Generative AI Tools, see ChatGPT Dominates Web Traffic Among GenAI Tools for more information
- Chart: AI Chatbots and Playgrounds by AI Model
- Stanford’s 2024 AI Index Tracks Generative AI and More
featured image
Sponsored by the Fordham Faculty AI Interest Group | facultyai@fordham.edu | www.fordham.edu/AI
AI Events sponsored by the Fordham Faculty AI Interest Group