学进去

Bryan Caplan, an economics professor at George Mason University (GMU), said the progress ChatGPT made — it improved is score from a D to an A on his economics test in just three months — has shocked him.

Writing in a blog post, Caplan said he gave ChatGPT questions which test students’ understanding of economics rather than complete what are essentially memory exercises. It’s here where the old version of ChatGPT tipped up. The bot scored 31 out of a possible 100 on his test, similar to a D and well below his 50% median (中位数).

Caplan told Insider that the bot failed to understand basic concepts. “ChatGPT does a fine job of imitating a very weak GMU econ student,” Caplan wrote in his January blog post.

He isn’t the only academic that ChatGPT has disappointed. While it passed a Wharton Business School exam in January, its professor said it made “surprising mistakes” on simple calculations (计算).

But when ChatGPT-4 was released, its progress shocked Caplan. It scored 73% on the same midterm test, similar to an A and among the best scores in his class.

For Caplan, the improvements were obvious. The bot gave clear answers to his questions, understanding and explaining principles it previously struggled with. Caplan thought ChatGPT’s training data might have picked up his previous blog post where he explained his answers, but colleagues told him this was highly unlikely. He added that he’s already fed the bot new tests it hadn’t seen before, where it did even better than its previous 73% grade.

AI bots have caused headaches for examiners. Professors told Insider that plagiarism (抄袭) can be hard to prove with material from ChatGPT because there is no material evidence of wrongdoing.

Caplan said he’s thinking of doing away with graded homework with the rise of ChatGPT. He hopes his habit of regularly changing questions will be enough to stop students from learning and copying ChatGPT’s responses in exam settings.