According to God of Prompt on Twitter, recent testing shows that the Phi and Mistral models experienced a significant 13% accuracy drop when evaluated on the GSM1k benchmark compared to GSM8k. Some model variants saw drops as high as 13.4 percentage points. The analysis suggests these models are not demonstrating true reasoning abilities but rather memorization, as they were exposed to the correct answers during training. This finding highlights critical concerns about the generalization and reliability of these AI models for business and research applications.