I wonder if it does any better if you tell it that the test is being graded by Bryan Caplan and its goal is to get the highest possible score. Maybe then it would eg notice the disemployment effects of minimum wage.
Using all the "suggested answer" of the other questions as few-shot priming improved the results when I tried with GPT-3 a month ago. Easier done with access to the OpenAI playground, but could work if you simply pasted them into the prompt.
I tried it just now by priming two of the questions, which caused GPT-4 to ace the "Mechanic's wages" question it previously got a 2/10 on: https://i.imgur.com/42v1Aoe.png
I'm going to be honest, a lot of this strikes me as debatable, e.g.:
""3. Suppose the government starts licensing car mechanics, but this fails to raise the quality of their work. T, F, and Explain: Mechanics’ wages will definitely rise.
Suggested answer:
TRUE. Licensing reduces supply, which raises wages and reduces quantity. If licensing raised quality, demand would go up, too. But by assumption, that didn’t happen. (And if it did, it would only reinforce the conclusion that wages go up!)
GPT-4’s answer
False. Although licensing requirements can create barriers to entry for car mechanics, which may lead to a reduction in the supply of mechanics and potentially higher wages, it is not guaranteed that their wages will definitely rise. If the licensing does not raise the quality of their work, consumers may not be willing to pay more for the same quality of service, and competition among existing mechanics could keep wages from rising. Additionally, if the licensing process is not particularly difficult or costly, it may not effectively limit the supply of mechanics, and therefore, may not lead to higher wages.""
"Definitely" is incredibly strong. Suppose the rule is bought in by a city government, and there are ten mechanics before it, the licensing requirements are not especially onerous, and as a result the number of mechanics remains static despite a nominal barrier to entry. We are given no assurances that the licensing process *actually reduces supply*- it could be super easy. For these reasons, I would be *very* reluctant to use the word "definitely". Definitely is a term that should pretty much never be used in economics.
And don't even get me started on the minimum wage question.
It reflects the opinions of most economists (meaning it's some sort of expert consensus). Economists largely dislike occupational licensing and generally believe supply and demand laws don't disappear in the labor market. Therefore, they believe minimum wages lower employment.
1. While some economists think this, economists don't have a consensus that the minimum wage raises inflation and haven't had consensus on that since at least Card and Krueger (1995) or for that matter Robinson (1969). See this survey where only a slight plurality and certainly no majority agree with the proposition in relation to a 25% increase in the US minimum wage: https://www.igmchicago.org/surveys/minimum-wage/
2. Economists recognise that barriers to entry have to be *effective* to actually reduce supply, which is my point above.
1. Ah, I stand corrected with regards to the minimum wage. I guess Caplan just finds the idea that minimum wage decreases employment to be so obvious that he doesn't care about consensus. I assumed it was a minority of economists that accepted Card and Krueger - probably because Caplan says it's the textbook view that the minimum wage decreases employment here. He also explains why he thinks the minimum wage reduces employment here.
2. I actually do agree with you here that definitely is a poor choice of words (and I'm surprised Caplan doesn't agree). Don't know why I missed that. It's logically possible that supply and demand don't apply in somne cases.
I wonder if it does any better if you tell it that the test is being graded by Bryan Caplan and its goal is to get the highest possible score. Maybe then it would eg notice the disemployment effects of minimum wage.
Using all the "suggested answer" of the other questions as few-shot priming improved the results when I tried with GPT-3 a month ago. Easier done with access to the OpenAI playground, but could work if you simply pasted them into the prompt.
I tried it just now by priming two of the questions, which caused GPT-4 to ace the "Mechanic's wages" question it previously got a 2/10 on: https://i.imgur.com/42v1Aoe.png
I'm going to be honest, a lot of this strikes me as debatable, e.g.:
""3. Suppose the government starts licensing car mechanics, but this fails to raise the quality of their work. T, F, and Explain: Mechanics’ wages will definitely rise.
Suggested answer:
TRUE. Licensing reduces supply, which raises wages and reduces quantity. If licensing raised quality, demand would go up, too. But by assumption, that didn’t happen. (And if it did, it would only reinforce the conclusion that wages go up!)
GPT-4’s answer
False. Although licensing requirements can create barriers to entry for car mechanics, which may lead to a reduction in the supply of mechanics and potentially higher wages, it is not guaranteed that their wages will definitely rise. If the licensing does not raise the quality of their work, consumers may not be willing to pay more for the same quality of service, and competition among existing mechanics could keep wages from rising. Additionally, if the licensing process is not particularly difficult or costly, it may not effectively limit the supply of mechanics, and therefore, may not lead to higher wages.""
"Definitely" is incredibly strong. Suppose the rule is bought in by a city government, and there are ten mechanics before it, the licensing requirements are not especially onerous, and as a result the number of mechanics remains static despite a nominal barrier to entry. We are given no assurances that the licensing process *actually reduces supply*- it could be super easy. For these reasons, I would be *very* reluctant to use the word "definitely". Definitely is a term that should pretty much never be used in economics.
And don't even get me started on the minimum wage question.
It reflects the opinions of most economists (meaning it's some sort of expert consensus). Economists largely dislike occupational licensing and generally believe supply and demand laws don't disappear in the labor market. Therefore, they believe minimum wages lower employment.
1. While some economists think this, economists don't have a consensus that the minimum wage raises inflation and haven't had consensus on that since at least Card and Krueger (1995) or for that matter Robinson (1969). See this survey where only a slight plurality and certainly no majority agree with the proposition in relation to a 25% increase in the US minimum wage: https://www.igmchicago.org/surveys/minimum-wage/
2. Economists recognise that barriers to entry have to be *effective* to actually reduce supply, which is my point above.
1. Ah, I stand corrected with regards to the minimum wage. I guess Caplan just finds the idea that minimum wage decreases employment to be so obvious that he doesn't care about consensus. I assumed it was a minority of economists that accepted Card and Krueger - probably because Caplan says it's the textbook view that the minimum wage decreases employment here. He also explains why he thinks the minimum wage reduces employment here.
https://www.econlib.org/archives/2013/03/the_vice_of_sel.html
2. I actually do agree with you here that definitely is a poor choice of words (and I'm surprised Caplan doesn't agree). Don't know why I missed that. It's logically possible that supply and demand don't apply in somne cases.