Designing An Economics Essay Writing and Marking Software

The Grass is Not Always Greener on the Other Side

As an A Level Economics tuition practitioner, it is not unusual for us to encounter groans and grouses whenever we set writing homework for students. Indeed, students often lament about how tedious and time-consuming it is for them to produce a full-length Economics essay.

If writing an Economics essay is tough, it would be injudicious to think that marking the same essay (or for that matter, dozens of such essays) would be a breeze. I once went through 120 essay scripts in 2 days and promptly went into a state of stupor after that. With that experience behind me, I can only marvel at the steely resolve that school teachers have to carry them through the various challenging tasks at school and after school hours.

To understand why the task of writing and marking is so challenging on both ends, we need to consider what makes up a good Economics essay.

Components of a Good ‘A’ Level Economics Essay

Economics, being a social science, differ from the physical sciences such as Physics and Chemistry in that there is no single law to ensure that a certain outcome will occur. Without belittling the academic rigour associated with the sciences, what this means is that there is no single “correct” answer to address an issue in Economics. The same extends to writing an Economics essay. A balance of different views needs to be presented, followed by a well-thought synthesis to conclude the essay’s overall position. This narrative also needs to be backed up with good content rigour.

In simpler terms, we can distill and project the key components of a good A Level Economics essay as follows: –

Balance: Students are expected to think critically and juggle various theses and anti-theses surrounding an Economics issue. For example, when assessing the impact of slower growth in China on the Singapore economy, one would usually put forward the view that being externally-driven, the Singapore economy would suffer from weaker export sales to the Chinese market. This can be balanced off against the opinion that weaker Chinese growth means lower demand for global commodities and hence lower global prices. This could benefit Singapore through lower import-push inflation.

• Implications for examiners: This means that instead of a clear-cut binary answer, markers need to look through different writing styles (not to mention handwriting) to decipher the underlying arguments being made and then assess whether a balanced position has been provided. For the readers out there who are doing A Level Economics, I have sagely advice – do your teachers a favour and cut to the chase when you write. An essay with clear theses and anti-theses is always the easiest to mark.

Content: Content rigour in Economics is usually demonstrated through the use of economic analysis. This entails the use of well-explained diagrams or detailed interpretation of economic theories such as the Theory of Comparative Advantage (used to explain the basis for international trade). In addition, definitions for important concepts should also be provided.

• Implications for examiners: In short, students need to allocate cognitive bandwidth to memorising key content, while markers need to be eagle-eyed in marking scripts. Never mind if it’s already past midnight. Never mind if this is already the 35^th script being marked. Content errors usually suggest conceptual mistakes and should not be overlooked in marking an essay.

Context: Capturing the right context is perhaps one of the most important aspects of scoring in an Economics essay.  Essential nuances are often embedded in a question and students who replicate model essays would often find themselves performing poorly. The context varies depending on the topic being assessed. For example, under macroeconomics, students need to identify the specific nature and state of an economy; while in microeconomics, specific market characteristics such as the relevant elasticity values or the various sources of market failure need to be identified.

• Implications for examiners: Good writing captures the context and brings it to the attention of the marker. In many students’ essays, the context is hard to find, like the TV remote in my living room. Tender-hearted examiners would nonetheless trawl through a student’s essay and sift out contextual points which they can award marks for. This takes time, patience and effort. When either component runs scarce, so do the marks awarded.

Technology as an Enabler

I watched a YouTube video recently which showcased the revolutionary Panasonic Assist Robot. In the video, factory workers wearing huge robot exoskeleton suits were shown lifting bulky objects without breaking into a sweat. These suits work by embedding engines in the suit that automatically coordinate and amplify the effects of human movement. The end result is that they reduce the energy which a person needs to use to perform an action, such as lifting a bulky object.

It is tempting to imagine the amount of work effort saved if technology can be deployed to do the heavy lifting for Economics teachers. This can take the form of a software to help with the marking of Economics essays. Such a development would free up time for the teachers to prepare more highly customised lesson plans to address student weaknesses identified after each round of assessment.

However, in designing an Economics essay marking software, care must be taken not to give too much autonomy to the machines. For example, in 2012, Les Perelman from the Massachusetts Institute of Technology demonstrated that long and incoherent essays could achieve higher scores at the Scholastic Assessment Test’s (better known as the SAT) writing test compared to well written essays. This is because SAT essays were marked by an automated scoring machine (e-Rater), and Perelman proved that computer systems like these could be “gamed” because they often focused on factors associated with good essays, such as strong vocabulary and good grammar. Hence, as long as an essay was peppered generously with such factors, the writing did not necessarily have to make sense in order to attain a good score.

Other than the above challenge, the task in designing an Economics essay marking software is made more difficult since a good essay entails more than just avoiding the errors of grammar and style. By this, I’m referring to the key elements of BCC (Balance, Content and Context) as discussed earlier. Hence, a selective approach needs to be undertaken. Specifically, this means ditching the elements that can be manipulated with “machine-tricking strategies” and focusing on the key aspects of a good Economics essay.

Hence, instead of seeking to develop an autonomous artificial intelligence (AI) that can do all of a teacher’s work (i.e. a robot marker like e-Rater), a robot Sherpa should be the end goal.

An Economics Sherpa

Without the help of the Sherpa people, many people would not have made it to the top of Everest. The Sherpa population, known for their skills in mountaineering, have two important roles. First, because of their innate ability to adapt to harsh climates, many climbers use Sherpas to help carry their logistical load such as ropes and tents up the mountain. Second, being indigenous people, Sherpas know the landscape like the back of their hands. Hence, they serve as valuable location guides to foreign climbers, especially at extreme terrains.

Similar to the role of a Sherpa, an automated marking system can serve as a dual-purpose intermediary between students and examiners. First, the software can guide students on the proper steps to writing a balanced, content-rich and contextual essay. Next, having a systemised essay writing software also means that a marking module can be designed to bear some of the teacher’s workload.

To design such a software, a few key steps grouped according to the 3 “Ds” are needed:

1. Distillation: Every Economics topic has a number of key concepts that are frequently assessed in the examinations. These are usually concepts that cover a significant scope in the topic, which makes them ideal candidates to assess a student’s breadth and depth of knowledge. They can be thought of as parachute concepts since once opened up, they cover a significant scope in the syllabus. These parachute concepts can be screened out through a trend analysis i.e. through reviewing past year Prelim and A Level questions. Minimally, they should be assessed at least 3 times during a 10 year period.
2. Decomposition: After these parachute concepts are distilled out, methodologies should be designed to help students answer essay questions set based on these concepts. By methodologies, I am referring to the breakdown of the answer structure into a series of steps (akin to an answer structure). This is known as a decomposition exercise. Where certain steps consist of fixed outcomes, multiple choice answers should be provided. This positions the software for the inclusion of a marking tool which screens the submitted essay plan based on pre-determined parameters (i.e. “correct” answers).

This is a tricky step because an unintended consequence would be to encourage rote learning or the routine memorisation of essay plans. To avoid this pitfall, the structures should guide students on the steps involved to infuse Balance and Content rigour into their essays. However, they should not be prescriptive to the extent that they dilute the Context in an essay question. Hence, while these structures should steer students into recognising that a balanced and content-rich essay should consist of certain key steps, they should be sufficiently broad to accommodate different contexts embedded in essay questions.

3. Details: Once the essay structures have been decomposed into a series of steps, students can take guidance from the software to draw up their own essay plans. The interface involves students (i) selecting the relevant parachute concept assessed in the question; (ii) the relevant essay structure; and (iii) following the designated steps in the selected essay structure. Other than clicking the correct options to bring them to the next step, students are allowed to add details at each step to infuse contextual elaboration into their answers.

In essence, this means that with the software, students do not need to turn in a full essay immediately. A well drafted essay plan can already demonstrate to examiners the strengths of a student’s content knowledge, and the student’s ability to churn out arguments in a lucid and balanced manner. Upon approval of the essay plan, students can then proceed to write out a full essay with the assurance that they are heading in the right direction.

A Not-so-FEEBLE Example

The above idea can be demonstrated using the topic on Resource Allocation in Competitive Markets (more commonly referred to as “Demand and Supply”). In the past 10 years, questions pertaining to the “impact of certain events on a market” were assessed 8 times in the A Levels. For example, in 2012, students were asked to examine the effects of a rise in income and higher cost of car manufacturing on the sales revenue of car manufacturers. In 2011, students had to assess the impacts of an ageing population on related product markets in Singapore. The recurrence rate of 80% makes this a definite parachute concept.

To assess how certain events affect price, output or revenue in a market, we need to consider the various demand and supply factors and the relevant elasticity concepts. I have developed a FEEBLE structure to help students tackle such questions. The essay structure can be decomposed into 6 steps as follows:

1. Identify Factors affecting demand and supply based on the preamble in the question.
2. Invoke price Elasticity of demand to explain the impact of a shift in supply on the market.

iii.    Invoke price Elasticity of supply to explain the impact of a shift in demand on the market.

1. Explain whether demand or supply curve exhibits a Bigger shift.
2. Link the adjustment to equilibrium process to the diagram.
3. Evaluation e.g. consider alternative elasticity values/whether ceteris paribus condition holds.

A walkthrough of the essay writing guide is presented below.

Step 1: Students are asked to pick whether to assess the impact on price, quantity or revenue based on the requirements of the question. Students who made a wrong choice here will be screened out by the marking module. These students have clearly not read the question properly.

Step 2: Students are asked to pick the correct demand factor(s) based on the preamble. A wrong choice as screened out by the marking module indicates content weakness.

Step 3: Students are asked to pick the correct supply factor(s) based on the preamble. A wrong choice as screened out by the marking module indicates content weakness.

Step 4: Students are asked to pick the correct price elasticity of supply factor(s) and value based on the good in question. Both content and contextual understanding are assessed here.

 

Step 5: Students are asked to pick the correct price elasticity of demand factor(s) and value based on the good in question. Both content and contextual understanding are assessed here.

Step 6: The entire essay plan is generated and students can add contextual elaboration to each step. While the earlier steps can be automatically marked by the software, human factor authentication by the teacher is required here. Hence, the teacher is required to assess the student’s elaboration at each step to ensure that the arguments are presented in a conceptually sound and contextually accurate manner.

No Technological Unemployment Here

If the above proposal is put to proper use, meaningful rewards can be reaped. For example, the marking module can easily collate student mistakes and if bunching of mistakes is found to occur around a specific step, the teacher can then concentrate subsequent efforts towards rectifying any conceptual mistake identified. Similarly, parameters in the question can be tweaked to ensure that students more practice in any identified areas of weakness.

While the idea of designing an automated marking tool is attractive, one should always caution against over-reliance on such a tool. Human interface is necessary for the software to operate properly. Specifically, students need to possess strong content foundation before they can direct the software towards generating the correct framework.  Educators need to be able to value-add the marking software by looking beyond the structural layout of the plan to assess possible conceptual errors or application weaknesses. This is in line with the intent and purpose of education – i.e. no learning can ever be made passive to the extent that students withdraw themselves from the process. Similarly, the role of a teacher can be complemented with the use of technology, but it can never be replaced.