Chapter 9: Contextual Variables in Written Assessment Feedback in a University-level Spanish Program

Chapter 9

Contextual Variables in Written Assessment Feedback in a University-level Spanish Program[1]  

Kathryn Hill, Deakin University

Ana Maria Ducasse, RMIT University

Abstract

The ‘situated’ nature of assessment may help to explain why feedback interventions are successful in one setting but not in another. This study reanalyses data from an earlier study (Ducasse & Hill, 2019) using a coherent theory of context, Bronfenbrenner’s ecological framework (1979), to investigate contextual influences on teacher feedback practices and learner responses respectively. Participants comprised 15 beginner, intermediate and advanced level students in an Australian university Spanish language program. Data comprised summative feedback on writing tasks and audio-recordings and transcripts of teacher and student think-aloud protocols, and discussions. Data were analysed in relation to the five systems of Bronfenbrenner’s framework (microsystem, mesosystem, exosystem, macrosystem and chronosystem). The study found teacher practices appeared to be influenced by perceptions of the learners in addition to task and performance characteristics (microsystem), by institutional policies and practices (exosystem) and by external professional accreditation standards (macrosystem). Contextual factors found to influence learner responses included perceptions of the teacher, the timing and valence of feedback (microsystem), course level and maturity as a learner (chronosystem) as well as by other subjects the learners were enrolled in (mesosystem). The study demonstrates the utility of Bronfenbrenner’s framework for systematic reflection on contextual influences in language assessment research.

Key words: feedback, context, Bronfenbrenner’s ecological framework, feedback literacy, language assessment literacy

Introduction

The evidence for ‘the power of feedback’ for promoting learning (Black & Wiliam, 1998; Hattie & Timperley, 2007) has made an understanding of the role of feedback in assessment, a core component of teacher assessment literacy. However, too often teachers find the effort put into providing detailed constructive feedback seems to produce little or no effect.  Feedback is often understood as a one-way process by both teachers and learners alike (Urquhart et al, 2014). More recently, however, there has been a shift from the traditional view of feedback as ‘information transmission’ to focus on a more active role for the learner as elicitor as well as user of feedback (Ajjawi & Boud, 2017). The centrality of the learner is emphasized in Dawson, et al.’s (2018) definition of feedback, namely, ‘a process in which learners make sense of information about their performance and use it to enhance the quality of their work or learning strategies.’ (p.2) [emphasis added]. Hence, rather than a one-off, one-way (teacher to learner) episode, feedback is characterised as an ongoing iterative process, or dialogue, between learners and their teachers (Leung, 2020; Nicol, 2010). Moreover, feedback is only considered effective, or ‘productive’, to the extent that it is actually engaged with by the learner and used to effect a change in understanding or behaviour (Boud & Molloy, 2013a; Carless et al., 2011; Price et al., 2011). Hence, understanding why learners fail to engage with feedback, has become an important focus for research (Pitt & Norton, 2017).

Influences on learner responses to feedback

A range of individual learner attributes impacting their responsiveness to feedback have been identified in the literature. These include emotional responses (Dippenaar, 2018; Forsyth & Johnson, 2017), learner ‘mindset’ (i.e., perceptions of whether ability is fixed or malleable) (Dweck, 2002), defence style (response to perceived threats to self-concept) (Forsyth & Johnson, 2017), and goal-orientation (Storch & Wigglesworth, 2010).

In addition to learner attributes, there has been an increasing recognition of the ‘situated’ nature of learning (Brookhart et al., 2006) as well as of assessment and feedback (Larenas & Brunfaut, 2018; Scarino, 2013; Xu & Brown, 2016). For example, Ajjawi et al. (2017) suggest that contextual differences may explain why feedback interventions which have been successful in one setting may not necessarily succeed in others. Hence, they argue that an understanding of the limitations and affordances for effective feedback within the broader system is critical to improving feedback literacy – teachers’ ability to provide effective feedback and students’ ability to “make sense of [feedback] information and use it to enhance work or learning strategies” (Carless & Boud, 2018).

Theory of context

The context for teaching and learning has been characterized in various ways, e.g., as macro-and micro-levels (Turner & Purpura, 2015), macro-sociocultural and micro-institutional levels (Larenas & Brunfaut, 2018), external and local levels (Ivinson & Murphy, 2007) or textual, interpersonal, instructional and sociocultural levels (Chong, 2020). However, Van Lier (2005) argues that researchers need to employ an overarching theory of context to enable a ‘consistent and systematic view of context and a clear connection between person and context’ (p. 205). Ajjawi et al. (2017) investigated the utility of one such theory, Bronfenbrenner’s ecological framework of human development (1979), for exploring the relationship between context and feedback interactions and outcomes in health professions education. Bronfenbrenner’s framework allows consideration of the impact of contextual factors on a “focal individual’’ (e.g., a teacher or student). The original framework comprises five nested systems, each seen as influencing an individual’s development: the microsystem, mesosystem, exosystem, macrosystem and chronosystem. More recently, in line with more contemporary, socio-constructivist, views of learning, Neal and Neal (2013) reconceptualised the framework as a set of networked and overlapping (rather than discrete nested) systems to include a focus on social interactions within and between systems. The revised framework comprises:

• • the microsystem comprises the interactions between participants and materials (e.g., curriculum, equipment) in a setting that includes the focal individual (i.e., the language classroom)
  • the mesosystem comprises interactions between the various microsystems which include the focal individual
  • the exosystem comprises interactions external to, yet impacting on, the focal individual
  • the macrosystem comprises socially or culturally determined patterns of interaction, and
  • the chronosystem comprises changes in patterns of interaction over time induced by changes in the environment and/or within the focal individual (e.g., maturation).

Using Bronfenbrenner’s five systems as the organizing principle, the section below on context and feedback will review the evidence on potential contextual influences on teacher practices and learner responses to feedback in the language classroom.

Microsystem

Bronfenbrenner (1979) defines the microsystem as a ‘pattern of activities, roles and interpersonal relations experienced by the [focal individual] in a given setting with particular physical and material characteristics’ (p. 22). For our purposes the ‘focal individual’ is the teacher or learner within a classroom setting

According to Brookhart et al. (2006) the classroom context (termed the ‘classroom assessment environment’) comprises:

• • the teacher’s assessment purposes, methods (and underlying rationale)
  • the characteristics of the assessment tasks
  • the characteristics of feedback
  • the characteristics of the teacher as assessor (e.g., background, training, and experience)
  • the teacher’s perceptions of the students, and
  • the broader assessment policy environment (Stiggins & Conklin, 1992).

Brookhart, et al.’s (2006) study found that the influence of teacher characteristics and assessment practices on achievement eclipsed all other aspects of the classroom context, including learners’ prior achievement. However, the evidence suggests that learners’ responses are influenced both by the objective feedback the teacher provides and how they in fact perceive it (Ajjawi & Boud 2017). For example, whether the feedback reflects the learner’s own beliefs about their ability or the amount of effort they invested (‘face validity’) can affect their perceptions of its value (Boudrias et al., 2014). Another factor is whether learners perceive the feedback to be positive or negative (‘valence’). For example, learners often try to avoid failure or negative evaluation and are less likely to accept feedback which produces a negative emotional response (Elliot & Covington, 2001). Finally, feedback which challenges, or destabilises, the learner’s pre-existing understandings or experience, has also been shown to cause a defensive response (Rogers, 2012).

Learner perceptions of the person providing the feedback have also been shown to be important. In particular, perceptions of the credibility (or professional competence), and personal characteristics (such as perceived authenticity) of the teacher have both been found to influence the value learners attribute to feedback (Boudrias et al. 2014; Eva et al. 2012). Telio et al. (2015) argue that these credibility judgements are influenced by learners’ perceptions of the quality of the learning relationship, or ‘educational alliance’. This alliance, they argue, stems from the learner’s perception of a shared understanding of goals and how these can be achieved; credibility judgements; and a relationship of mutual liking, trust, and value (Telio et al., 2015). The authors conclude that developing strong ‘educational alliances’ between teachers and learners is critical to improving learners’ responsiveness to feedback. Together these findings serve to underscore the importance of the social and relational aspects of context which motivated Neal & Neal’s (2017) reformulation of Bronfenbrenner’s original framework (Ajjawi & Boud 2017; Esterhazy, 2018). Research also suggests that uptake of feedback is more likely where there is a relationship of trust (Carless, 2009), and where learners experience some level of control over the process (Dann, 2019), particularly in the case of low-performing students (Carless, 2009).

Mesosystem

The mesosystem comprises the different microsystems in which the focal teacher or learner participates. A university student, for example, is typically enrolled in subjects from different disciplines in addition to their language studies. Ajjawi et al. (2017) hypothesize that a lack of continuity between expectations and feedback practices across different microsystems may mean a learner who regularly seeks and engages with feedback in one class may feel less motivated to do so in another. Similarly, a teacher who teaches across multiple classes may find feedback strategies which actively engage learners in one class are less effective in another.

Exosystem

The exosystem comprises the systems and personnel within the institutional setting which interact with the settings containing the focal individual. According to Leung (2020), the ‘infrastructure of any teaching programme is a complex web of cultural, intellectual, financial, organizational, policy, physical … and social affordances and constraints’ (p. 99). Examples include:

• • the institution’s assessment policies, procedures, and culture (i.e., the social and pedagogical values, beliefs, and practices specific to that institution)
  • features of the learning infrastructure such class size, teaching environment (e.g., lecture theatre, lab, online), and timetabling (Andon et al., 2017; Inbar-Lourie & Levi, 2015; Inbar-Lourie, 2008), and
  • aspects of curriculum design such as the timing of assessment and feedback (Ajjawi et al., 2017).

Macrosystem

At the level of the macrosystem, teacher feedback practices and learner expectations can be seen as framed by socio-political factors such as the government education and assessment policies and sociocultural norms (e.g., societal expectations and beliefs about teaching, learning and assessment) which distinguish one educational jurisdiction from another (Bronfenbrenner, 1979; Ivinson & Murphy, 2007; Neal & Neal 2013; Turner & Purpura, 2015). For example, Winstone & Boud (2019) found the Australian university students in their study were more responsive to feedback than their UK counterparts. In addition, teachers are often called on to reconcile their assessment and feedback practices with contemporary views of best practice in language learning and assessment (Larenas & Brunfaut, 2018; Scarino, 2013). Finally, the literature suggests discontinuity between discipline-specific feedback conventions also has the potential to influence how learners experience feedback (Anderson, 2013; Clarke & Gipps, 2000; Esterhazy, 2018; Xu & Brown, 2016).

Chronosystem

Finally, the chronosystem allows us to consider how a teacher’s practices or learner’s responses to feedback may change over time, e.g., in response to changes in the learning environment. Ajjawi et al. (2017) for example, reflected on how feedback interactions and outcomes might change for a notional learner as they progress through the campus-based and clinical stages of their studies in the health sciences. Similarly, in his study Leung (2020) suggests the disposition of an international student towards feedback may have been influenced by her experiences with feedback in her country of origin. Practices or responses may also change as a result of changes within the teacher or learner themselves over time. For example, the teacher may have gained more experience and the student may have matured as a language learner.

A summary of the potential contextual influences on teacher feedback practices and learner responses to feedback discussed in the literature reviewed in this section is presented in Table 1 under the relevant column or centred when applying to both Learner Responses and Teacher Practices.

Table 1. Summary of contextual factors

Rationale

Leung (2020) has identified a need for research into learner responses to feedback which is situated in specific curricular contexts (p.105). Others have called for empirical studies on the influence of contextual factors on teacher assessment (and feedback) practices (Xu & Brown, 2016). Ajjawi et al.’s (2017) paper appears to be one of the few to use a coherent theoretical framework to consider potential contextual influences on both feedback practices and learner responses. However, while Ajjawi et al. (2017) used Bronfenbrenner’s ecological framework to reflect on the experience of a hypothetical learner at different stages of a health professions education course, this study will use the framework to consider how contextual factors appear to influence actual teacher feedback practices and learner responses across different levels of a university-level language program.

Research questions

This paper investigates the influence of context on feedback using data from a previous study of summative feedback in a Spanish language program at an Australian university (Ducasse & Hill, 2019). Specifically, this previous study investigated how the teacher’s feedback practices might inform revisions to a teacher assessment literacy research tool and vice versa (praxis). In this study we revisited the data using the five systems of Bronfenbrenner’s ecological framework of human development (1979) to investigate two main research questions:

1. 1. How do contextual factors influence a teacher’s written feedback practices, and
   2. How do contextual factors influence learner responses to written feedback?

For RQ1 the focal individual is the language teacher and for RQ2 the focal individual is the student.

Details of original study

This section summarises the methods used in the original study (Ducasse & Hill, 2019).

The study was carried out in collaboration by a university Spanish teacher (herein referred to as T) and an external language assessment researcher (herein referred to as R). Participants comprised 15 student volunteers from beginner (CEFR A1), intermediate (CEFR B1), and advanced (CEFR C) levels of a university Spanish program (i.e., five from each level). Data collection progressed in four distinct stages and focussed on feedback on the final assessment task for each of the three course levels (Table 2).

Firstly, T produced think aloud protocols (Ericsson & Simon, 1993) as she provided written feedback on each student. Secondly, R met with each student (herein referred to as S) and asked them to think aloud as they read their feedback and then discuss their responses. Thirdly, R asked T to think aloud as she read her original feedback and again as she read a transcript of the respective student’s response to their feedback. Both then discussed her reactions. Finally, the T and R met to discuss themes emerging from data from each of the three levels in turn.

Transcriptions of audio-recordings of these sessions provided the data used in the current study.

Table 2. Assessment tasks for each level

Analysis

Data were analysed in NVIVO (Version 11) by both T and R. Each independently assigned provisional codes for each transcription before comparing coding, revising, re-organising, or merging existing codes and creating new coding categories as the analysis progressed.

In the first instance, coding was organised under the headings of macro, institutional and classroom contexts. However, this representation of context did not allow us to account for potentially important individual teacher or student variables, which have traditionally been viewed as distinct from the classroom context (Brookhart et. al. 2006; Norris, 2014). By taking the perspective of the focal individual, Bronfenbrenner’s framework allows teachers and learners and the relations between them to be viewed as integral to their respective contexts. That is, when the learner is the focal individual, the teacher can be treated as a component of the learner’s classroom context and vice versa.

Results and discussion

This section will discuss the results for each of the two research questions respectively.

RQ 1. How do contextual factors influence the teacher’s written feedback practices?

The nature of the assessment task (microsystem) appeared to have a significant influence on the type of feedback provided to students. Feedback for the Beginner-level task, a grammar-focussed written exam, largely comprised a summative tick for each correct sentence, whereas feedback for the Advanced-level (Reflection) task, took a variety of formative forms from ‘smiley faces’ to explanatory comments.

There was also evidence that T instinctively adjusted the terminology used in her feedback according to her beliefs about what students “should know” (microsystem), based on her knowledge of the students’ first language, previous experience in learning Spanish and/or other languages, what they had previously been taught (by herself or others) and previous interactions in class:

T: From the questions the students ask in class I get a feel for the kind of comments I can put on their work. If they ask in grammatical terms, then I can write that on their paper. But if they ask like “what’s the difference between a direct object and indirect object?” and the rest of the class is going aargh [I know that most students already understand this]

Regarding the focus of feedback, several Advanced-level students questioned the emphasis on grammatical accuracy in their feedback when it was not one of the published criteria for the Reflection task (Table 2).

T: Ok. I have to look a bit closely at this because I must be missing them. There can’t be no errors. […] there we go, I found one. He incorrectly formed a noun. He needs to put capital letters for ‘United States’ [S 8_Pe].

Here T appeared to have been influenced by some unstated disciplinary conventions (macrosystem). That is, while initially justifying this discrepancy to R by insisting students at [advanced] level should demonstrate a high level of accuracy as a matter of principle, T later reflected that she had probably been influenced by her training as an assessor of the official test for accreditation of Spanish teachers in Spain (the DELE), which demands a high level of accuracy.

The focus on accuracy at the expense of other features of the writing was also partly an artefact of the way T had set up the marking sheet for the Reflection task (microsystem). That is, she realised that she had marked the ‘official’ criteria on the marking sheet while feedback on the paper itself (i.e., the feedback students would see) related to additional, ‘unpublished’, criteria including accuracy: ‘T: I’ve really mucked up… When I tick on the column [in the marking grid] I’m not really ticking on the page, so they can’t see, I’m not marking on the page what’s good actually.’ This omission was also possibly influenced by the fact that, while students regularly received feedback during semester, they would not normally receive written feedback on their end of semester assessments (exosystem: a discussion follows). When reflecting on the feedback she provided for students’ earlier draft versions of the Advanced-level (Reflection) task (which were not graded), T remarked:

T: I think it is interesting what is marked as feedback to the student when I am not thinking remotely about the mark or providing a grade. The feedback is not from a ‘this is all wrong’ perspective rather than the idea that a ‘you might like to know this’. ‘You can take it up ‘.

This more formative orientation on earlier drafts of their work was corroborated by students: It was not all correcting structure… cf Sometimes it was like “I really like this phrase”. [S8_E]

The number of errors in a piece of work (microsystem) appeared to have influenced the extent to which T was able to focus on other features of the writing.

T: I’m able to comment a lot more on her reflection because by the time I get to this I’m spending less time on corrections. It shows I don’t only focus on mistakes when they’re not there distracting me. I can actually comment on what’s going on in the rest of the work.

In addition, there is evidence that T unconsciously tailored her feedback to reflect the learners’ perceived orientation to the task (microsystem). For example, when asked if there were any differences in her feedback patterns across students, T noticed that for two of the Advanced-level students she had focused on accuracy at the expense of other features of the work:

T: Looking across the five Level-8 pieces three are showered with content ticks and two have none. I find this very unnatural for me not to have ticked for ideas. It is almost as if I was busy looking at expression without even noticing the content!

Further discussion revealed that this focus reflected the students’ known preference for detailed grammatical feedback:

T: So it’s interesting and knowing that he wants the correction, also the little microscopic things, and then I get distracted and forget the reflections and like, the singular and plural, so it’s been corrected.

In contrast, the following comment reflects T’s knowledge about this student’s more literary orientation:

T: I’m going to give her a bit of stylistics here because she would probably like that. [S8_E]

During discussions, T commented that she knew far less about Beginner-level students due to their limited ability to communicate in the exclusively Spanish-medium classes. (microsystem), as well as to the larger class size (microsystem) (n=28). As a result, feedback to them was less likely to be tailored to reflect personal preferences.

Finally, a range of institutional (exosystem) factors were found to influence the teacher’s written feedback practices (RQ1) specifically, the primary audience for feedback as well as the level of detail provided. The first was an institutional documentation policy stipulating that formal assessment tasks must be retained for the records. This meant that students needed to request a formal appointment with T if they wanted to see the feedback on their end of semester assessments and this rarely happened in practice. The second was a re-marking policy where students could apply to have their work re-marked (i.e., by another assessor) on request. As a result, T commented that she tends to mark final assessments with the second marker, rather than the student, in mind:

T: That’s very perceptive of [the student] to say she didn’t really understand the marking but she never expected to get it back anyway and to be fair none of the markings that were on those texts that were corrected were ever meant to be seen by students. They weren’t going to see them, ever. so … in a way she wasn’t the audience. Well, she is the audience if she made an appointment to see her marks but so few people do it that’s why I said she wasn’t the audience. It’s just a reminder for me to calculate the marks.

In other words, the primary audience for the feedback was herself (for the purpose of calculating marks) and a potential second marker, rather than the student. This meant that she needed to provide sufficient detail for a second marker to see the basis for the mark awarded. This also reflects a commonly reported practice of using feedback to justify the grades awarded, rather than to support the learners to improve (Boud & Molloy, 2013b).

In summary, the findings suggest that T’s feedback was influenced by a range of micro-, exo- and macrosystem factors. These findings are summarised in Table 3 below.

Table 3. Contextual influences on Teacher feedback practices

 RQ2 How do contextual factors influence student responses to feedback?

Contextual influences on students’ responses to feedback (RQ2) will be considered in terms of their ‘dispositions’, or attitudes, towards the feedback and of their intention to act on the feedback respectively.

Disposition

In a study of postgraduate level TESOL students, Andon et. al (2017) distinguished four distinct dispositions towards feedback ranging from outright rejection to uncritical acceptance. However, while some types of feedback may have been preferred over others, none of the participants questioned the essential value of the feedback they received.

At the microsystem level the high degree of acceptance was attributed to T’s undisputed authority (or credibility) in this context:

T knows Spanish; I’m not going to argue [Intermediate_Ci]

Interestingly, this is contrasted with the perceived authority of feedback providers in other disciplines:

In my other subjects I can usually justify why I’ve done something. And [in Spanish] if I’m not right I’m definitely wrong… [and the] authority she has as the most accomplished Spanish person … just makes it very easy to receive that feedback [Intermediate_He]

This aligns with research showing that learner responses to feedback are influenced by their perception of the professional competence of the teacher (Boudrias, et al. 2014; Eva, et al. 2012).

There also appeared to be an interpersonal aspect to responses with some students perceiving feedback as an indication of T’s investment in them as individuals as well as in her subject:

I think it was more a curiosity [about what the feedback said] because I always want to do well in Spanish because it’s the subject that I care about the most I think maybe because it makes me [feel] most cared about. [Advanced_E]

R: You don’t feel overwhelmed by the amount of feedback?

No, it’s definitely an indication that the teacher cares and I really respond to it when you can tell the teachers [care] [Advanced_M]

This again highlights the relational nature of feedback (Esterhazy, 2018), with learners responding more positively within a relationship of trust (Carless, 2009).

Students’ relative maturity as second language learners (chronosystem), also appeared to make them more receptive to what might otherwise be perceived as negative feedback.

I like [feedback] but I also detest it. I’ve got a lot better at getting feedback. I’ve got a lot better at being wrong. I definitely used to, but I’ve got a lot better at not taking it personally. I think learning a language throughout school probably made it easier for me. So really not having total command of something but just trying to work [at] it … No one likes to get not great marks but in a language that’s where I kind of expect it a little bit more. [Intermediate_He]

I got so used to just getting everything wrong… I’m just completely used to [the fact that] my grasp of the language is extremely poor and that’s fine coz if I’m just being defenseful [sic] I won’t get better at it. [Intermediate_Ci]

This maturity also appeared to lead to an increase in feedback-seeking behaviour for this student:

I’ve got a lot better at giving things in to get marked, just homework or something, just getting [feedback] so that I can make it better … I knew that I had to do a little bit more to up my level a little bit. I think it’s also to do with the fact that I know how far you have to go to be fluent in a language, how much effort you have to put in [Intermediate_He]

Another aspect of the chronosystem, course (or proficiency) level, appears to have had an influence of the type of feedback students found most valuable, with respondents from Beginner according greater value to feedback on accuracy than students in levels 4 and 8:

I’m only in Spanish 1 so it’s more difficult for me to recognize when I’ve made a mistake because I don’t have as much knowledge about it. [Beginner_R]

I think now that we’re starting to write longer pieces of writing maybe just a comment about the piece as a whole would be useful as well and the quality of writing not just the actual grammatical and spelling mistakes, would definitely be useful [Intermediate_Cl]

The feedback’s pretty good in the sense that she’s corrected my grammar, but I feel like sometimes maybe I phrase things a bit clumsily so maybe if she could give a bit more, I might phrase them in a way that an English person might phrase them but not so much the Spanish was, like it might be grammatically correct but might not sound quite right… most of the students in Spanish 8 sort of have their head around the grammatical rules [Advanced_Pa]

Uptake

Despite acknowledging the value of feedback, students varied in the extent to which they tended to act on it:

If I noticed that there was a repeating pattern like I was making a grammar mistake wrong [sic] a lot, I usually would go and do a few of these exercises [Beginner_R]

I normally read over it and take some of the major bits that like if there’s a rule that I’m consistently getting wrong I’ll try to remember that next time I’m writing but I wouldn’t normally do much more than that [Intermediate_Ci]

If I’m really honest it’s not that I don’t pay any attention to feedback … but I’m not someone who is really pedantic about it and sits there and goes, “oh, I need to fix this” [Advanced_M]

At the level of microsystem, certain characteristics of feedback appeared to influence uptake. The first of these was the timing of feedback. That is, “[f]eedback comments need to be provided at a time that learners are best able to use them.” (Henderson et al., 2019).

The teacher provided feedback] right up until the final date for submission so I definitely proofed it all and definitely missed some things [Advanced_E]

Secondly, the valence (positive or negative tone) of the feedback (microsystem) also appeared to have an influence on uptake. However, contrary to Elliot and Covington’s (2001) findings, this student reported that he was more likely to act on feedback he perceived to be highly critical.

R: So, at some level you’re taking [the feedback] on board, but not in a conscious way?

S: Unless it’s really negative, and this is me being really honest, if it’s really

negative and I get affected by it you know? [Advanced_M]

There also appeared to be a relational aspect to uptake (microsystem) with some students recognising that feedback entails a form of mutual obligation:

T is very passionate about what she does, and she really takes on board with students who ask for more how to get better, how to become a more proficient speaker, so I think [that T thinks], “the people who come to me I’ll help and help and help. If they don’t come to class, there’s not much I could do”. The thing is, you didn’t respect her; you didn’t come to class so how can you expect to pass? [Beginner_S]

Finally, students are typically juggling demands from the various microsystems they are involved in (the mesosystem). Hence a student may fail to act on feedback the context of too many competing demands:

Admittedly a lot of this semester I’ve either looked at it and been like “ok, I need to fix that” or I’ve been like “Ah, I’ve got too many other things to do, I’m not going to worry about it too much”. But after this one I had a look through quite extensively, as [I did] in the quiz [Advanced_Pe]

In summary, contextual influences on learner responses to feedback (RQ2) appeared to be influenced by a range of micro-, meso- and chronosystem factors (Table 4).

Table 4. Contextual influences on Learner responses to feedback

 Conclusion

This study used the 1979 Bronfenbrenner’s ecological framework (Neal & Neal, 2013), to investigate contextual influences on teacher feedback practices and learner responses at three levels of a university-level Spanish language program.

The study identified a broad range of contextual factors which appeared to influence the type, focus and amount of feedback provided by the teacher as well as learners’ disposition towards, and intentions to act on, the feedback they received. Specifically, teacher practices appeared to be influenced by perceptions of the learners as well as task and performance characteristics (microsystem), by institutional policies and practices (exosystem) and by external professional accreditation standards (macrosystem). Contextual factors found to influence learner responses included perceptions of the teacher, the timing and valence of feedback (microsystem), course level and maturity as a learner (chronosystem) and other subjects they were enrolled in (mesosystem).

However, it is important to acknowledge a number of limitations of the study. Firstly, it is based on a small sample of students, from a single language program, taught by a single teacher at a tertiary institution in Australia.  Moreover, the small number of participants and voluntary nature of their participation means that responses were not necessarily representative of the student cohort as a whole and that potentially important factors may not have been captured by the data. The use of think aloud protocols and self-reports, e.g., rather than observational data, means the data are inherently subjective. Furthermore, as they are based on re-analysis of the data, the findings may be biased by the original interpretations and others may have interpreted the data differently.

Nonetheless, the findings point to a number of areas for further investigation. From the learner perspective, what is the basis for the perceived relationship (or educational alliance) with the teacher as well as for judgements of teacher credibility (Telio et al., 2015)? How do students respond to differences in feedback practices across different teachers (mesosystem), discipline areas (mesosystem), and course levels (chronosystem) (Ajjawi et al., 2017)? With regards to teachers, to what extent do differences in feedback practices reflect discipline-specific feedback conventions (Winstone et al., 2020) or indeed, differences in the nature of the target languages themselves (e.g., between script and character-based languages) (macrosystem)?

In summary, this study demonstrates the utility of Bronfenbrenner’s ecological framework Neal & Neal, 2013), for investigating contextual influences in classroom-based assessment. Most importantly, in our view, the framework allows the teacher and learners to be viewed as part of their respective contexts, which means that factors previously seen as distinct from that classroom context, such as learner perceptions of the teacher’s ‘credibility’ or teacher perceptions of student preferences, can now be accounted for.

References

Ajjawi, R., & Boud, D. (2017). Researching feedback dialogue: An interactional analysis approach. Assessment & Evaluation in Higher Education, 4(2), 252-265. https://doi.org/10.1080/02602938.2015.1102863

Ajjawi, R., Molloy, E., Bearman, M. & Rees, C.E. (2017). Contextual influences on feedback practices: An ecological perspective. In D. Carless, S. M. Bridges, C. Chan & R.

Glofcheski (Eds.), Scaling up assessment for learning in higher education (pp. 129-143). Springer. https://doi.org/10.1007/978-981-10-3045-1_9

Anderson, J. R. (2013). Rules of the mind. Psychology Press. https://doi.org/10.4324/9781315806938

Andon, N. J., Dewey, M., & Leung, C. (2017). Tasks in the pedagogic space: Using online discussion forum tasks and formative feedback to develop academic discourse skills at Master’s level. In V. Samuda, K. Van der Branden, & M. Bygate (Eds.), TBLT as a researched pedagogy (pp. 236–263). John Benjamins.  https://doi.org/10.1075/tblt.12.10and

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. https://doi.org/10.1080/0969595980050102

Boud, D., & Molloy, E. K. (2013a). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698–712. https://doi.org/10.1080/02602938.2012.691462

Boud, D., & Molloy, E. K. (2013b). What’s the problem with feedback? In D. Boud & E. Molloy (Eds.) Feedback in higher and professional education: Understanding it and doing it well (pp. 1-10). Routledge.

Boudrias, J-S., Bernard, J-L., & Plunier, P. (2014).  Candidates’ integration of individual psychological assessment feedback. Journal of Managerial Psychology, 29(3), 341-359.  https://doi.org/10.1108/JMP-01-2012-0016

Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Harvard University Press.

Brookhart, S. M., Walsh, J. M., & Zientarski, W. A. (2006). The dynamics of motivation and effort for classroom assessments in middle school science and social studies. Applied Measurement in Education, 19(2), 151-184. https://doi.org/10.1207/s15324818ame1902_5

Carless, D., & Boud, D. (2018). The development of student feedback literacy: enabling uptake of feedback, Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354

Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395–407. https://doi.org//10.1080/03075071003642449

Carless, D. (2009). Trust, distrust and their impact on assessment reform. Assessment & Evaluation in Higher Education, 34(1), 79-89.  https://doi.org/10.1080/02602930801895786

Chong, S. W. (2020). Reconsidering student feedback literacy from an ecological perspective, Assessment & Evaluation in Higher Education, 46(1), 92-104. https://doi.org/10.1080/02602938.2020.1730765

Clarke, S., & Gipps, C. (2000). The role of teachers in teacher assessment in England 1996-1998. Evaluation & Research in Education, 14(1), 38-52 https://doi.org/10.1080/09500790008666960

Dann, R. (2019). Feedback as a relational concept in the classroom. The Curriculum Journal, 30(4), 352-374. https://doi.org/10.1080/09585176.2019.1636839

Dawson, P., Henderson, M., Ryan, T., Mahoney, P., Boud, D., Phillips, M., & Molloy, E. (2018). Technology and feedback design. In M. Spector, B. Lockee., & M. Childress (Eds), Learning, design, and technology. Springer. https://doi.org/10.1007/978-3-319-17727-4_124-1

Dippenaar A. (2018). Stressful feedback post paramedic simulation training. Paper presented at the International Assessment in Higher Education Conference, Manchester, June 2018.

Ducasse, A. M., and K. Hill. (2019). Developing student feedback literacy using educational technology and the reflective feedback conversation. Practitioner Research in Higher Education Journal, 12 (1), 24–37.

Dweck, C. S. (2002). The development of ability conceptions. In A. Wigfield & J. S. Eccles (Eds.), Development of achievement motivation (pp. 57–88). Academic Press. https://doi.org/10.1016/B978-012750053-9/50005-X

Elliot, A. J., & Covington, M. V. (2001). Approach and avoidance motivation. Educational Psychology Review, 13(2), 73-92.  https://doi.org/10.1023/A:1009009018235

Ericsson, K. A., & Simon, H. A. (1998). How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity, 5(3), 178-186.  https://doi.org/10.1207/s15327884mca0503_3

Esterhazy, R. (2018). What matters for productive feedback? Disciplinary practices and their relational dynamics. Assessment & Evaluation in Higher Education, 43(8), 1302-1314. https://doi.org/10.1080/02602938.2018.1463353

Eva, K. W., Armson, H., Holmboe, E., Lockyer, J., Loney, E., Mann, K. & Sargeant, J. (2012). Factors influencing responsiveness to feedback: On the interplay between fear, confidence, and reasoning processes. Advances in Health Sciences Education, 17(1), 15-26. https://doi.org/10.1007/s10459-011-9290-7

Forsyth, A. & Johnson, S. (2017). Thanks, but no-thanks for the feedback. Assessment & Evaluation in Higher Education, 42(6), 850-859.  https://doi.org/10.1080/02602938.2016.1202190

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/10.3102/003465430298487

Henderson, M., Ryan, T., Boud, D., Dawson, P., Phillips, M., Molloy, E., & Mahoney, P. (2019). Conditions that enable effective feedback. Higher Education Research & Development, 38(7), 1401-1416. https://doi.org/10.1080/07294360.2019.1657807

Inbar-Lourie, O., & Levi, T. (2015, March). Implementing formative classroom assessment initiatives: What language assessment literacy knowledge is required? Paper presented at the 37 Annual Language Testing Research Colloquium (LTRC), Toronto.

Inbar-Lourie, O. (2008). Constructing a language assessment knowledge base: A focus on language assessment courses. Language Testing, 25(3), 385-402. https://doi.org/10.1177/0265532208090158

Ivinson, G., & Murphy, P. (2007). Rethinking single sex teaching. Open University Press.

Larenas, S. V. & Brunfaut, T. (2018). An investigation of the language assessment literacy of teacher educators in Chile: Knowledge, practices, learning, beliefs, and context. Final project report. https://www.tirfonline.org/wp-content/uploads/2020/04/TIRF_DDG_2018_Salom%C3%A9VillaLarenas_Final.pdf

Leung, C. (2020). Learning-oriented assessment: More than the chalkface. In M.E. Poehner, & O. Inbar-Lourie (Eds.), Toward a reconceptualization of second language classroom assessment (pp. 85-106). Springer. https://doi.org/10.1007/978-3-030-35081-9_5

Neal, J. W., & Neal, Z. P. (2013). Nested or networked? Future directions for ecological systems theory. Social Development, 22(4), 722–737.  https://doi.org/10.1111/sode.12018

Nicol, D. (2010). From monologue to dialogue: improving written feedback processes in mass higher education, Assessment & Evaluation in Higher Education, 35(5), 501–517. https://doi.org/10.1080/02602931003786559

Norris, J. M. (2014). Some reflections on learning-oriented assessment. Presentation at Roundtable on Learning-Oriented Assessment in Language Classrooms and Large-Scale Assessment Contexts. (2014, October). Teachers College, Columbia University.

Pitt, E., & Norton, L. (2017). ‘Now that’s the feedback I want! ’Students’ reactions to feedback on graded work and what they do with it. Assessment & Evaluation in Higher Education, 42(4), 499-516.  https://doi.org/10.1080/02602938.2016.1142500

Price, M., Handley, K. & Millar, J. (2011). Feedback: Focusing attention on engagement. Studies in Higher Education 36(8), 879–896.  https://doi.org/10.1080/03075079.2010.483513

Rogers, J. (2012). Coaching skills: A handbook (3rd ed.) McGraw Hill.

Scarino, A. (2013). Language assessment literacy as self-awareness: Understanding the role of interpretation in assessment and in teacher learning. Language Testing 30(3), 309–327. https://doi.org/10.1177/0265532213480128

Stiggins, R. J., & Conklin, N. F. (1992). In teachers’ hands: Investigating the practices of classroom assessment. SUNY Press.

Storch, N., & Wigglesworth, G. (2010). Learners’ processing, uptake, and retention of corrective feedback on writing: Case studies. Studies in Second Language acquisition, 32(2) 303-334. https://doi.org/10.1017/S0272263109990532

Telio, S., Ajjawi, R., & Regehr, G. (2015). The ‘educational alliance’ as a framework for reconceptualizing feedback in medical education. Academic Medicine, 90(5), 609–614. https://doi.org/10.1097/ACM.0000000000000560

Turner, C. E., & Purpura, J. E. (2015). Learning-oriented assessment in the classroom. Handbook of second language assessment. DeGruyter Mouton. https://doi.org/10.1515/9781614513827-018

Urquhart, L. M., Rees, C. E., & Ker, J. S. (2014). Making sense of feedback experiences: A multi- school study of medical students’ narratives. Medical Education, 48(2), 189–203. https://doi.org/10.1111/medu.12304

van Lier, L. (2005). Case study. In E. Hinkel (Ed.) Handbook of research in second language teaching and learning (pp.195-208). Routledge. https://doi.org/10.4324/9781410612700

Winstone, N., & Boud, D. (2019). Exploring Cultures of feedback practice: The adoption of learning-focussed feedback practices in the UK and Australia. Higher Education Research & Development, 38(2), 411-425. https://doi.org/10.1080/07294360.2018.1532985

Winstone, N.E., Balloo, K., & Carless, D. (2020). Discipline-specific feedback literacies: A framework for curriculum design. Higher Education, 83, 57–77. https://doi.org/10.1007/s10734-020-00632-0

Xu, Y., & Brown, G. T. L. (2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58, 149-162. https://doi.org/10.1016/j.tate.2016.05.010

Notes