Topic: What is the linguistic wealth that we bring to this course? Date: 14 November 2006 3:06 PM
Subject: Theory into practise... Author: Mondy, Steven
Where do we kick this discussion off? Well, at the moment, our college is in the middle of our mid-term examinations. And I am in the process of thinking about how to test, not only a variety of skill areas, but varying levels and students with different purposes. I have many concerns regarding the appropriateness or validity of all that I do in testing situations. Reading Hughes, I see that testing where possible is to be seen as a group task, and that we should be assessing our own tests and their purposes. I think that essentially this is needed and in theory a given, but I sometimes find that the situations that we find ourselves in make it hard to do. Making our mid-term examinations, while juggling class teaching requirements and college commitments can be quite a handful. I guess what I’m trying to say is that it is not always possible to take time with the test making process, and we may not be able to always give very much attention to evaluation until much later. How do we as teachers start to implement some of the necessary theory into our real life working situations? This is where balance and fortitude is needed in making small changes to the things that we are already doing. We are already using various strategies and techniques, such as suggested in the readings under the different approaches (integrative, communicative etc…). There is definitely a need to understand the effect that our choices of items and complete examinations have on the overall motivation of students within our classes, and to their whole attitude toward language. What do you think?
Steven Mondy
Sunday, October 21, 2007
Validity and reliability - issues and discussion
Topic: Validity and reliability - issues and discussion Date: 22 November 2006 1:35 PM
Subject: Usefulness??? Author: Mondy, Steven View PeopleLink Options for this User
It should be understood that validity and reliability are but two categories within the general notion of understanding a test’s usefulness. According to Bachman & Palmer, we need to understand a test’s usefulness ~ encompassing reliability, construct validity, authenticity, interactiveness, impact, and practicality (Bachman & Palmer, 1996, Language Testing in Practice, Oxford University Press, p. 18).
They go on to talk about how reliability is a necessary aspect of validity, but it does not necessarily meet the needs of validity, in all cases. What does this mean? Well, it may have something to do with a test being able to fulfill the requirements of getting consistent results over time, however, using the same test in varying situations (such as a proven grammar test, being used to test someone’s speaking ability) may be inappropriate as it does not address all the other aspects to a students speaking ability. It is, as a solitary test for speaking ability, invalid. What is to be learned from this? One thing is that we have to understand the aspects of the skill area that we are testing, to design not only a reliable test, but also a test that retrieves the results that we should be looking for.
We need to also understand that we should to be careful about using tests or test items that we know have worked in the past, with our existing students. There could be many things that invalidate the test. If we aren’t careful of what it is we need to test (our objectives), what we have actually taught in our classes, and our student’s needs, we may end up with results that are actually more harmful, because of bad backwash. Do I make any sense?
Steven Mondy
Subject: Usefulness??? Author: Mondy, Steven View PeopleLink Options for this User
It should be understood that validity and reliability are but two categories within the general notion of understanding a test’s usefulness. According to Bachman & Palmer, we need to understand a test’s usefulness ~ encompassing reliability, construct validity, authenticity, interactiveness, impact, and practicality (Bachman & Palmer, 1996, Language Testing in Practice, Oxford University Press, p. 18).
They go on to talk about how reliability is a necessary aspect of validity, but it does not necessarily meet the needs of validity, in all cases. What does this mean? Well, it may have something to do with a test being able to fulfill the requirements of getting consistent results over time, however, using the same test in varying situations (such as a proven grammar test, being used to test someone’s speaking ability) may be inappropriate as it does not address all the other aspects to a students speaking ability. It is, as a solitary test for speaking ability, invalid. What is to be learned from this? One thing is that we have to understand the aspects of the skill area that we are testing, to design not only a reliable test, but also a test that retrieves the results that we should be looking for.
We need to also understand that we should to be careful about using tests or test items that we know have worked in the past, with our existing students. There could be many things that invalidate the test. If we aren’t careful of what it is we need to test (our objectives), what we have actually taught in our classes, and our student’s needs, we may end up with results that are actually more harmful, because of bad backwash. Do I make any sense?
Steven Mondy
Validity and reliability - issues and discussion 2
Topic: Validity and reliability - issues and discussion Date: 4 December 2006 11:41 PM
Subject: Re:Very testy indeed Author: Mondy, Steven
I kind of resist using anything more than once as I feel, now this is only on a personal level, that it isn't fair to the student who pays X amount of $ to get up to date, well thought out, original, and unique material. Of course, re-inventing the wheel is a laborious and somewhat futile endeavor, yet shouldn't we try to personalize our teaching as much as possible. If students know that a test is being used in basically the same format for years on end, then doesn't that automatically invalidate the test.
I understand that the issue of creating and maintaining reliability is paramount, yet I think that is also essential that we remain vibrant and abreast to the changes in our teaching environment. That may mean whole changes to not only the test, but to the content of the course and the objectives.
Each student is an individual, learning within a certain context. Our programs need to have flexibility in order to respond to that. It’s too easy to expect students to fit our models. Yet, maintaining our formats, will also maintain our teaching objectives, and keep up standards, not to mention strengthen the basis on which we can make judgments about overall ability.
What are we to do?
Steven Mondy
Subject: Re:Very testy indeed Author: Mondy, Steven
I kind of resist using anything more than once as I feel, now this is only on a personal level, that it isn't fair to the student who pays X amount of $ to get up to date, well thought out, original, and unique material. Of course, re-inventing the wheel is a laborious and somewhat futile endeavor, yet shouldn't we try to personalize our teaching as much as possible. If students know that a test is being used in basically the same format for years on end, then doesn't that automatically invalidate the test.
I understand that the issue of creating and maintaining reliability is paramount, yet I think that is also essential that we remain vibrant and abreast to the changes in our teaching environment. That may mean whole changes to not only the test, but to the content of the course and the objectives.
Each student is an individual, learning within a certain context. Our programs need to have flexibility in order to respond to that. It’s too easy to expect students to fit our models. Yet, maintaining our formats, will also maintain our teaching objectives, and keep up standards, not to mention strengthen the basis on which we can make judgments about overall ability.
What are we to do?
Steven Mondy
Reliability in once-off tests
Topic: Validity and reliability - issues and discussion Date: 26 November 2006 5:45 PM
Subject: Reliability in once-off tests Author: Mondy, Steven
How do we ensure reliability in a test we make and use only once? Often I find myself in a situation in which I’m unable to use a test again in the same kind of context, or with students with similar abilities. I often find that I may need to wait 2-3 years before actually being able to teach the same content. By that time, the tests lose validity? Reliability is closely related to having items that are consistent over time. Hughes and Bachman talk about creating item banks, which store items that we know are reliable, in order to reduce the effort that’s needed to make tests. I guess this is one way to be more certain that whether a test is a one-time thing, or to be used again, the content of the test will be reliable. Also, we shouldn’t understand a test only in terms of a complete, unchangeable entity, but a series of well-formulated items that put together correctly, can be reliable and valid for each and every time we use them. So, what I’m trying to say here is, whenever we make a test, we should try to create a test that may have items that are often used again and again, but not necessarily tests as a whole that are used again and again. Is it really possible to use an intact, unchangeable test in exactly the same way with new students anyway?
I’ve also been thinking of how one can develop tests that are specifically used more as a teaching tool, rather than a measurement or evaluation instrument. If students are prevented from seeing a test and analyzing the mistakes that are made in order to solely ensure reliability, then the student is missing a valuable learning experience. I think that tests can be used as a way to have students experience giving an answer, and then checking their answers with a model, or having an instructed lesson following the tests administration (looking over the actual test) in order to clarify the main points of the test within a student’s mind. Of course, we may need to forgo the ability to use the test in the future, but what can be gained may outweigh the possible disadvantages. This may be where the idea of item banks may help?! What do you think?
Steven Mondy
Subject: Reliability in once-off tests Author: Mondy, Steven
How do we ensure reliability in a test we make and use only once? Often I find myself in a situation in which I’m unable to use a test again in the same kind of context, or with students with similar abilities. I often find that I may need to wait 2-3 years before actually being able to teach the same content. By that time, the tests lose validity? Reliability is closely related to having items that are consistent over time. Hughes and Bachman talk about creating item banks, which store items that we know are reliable, in order to reduce the effort that’s needed to make tests. I guess this is one way to be more certain that whether a test is a one-time thing, or to be used again, the content of the test will be reliable. Also, we shouldn’t understand a test only in terms of a complete, unchangeable entity, but a series of well-formulated items that put together correctly, can be reliable and valid for each and every time we use them. So, what I’m trying to say here is, whenever we make a test, we should try to create a test that may have items that are often used again and again, but not necessarily tests as a whole that are used again and again. Is it really possible to use an intact, unchangeable test in exactly the same way with new students anyway?
I’ve also been thinking of how one can develop tests that are specifically used more as a teaching tool, rather than a measurement or evaluation instrument. If students are prevented from seeing a test and analyzing the mistakes that are made in order to solely ensure reliability, then the student is missing a valuable learning experience. I think that tests can be used as a way to have students experience giving an answer, and then checking their answers with a model, or having an instructed lesson following the tests administration (looking over the actual test) in order to clarify the main points of the test within a student’s mind. Of course, we may need to forgo the ability to use the test in the future, but what can be gained may outweigh the possible disadvantages. This may be where the idea of item banks may help?! What do you think?
Steven Mondy
Reliability in once-off tests 2
Topic: Validity and reliability - issues and discussion Date: 30 November 2006 10:07 PM
Subject: Reliability in once-off tests Author: Mondy, Steven
I like to think of a test as something that not only gauge someone's ability, but as a tool that helps a student see where they need to improve. I was doing some speed reading work in a class today that had simple multiple choice recognition items that helped students see a general overall comprehension score in relation to the time it took them to read a passage. By the time we got through 4 passages, students were able to see a pattern being established, which helped them make their own judgements on their reading ability.
In other classes, I tend to buck our school policy a little and let students mark their own tests, or their partners. One disadvantage would be that it could discourage students who are doing badly. On the other hand, it promotes a common mateship within a class, and a little bit of a competative spirit. Handled well, it can be quite a motivating tool, and reduce the negative image toward taking tests. I understand the need to withold tests on large scale apptitude type tests, but for classroom evaluation and learning, there are times when students need to see the test.
Does anyone else agree with this?
Steven
Subject: Reliability in once-off tests Author: Mondy, Steven
I like to think of a test as something that not only gauge someone's ability, but as a tool that helps a student see where they need to improve. I was doing some speed reading work in a class today that had simple multiple choice recognition items that helped students see a general overall comprehension score in relation to the time it took them to read a passage. By the time we got through 4 passages, students were able to see a pattern being established, which helped them make their own judgements on their reading ability.
In other classes, I tend to buck our school policy a little and let students mark their own tests, or their partners. One disadvantage would be that it could discourage students who are doing badly. On the other hand, it promotes a common mateship within a class, and a little bit of a competative spirit. Handled well, it can be quite a motivating tool, and reduce the negative image toward taking tests. I understand the need to withold tests on large scale apptitude type tests, but for classroom evaluation and learning, there are times when students need to see the test.
Does anyone else agree with this?
Steven
Reliability in once-off tests 3
Topic: Validity and reliability - issues and discussion Date: 4 December 2006 9:08 PM
Subject:Reliability in once-off tests Author: Mondy, Steven
Unfortunately, the speed reading is only a unit within a more general course on reading skills, the purpose of which is to increase students ability in gaining information more efficiently. The book I've enlisted is "More Reading Power", which has a series of reading passages in it (all roughly the same length and complexity). Each of the readings can/should be done within a five minute time span. It has lists of reading time to words/minute and a graph for students to plot their own reading time. The onus is on the student themselves to do the comprehension without cheating, mark their own papers, and record all information.
The activity has worked, but I recommend strongly that it be used with high intermediate~advanced students anyway (who are more likely to be motivated to doing the activity).
As for assessment, I'm still struggling on that one. As it is a skill that I want them to improve, I consider their achievements in increasing their comprehension to reading speed a key factor. This is where some kind of criterion referenced list (as a rubric) may come into play to see if each student has 1. been able to improve and 2. If students have achieved a certain level of proficiency.
I'm still juggling what to do, but luckily I still have time (with the approach of our winter vacation).
Hope that I have shed some light on how I approach or plan to approach speed reading.
As for more standardized tests that you are referring to, I can just say that lots of practice may be the key. In those practice tests, you can show students visually on their graphs, and this may give them the motivation to soldier on. You may need to think very carefully about the material you're teaching in close relationship to the students actual abilities, and think about how to test to increase positive results in the students minds. We can't just rely on intrinsic motivation (within the students), but look at giving them an outside reason to be motivated. This may have to be a step by step process, somewhat like a reward system, however, the rewards may need to be made explicit, in terms of concrete visual pictures (graphs and so forth). Do I make any sense? I think I may have just babbled and not really synthesized what I actually mean...!
Steven Mondy
Subject:Reliability in once-off tests Author: Mondy, Steven
Unfortunately, the speed reading is only a unit within a more general course on reading skills, the purpose of which is to increase students ability in gaining information more efficiently. The book I've enlisted is "More Reading Power", which has a series of reading passages in it (all roughly the same length and complexity). Each of the readings can/should be done within a five minute time span. It has lists of reading time to words/minute and a graph for students to plot their own reading time. The onus is on the student themselves to do the comprehension without cheating, mark their own papers, and record all information.
The activity has worked, but I recommend strongly that it be used with high intermediate~advanced students anyway (who are more likely to be motivated to doing the activity).
As for assessment, I'm still struggling on that one. As it is a skill that I want them to improve, I consider their achievements in increasing their comprehension to reading speed a key factor. This is where some kind of criterion referenced list (as a rubric) may come into play to see if each student has 1. been able to improve and 2. If students have achieved a certain level of proficiency.
I'm still juggling what to do, but luckily I still have time (with the approach of our winter vacation).
Hope that I have shed some light on how I approach or plan to approach speed reading.
As for more standardized tests that you are referring to, I can just say that lots of practice may be the key. In those practice tests, you can show students visually on their graphs, and this may give them the motivation to soldier on. You may need to think very carefully about the material you're teaching in close relationship to the students actual abilities, and think about how to test to increase positive results in the students minds. We can't just rely on intrinsic motivation (within the students), but look at giving them an outside reason to be motivated. This may have to be a step by step process, somewhat like a reward system, however, the rewards may need to be made explicit, in terms of concrete visual pictures (graphs and so forth). Do I make any sense? I think I may have just babbled and not really synthesized what I actually mean...!
Steven Mondy
correlation coefficients and true score?
Topic: Validity and reliability - issues and discussion Date: 27 November 2006 11:36 AM
Subject: correlation coefficients and true score? Author: Mondy, Steven
Last night, I was reading about how knowing the correlation coefficient (positive, negative, and no relationship) is quite a useful way to determine the reliability of a test. The study book was talking about how highly correlated scores can mean high reliability. It got me thinking about specific ways of gauging the reliability of our tests and test items. It was suggested that we could do things like give the same test twice (as long as not too much time has elapsed between the administration of the tests), or have such things as split tests, where we compare the two parts of one test. Up until this point, I didn’t really see the value of giving a test more than once in relatively the same, if not exactly the same format. I thought that it might be a waste of time, as I have been always encouraged to believe a test must be held back in extreme secrecy to not influence cheating and studying for specific items. According to the study materials, we see that there may be a change in the scores in that they may be slightly higher, but we can account for this in working out the students overall performance, and correlate the scores to see how that student performed in relation to other students doing the same test (norm referencing?). Then we can see if it has been error from the test format or from other things such as communicative ability, personal attributes, or other random factors (study materials: module 4).
Another interesting thing that I came across was the comparison of the two students with different scores, and being able to say that a student with a slightly higher score in a test may not be better than the student with the lower score, if we take into account their true score (true score=observed score +-measurement error). This was quite striking as I had assumed that ranking students in order of best to worst according to their raw scores was sufficient to get an idea of general performance. This idea of true score has made me rethink how I look at data, especially in terms of what judgments I make when I see the scores of two or more students are very close. What makes the difference between an A and an A- for example, if the range overlaps so significantly?
What do you think about correlation coefficients and true score?
Steven Mondy
Subject: correlation coefficients and true score? Author: Mondy, Steven
Last night, I was reading about how knowing the correlation coefficient (positive, negative, and no relationship) is quite a useful way to determine the reliability of a test. The study book was talking about how highly correlated scores can mean high reliability. It got me thinking about specific ways of gauging the reliability of our tests and test items. It was suggested that we could do things like give the same test twice (as long as not too much time has elapsed between the administration of the tests), or have such things as split tests, where we compare the two parts of one test. Up until this point, I didn’t really see the value of giving a test more than once in relatively the same, if not exactly the same format. I thought that it might be a waste of time, as I have been always encouraged to believe a test must be held back in extreme secrecy to not influence cheating and studying for specific items. According to the study materials, we see that there may be a change in the scores in that they may be slightly higher, but we can account for this in working out the students overall performance, and correlate the scores to see how that student performed in relation to other students doing the same test (norm referencing?). Then we can see if it has been error from the test format or from other things such as communicative ability, personal attributes, or other random factors (study materials: module 4).
Another interesting thing that I came across was the comparison of the two students with different scores, and being able to say that a student with a slightly higher score in a test may not be better than the student with the lower score, if we take into account their true score (true score=observed score +-measurement error). This was quite striking as I had assumed that ranking students in order of best to worst according to their raw scores was sufficient to get an idea of general performance. This idea of true score has made me rethink how I look at data, especially in terms of what judgments I make when I see the scores of two or more students are very close. What makes the difference between an A and an A- for example, if the range overlaps so significantly?
What do you think about correlation coefficients and true score?
Steven Mondy
Validity and Reliability Brainstorm
Topic: Validity and reliability - issues and discussion Date: 30 November 2006 12:51 PM
Subject: Validity and Reliability Brainstorm Author: Mondy, Steven
I've read the Hughes book as well as a lot of both Bachman books and what seems to be the case is that they present a very good definition of both construct validity and reliability and some of the problems associated with them, and give some comparison in the summaries at the end of the respective chapters. Though, I think it may be up to us to come up with a good idea of how they inter-relate, based on what we have learned about the essence of each.
Personally, I've found the Bachman books (Fundamental considerations in testing and Language testing in practice) more helpful for understanding the concepts, as they go into greater detail (a great part of those books is in the material that we were supplied with).
Maybe if we can start a brainstorm on this discussion board of the major issues involved in looking at how they inter-relate, or affect each other. Let me start listing a few ideas that I have come across in the readings…(please correct me if I’m way off base)
1. Reliability and construct validity are concerned with (a) minimizing measurement error (b) maximizing the effects of the language abilities that we are trying to measure. (Bachman, Fundamental considerations in testing, p.161)
2. Reliability is a necessary condition of construct validity, and hence for usefulness. (Bachman, Palmer, Language testing in practice, p.23). Bachman states in the summary on page 289 (Bachman, Fundamental considerations in testing, p.171) that validity is one of the most important qualities to consider in how we use tests, but reliability is necessary to determine potential sources of measurement error and for being able to predict the effects on test scores.
3. Reliability can be defined in terms of CTS theory (classical true score measurement theory) in terms of the correlation of true scores on tests (Bachman, Fundamental considerations in testing, p.171). As long as there is consistency between the scores of various candidates, we can be more certain of item reliability, and therefore of reduced error in testing technique.
4. Please correct me if I’m wrong, but validity seems to be concerned with having a test that closely matches the circumstances of the candidates taking the test, and creating a testing environment that allows for the best possible results to be obtained.
5………
6………
Can anyone add to the list…
Steven Mondy
Subject: Validity and Reliability Brainstorm Author: Mondy, Steven
I've read the Hughes book as well as a lot of both Bachman books and what seems to be the case is that they present a very good definition of both construct validity and reliability and some of the problems associated with them, and give some comparison in the summaries at the end of the respective chapters. Though, I think it may be up to us to come up with a good idea of how they inter-relate, based on what we have learned about the essence of each.
Personally, I've found the Bachman books (Fundamental considerations in testing and Language testing in practice) more helpful for understanding the concepts, as they go into greater detail (a great part of those books is in the material that we were supplied with).
Maybe if we can start a brainstorm on this discussion board of the major issues involved in looking at how they inter-relate, or affect each other. Let me start listing a few ideas that I have come across in the readings…(please correct me if I’m way off base)
1. Reliability and construct validity are concerned with (a) minimizing measurement error (b) maximizing the effects of the language abilities that we are trying to measure. (Bachman, Fundamental considerations in testing, p.161)
2. Reliability is a necessary condition of construct validity, and hence for usefulness. (Bachman, Palmer, Language testing in practice, p.23). Bachman states in the summary on page 289 (Bachman, Fundamental considerations in testing, p.171) that validity is one of the most important qualities to consider in how we use tests, but reliability is necessary to determine potential sources of measurement error and for being able to predict the effects on test scores.
3. Reliability can be defined in terms of CTS theory (classical true score measurement theory) in terms of the correlation of true scores on tests (Bachman, Fundamental considerations in testing, p.171). As long as there is consistency between the scores of various candidates, we can be more certain of item reliability, and therefore of reduced error in testing technique.
4. Please correct me if I’m wrong, but validity seems to be concerned with having a test that closely matches the circumstances of the candidates taking the test, and creating a testing environment that allows for the best possible results to be obtained.
5………
6………
Can anyone add to the list…
Steven Mondy
Two kinds of tests
Topic: Validity and reliability - issues and discussion Date: 4 December 2006 1:45 PM
Subject: Two kinds of tests Author: Mondy, Steven View
Trying to clarify some ideas, I wish to talk about the following case study, that I witnessed.
Case Study:
A teacher for some reason gives students a surprise test (with no previous warning) and then chooses to give them the same test again, but this time with warning.
The student’s are let in on the process of choosing an alternative testing date.
Alternative number 1: One week later (teacher’ choice)
Alternative number 2: Two weeks later (student’s choice)
Surprise tests are not really fair, in that they take students off guard, putting them into a situation in which they are highly resistant to the process at hand and encourage negative attitudes to the test, subject and teacher, which promotes terrible backwash. However, in such a case as above, where the teacher is trying to gauge student’s ability in a spontaneous situation and correlate it with the students prepared-for responses, then I imagine that alternative number 1 would be the better choice to maintain reliability, but would it be invalid, as the students may feel that they need more time to prepare for the test? However, is it really possible to correlate the two tests anyway? Can the formula for working out the correlation between tests be adapted to this scenario (Taking a student’s impromptu ability and comparing it to their prepared answer, I mean)? I think that it would be quite a stretch, without some sort of criterion referencing, and matching to the course objectives.
If alternative 2 is chosen, then there is too much time between the administration of one test and that of the other ~ and the test would become unreliable due to things such as ‘Test Wiseness, ‘ among other factors. (Study book ~ module 4: 4.8)…
The teacher in the above example seems to be in a no-win situation all round. The whole situation seems to be flawed anyway. The teacher will need to consider the reason for giving this test and what information is actually retrieved, and how beneficial it is to the whole learning environment. At the very least, if the tests were to be undertaken, the teacher will need to warn students of the likelihood that this kind of testing will be given.
I wonder, in what situations these kinds of surprise tests could be used in the classroom? There could be a situation, in some speaking classes, where there is a need to know how the student reacts to a new situation, based on their working knowledge of the language. Unfortunately, when in the real world, we cannot prepare for all situations. So, why not have tests that mimic this communicative task?
Steven Mondy
Subject: Two kinds of tests Author: Mondy, Steven View
Trying to clarify some ideas, I wish to talk about the following case study, that I witnessed.
Case Study:
A teacher for some reason gives students a surprise test (with no previous warning) and then chooses to give them the same test again, but this time with warning.
The student’s are let in on the process of choosing an alternative testing date.
Alternative number 1: One week later (teacher’ choice)
Alternative number 2: Two weeks later (student’s choice)
Surprise tests are not really fair, in that they take students off guard, putting them into a situation in which they are highly resistant to the process at hand and encourage negative attitudes to the test, subject and teacher, which promotes terrible backwash. However, in such a case as above, where the teacher is trying to gauge student’s ability in a spontaneous situation and correlate it with the students prepared-for responses, then I imagine that alternative number 1 would be the better choice to maintain reliability, but would it be invalid, as the students may feel that they need more time to prepare for the test? However, is it really possible to correlate the two tests anyway? Can the formula for working out the correlation between tests be adapted to this scenario (Taking a student’s impromptu ability and comparing it to their prepared answer, I mean)? I think that it would be quite a stretch, without some sort of criterion referencing, and matching to the course objectives.
If alternative 2 is chosen, then there is too much time between the administration of one test and that of the other ~ and the test would become unreliable due to things such as ‘Test Wiseness, ‘ among other factors. (Study book ~ module 4: 4.8)…
The teacher in the above example seems to be in a no-win situation all round. The whole situation seems to be flawed anyway. The teacher will need to consider the reason for giving this test and what information is actually retrieved, and how beneficial it is to the whole learning environment. At the very least, if the tests were to be undertaken, the teacher will need to warn students of the likelihood that this kind of testing will be given.
I wonder, in what situations these kinds of surprise tests could be used in the classroom? There could be a situation, in some speaking classes, where there is a need to know how the student reacts to a new situation, based on their working knowledge of the language. Unfortunately, when in the real world, we cannot prepare for all situations. So, why not have tests that mimic this communicative task?
Steven Mondy
Doing Letter Grades on Excel
Topic: Open for Discussion Date: 5 December 2006 1:27 AM
Subject: Doing Letter Grades on Excel Author: Mondy, Steven
If anyone is interested...
There's an easy way of doing your grades on excel.
You can assign letter grades to the results you get for students...
Check out the following website. I managed to finally do it on my computer. It works...
http://www.uwec.edu/help/Excel03/gr-5finalgrad.htm
careful with the formula...
Make the list of grades you want to give then use the formula,
=VLOOKUP(C21,grades,2,TRUE) for example,
~ the c21: is the actual student grade
~ grades: is the list
~ 2: is the column that has A, B, C in the list,
~True: rounds off the grade to the closest figure...
Try it out, it may help you (or may confuse you...)!
Steven Mondy
Subject: Doing Letter Grades on Excel Author: Mondy, Steven
If anyone is interested...
There's an easy way of doing your grades on excel.
You can assign letter grades to the results you get for students...
Check out the following website. I managed to finally do it on my computer. It works...
http://www.uwec.edu/help/Excel03/gr-5finalgrad.htm
careful with the formula...
Make the list of grades you want to give then use the formula,
=VLOOKUP(C21,grades,2,TRUE) for example,
~ the c21: is the actual student grade
~ grades: is the list
~ 2: is the column that has A, B, C in the list,
~True: rounds off the grade to the closest figure...
Try it out, it may help you (or may confuse you...)!
Steven Mondy
Pre-packaged test
Topic: Purposes and Types of Language Test Date: 7 December 2006 7:53 PM
Subject: Pre-packaged test Author: Mondy, Steven
Recently, I was interested to know the effect of a pre-made packaged test on students within our college. I have a class of Chinese students studying within a basic reading course. I’m using the Oxford Dominoes starter series, and they have downloadable book tests (Multiple-choice items) for each of the books. I initially imagined that the tests put out by the publisher had been analyzed and tested (although, here in Japan it’s somewhat a dangerous assumption to make with publishers, as there seems to be frequent mistakes within textbooks). Anyway, after correcting the tests, there seemed to be a definite pattern emerging with relation to the student’s overall results. I saw that what appeared to be a well-formed test that was directly related to the material that we were studying, produced shockingly low results from students. I tried to think about why this was so?
Looking at the test, I couldn’t see any particularly strange or misleading items. The test was divided into sections, “Setting, Characters, Dialog within the book, Vocabulary, and Plot”. All multiple-choice items had four alternatives as suggested in many of our textbooks, which had fairly straightforward distracters. I was fairly confident that it was a reliable test.
However, after getting the results back, I did start to wonder about its validity within the context of this class, and the test’s suitability for our student’s purposes for studying English. The reliability of this test might have been quite high, yet for some reason my students did not do well…
It may have something to do with why the students are actually studying this course in the first place. Chinese students come to Japan, hoping to skip from our course to some university place in Japan. While they are here, they are required to study in (1) Programs that encourage general English learning and (2) Activities that prepare them for university examinations, and other standardized tests.
The reading course above comes under the first heading, and is a required course for them.
Whether right or wrong, I made the following observations:
~ The test may have lacked Validity: in that the objectives of the school, the teacher (me) and the test may not exactly match the goals of the students. These students may not be able to see the purpose of this kind of course, and may be highly resistant to the class and the test itself, therefore invalidating any test, even before it is given.
~ Reliable and well thought-out tests may not produce the same results for different groups of students.
~ Motivation: I observed that many of the students just gave up, and finished the test early because they were either not ready for it, or didn’t have the ability to do it. Either way, they did not feel compelled to continue with the test.
~ Test format: The test that was prepared for these lower level students was solely comprised of written multiple-choice items. There were no pictures or varying techniques such as cloze or matching that would make it possible for students of differing capabilities to have more chances. Also, it was quite clear that although the length of the test did not disadvantage the students (50 items in 50 minutes, which complies with the 1 minute/1question rule), having 50 questions of the same kind did.
Having undertaken this test, and collating the results, I can see that there are some fundamental problems with validity. In the future, I will be wearier of doing tests that aren’t more closely related to the specific needs of my students. That also means I need to be careful of pre-packaged tests that on the surface look fine, but can end up costing in terms of the harmful effects. I will also be careful to provide more of a variety of techniques within a test for differing learning styles.
Steven Mondy
Subject: Pre-packaged test Author: Mondy, Steven
Recently, I was interested to know the effect of a pre-made packaged test on students within our college. I have a class of Chinese students studying within a basic reading course. I’m using the Oxford Dominoes starter series, and they have downloadable book tests (Multiple-choice items) for each of the books. I initially imagined that the tests put out by the publisher had been analyzed and tested (although, here in Japan it’s somewhat a dangerous assumption to make with publishers, as there seems to be frequent mistakes within textbooks). Anyway, after correcting the tests, there seemed to be a definite pattern emerging with relation to the student’s overall results. I saw that what appeared to be a well-formed test that was directly related to the material that we were studying, produced shockingly low results from students. I tried to think about why this was so?
Looking at the test, I couldn’t see any particularly strange or misleading items. The test was divided into sections, “Setting, Characters, Dialog within the book, Vocabulary, and Plot”. All multiple-choice items had four alternatives as suggested in many of our textbooks, which had fairly straightforward distracters. I was fairly confident that it was a reliable test.
However, after getting the results back, I did start to wonder about its validity within the context of this class, and the test’s suitability for our student’s purposes for studying English. The reliability of this test might have been quite high, yet for some reason my students did not do well…
It may have something to do with why the students are actually studying this course in the first place. Chinese students come to Japan, hoping to skip from our course to some university place in Japan. While they are here, they are required to study in (1) Programs that encourage general English learning and (2) Activities that prepare them for university examinations, and other standardized tests.
The reading course above comes under the first heading, and is a required course for them.
Whether right or wrong, I made the following observations:
~ The test may have lacked Validity: in that the objectives of the school, the teacher (me) and the test may not exactly match the goals of the students. These students may not be able to see the purpose of this kind of course, and may be highly resistant to the class and the test itself, therefore invalidating any test, even before it is given.
~ Reliable and well thought-out tests may not produce the same results for different groups of students.
~ Motivation: I observed that many of the students just gave up, and finished the test early because they were either not ready for it, or didn’t have the ability to do it. Either way, they did not feel compelled to continue with the test.
~ Test format: The test that was prepared for these lower level students was solely comprised of written multiple-choice items. There were no pictures or varying techniques such as cloze or matching that would make it possible for students of differing capabilities to have more chances. Also, it was quite clear that although the length of the test did not disadvantage the students (50 items in 50 minutes, which complies with the 1 minute/1question rule), having 50 questions of the same kind did.
Having undertaken this test, and collating the results, I can see that there are some fundamental problems with validity. In the future, I will be wearier of doing tests that aren’t more closely related to the specific needs of my students. That also means I need to be careful of pre-packaged tests that on the surface look fine, but can end up costing in terms of the harmful effects. I will also be careful to provide more of a variety of techniques within a test for differing learning styles.
Steven Mondy
Instructions
Topic: Validity and reliability - issues and discussion Date: 30 December 2006 11:43 PM
Subject: Instructions Author: Mondy, Steven
When we give a test, don't we do so after a long process of introducing the test procedures throughout our teaching? Before attempting the test, the students in many of my classes are often given a pre-test, or warm up test, so that they can become familiar with the kinds of questions and to the types of instructions that they will face.
Also, in the test, each new kind of test item is introduced with an example with answer, or at least the first question is done for them, so that they can work out what to do, if they actually know the content.
I've played with the idea of using the native language of the test takers, but have opted not to, as I feel that a properly presented test (pre-test, test, and post-test activities) can make it possible for any level student to undertake a test solely in the target language.
However, the kinds of instructions are significantly different depending on the level of the student. I find it quite difficult to write clear instructions for lower level students in particular. To resist the urge to use complicated or confusing jargon.
Another thing that I've been pondering is the use of 'please' in instructions. When writing instructions, do we say...Please circle the correct answer, or do we say circle the correct answer? Many would say that we should make our instructions as clear as possible, with little room for misinterpretation. Does that mean we should take the second option?
Steven
Subject: Instructions Author: Mondy, Steven
When we give a test, don't we do so after a long process of introducing the test procedures throughout our teaching? Before attempting the test, the students in many of my classes are often given a pre-test, or warm up test, so that they can become familiar with the kinds of questions and to the types of instructions that they will face.
Also, in the test, each new kind of test item is introduced with an example with answer, or at least the first question is done for them, so that they can work out what to do, if they actually know the content.
I've played with the idea of using the native language of the test takers, but have opted not to, as I feel that a properly presented test (pre-test, test, and post-test activities) can make it possible for any level student to undertake a test solely in the target language.
However, the kinds of instructions are significantly different depending on the level of the student. I find it quite difficult to write clear instructions for lower level students in particular. To resist the urge to use complicated or confusing jargon.
Another thing that I've been pondering is the use of 'please' in instructions. When writing instructions, do we say...Please circle the correct answer, or do we say circle the correct answer? Many would say that we should make our instructions as clear as possible, with little room for misinterpretation. Does that mean we should take the second option?
Steven
oral testing in pairs
Topic: Assignment 2 discussion and queries Date: 3 January 2007 12:16 PM
Subject: oral testing in pairs Author: Mondy, Steven
The point about the stronger student being able to perform in any situation is quite solid. I think that we as teachers can see their ability to handle a situation where the other speaker cannot answer fully (though are we now testing something different). This reflects an aspect of authenticity, in that it more closely resembles real life situations. However, as the proctor of the test, how do we alter our own perceptions to the changes and grading? This is obviously very subjective, as it is quite spontaneous, and we are unable to prepare fully for such a case. We can however, make provisions in our planning for this kind of test, by stating the course of action to take in such a case. This of course is an ideal situation, in which teachers have the time to do so. Still, there will be problems with reliability, especially proctor to proctor consistency, that will certainly need attention.
As for weaker students being paired, well from my experience, it has been rather detrimental as you have pointed out. Students tend to wind down into a state of nervousness about not being able to answer anything. Then there starts to be a problem with reliability (Bachman 1990), in that random factors, such as emotional state and changes to the test environment interfere with the successful completion of the test. Has anyone else had similiar problems?
Steven Mondy
Subject: oral testing in pairs Author: Mondy, Steven
The point about the stronger student being able to perform in any situation is quite solid. I think that we as teachers can see their ability to handle a situation where the other speaker cannot answer fully (though are we now testing something different). This reflects an aspect of authenticity, in that it more closely resembles real life situations. However, as the proctor of the test, how do we alter our own perceptions to the changes and grading? This is obviously very subjective, as it is quite spontaneous, and we are unable to prepare fully for such a case. We can however, make provisions in our planning for this kind of test, by stating the course of action to take in such a case. This of course is an ideal situation, in which teachers have the time to do so. Still, there will be problems with reliability, especially proctor to proctor consistency, that will certainly need attention.
As for weaker students being paired, well from my experience, it has been rather detrimental as you have pointed out. Students tend to wind down into a state of nervousness about not being able to answer anything. Then there starts to be a problem with reliability (Bachman 1990), in that random factors, such as emotional state and changes to the test environment interfere with the successful completion of the test. Has anyone else had similiar problems?
Steven Mondy
oral testing in pairs1
Topic: Assignment 2 discussion and queries Date: 4 January 2007 4:20 PM
Subject:oral testing in pairs Author: Mondy, Steven
I guess I'm talking about practicality here. Pairing up students and using criterion referenced type tests (both discreet and integrated) in the form of mini role-plays and distinct short conversations (both authentic and non). As for the level, I think that anything but true beginner can handle this kind of testing. As for the true beginner, then I would consider straight out student to teacher interviews (even though there may be a problem with performance anxiety in front of a native speaker in the TLU). Oral interviews for speaking type classes are essential to maintain face validity as Hughes points out on page 33. However, the true beginner may not have enough yet to work with to be able to handle true mini-conversation scenarios. Therefore, one may need to balance in favor of less authentic material to make sure validity is maintained…
Does this at all answer your question?
Steven Mondy
Subject:oral testing in pairs Author: Mondy, Steven
I guess I'm talking about practicality here. Pairing up students and using criterion referenced type tests (both discreet and integrated) in the form of mini role-plays and distinct short conversations (both authentic and non). As for the level, I think that anything but true beginner can handle this kind of testing. As for the true beginner, then I would consider straight out student to teacher interviews (even though there may be a problem with performance anxiety in front of a native speaker in the TLU). Oral interviews for speaking type classes are essential to maintain face validity as Hughes points out on page 33. However, the true beginner may not have enough yet to work with to be able to handle true mini-conversation scenarios. Therefore, one may need to balance in favor of less authentic material to make sure validity is maintained…
Does this at all answer your question?
Steven Mondy
oral testing in pairs2
Topic: Assignment 2 discussion and queries Date: 7 January 2007 11:41 AM
Subject: oral testing in pairs Author: Mondy, Steven
It is not always possible to test a wide range of skills within the objectives of your course, but only a selection. Discreet point testing will make it more likely that the skills you are looking for will more clearly be evaluated, whereas integrated tests might make it very difficult to make judgments about students fairly, as students may answer in a variety of ways.
Steven Mondy
Subject: oral testing in pairs Author: Mondy, Steven
It is not always possible to test a wide range of skills within the objectives of your course, but only a selection. Discreet point testing will make it more likely that the skills you are looking for will more clearly be evaluated, whereas integrated tests might make it very difficult to make judgments about students fairly, as students may answer in a variety of ways.
Steven Mondy
iBT TOEFL ~ reflecting authenticity...
Topic: Purposes and Types of Language Test Date: 13 January 2007 5:19 PM
Subject: iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
I was reading about an interesting development in the iBT (internet based TOEFL test) in the Daily Yomuiri, a Japan-based English language newspaper
(Lawrence J. Zwier, TOEFL Booster / Readings get longer, and tougher, Yomiurihttp://www.yomiuri.co.jp/dy/features/language/20070112TDY15001.htm).
The article talks about a change in the test that is an attempt to make the reading section more authentic, has made it more difficult for some test-takers. The most significant change is the move from what the article calls enumeration type readings (where a point is given, then supporting evidence) to a more complex form where several ideas are interlaced ~ a task requiring more understanding of various reading skills and the purpose for obtaining information. The article implies that the change in the test is to encompass, 1. Reading to find information (such things as speed reading, using thinking skills while reading), 2. Reading for comprehension (the more traditional focus), and 3. Reading to learn (being able to summarize, and then use the information).
This was very interesting to me as one of the main focuses in my college has been point number two, comprehension, then followed by point number one, the development of reading skills. Unfortunately, point number 3 has been missed in our curriculum design phase, which will make it quite difficult for our students in this new version of the iBT.
This semester, I have also discovered that this focus has been detrimental to our students, especially within short answer type questions. They are too used to multiple choice type questions, and are not sure what information to include in a written response. I have tried to change this by not only testing, but also teaching various techniques to get away from just reading for comprehension, such things as, discussion, inference making, and summarization. I believe that we need to essentially test how we teach, which means that we at the college need to adopt new curriculum ideas, to reflect changes to ideas out in the real world ~ which I suspect the new version of the iBT is attempting to do?!
Steven Mondy
Subject: iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
I was reading about an interesting development in the iBT (internet based TOEFL test) in the Daily Yomuiri, a Japan-based English language newspaper
(Lawrence J. Zwier, TOEFL Booster / Readings get longer, and tougher, Yomiurihttp://www.yomiuri.co.jp/dy/features/language/20070112TDY15001.htm).
The article talks about a change in the test that is an attempt to make the reading section more authentic, has made it more difficult for some test-takers. The most significant change is the move from what the article calls enumeration type readings (where a point is given, then supporting evidence) to a more complex form where several ideas are interlaced ~ a task requiring more understanding of various reading skills and the purpose for obtaining information. The article implies that the change in the test is to encompass, 1. Reading to find information (such things as speed reading, using thinking skills while reading), 2. Reading for comprehension (the more traditional focus), and 3. Reading to learn (being able to summarize, and then use the information).
This was very interesting to me as one of the main focuses in my college has been point number two, comprehension, then followed by point number one, the development of reading skills. Unfortunately, point number 3 has been missed in our curriculum design phase, which will make it quite difficult for our students in this new version of the iBT.
This semester, I have also discovered that this focus has been detrimental to our students, especially within short answer type questions. They are too used to multiple choice type questions, and are not sure what information to include in a written response. I have tried to change this by not only testing, but also teaching various techniques to get away from just reading for comprehension, such things as, discussion, inference making, and summarization. I believe that we need to essentially test how we teach, which means that we at the college need to adopt new curriculum ideas, to reflect changes to ideas out in the real world ~ which I suspect the new version of the iBT is attempting to do?!
Steven Mondy
iBT TOEFL ~ reflecting authenticity...2
Topic: Purposes and Types of Language Test Date: 15 January 2007 11:10 AM
Subject: Re:iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
It's good to here that others are having similar issues with the TOEFL/TOEIC...My biggest question is whether these new changes are reflective of what is needed by these students. Are we being led down the road in a chicken and egg race, that will end up in our students getting more confused? I think that all the skills that I mentioned before are purposeful and relevant to teaching practice and learning, yet I don't want a situation where it's the test that dictates what we need to focus on in our classes, unless it is truly what our particular students need to know. Isn't a test an evaluation instrument, and not a prescriptive force?
Steven Mondy
Subject: Re:iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
It's good to here that others are having similar issues with the TOEFL/TOEIC...My biggest question is whether these new changes are reflective of what is needed by these students. Are we being led down the road in a chicken and egg race, that will end up in our students getting more confused? I think that all the skills that I mentioned before are purposeful and relevant to teaching practice and learning, yet I don't want a situation where it's the test that dictates what we need to focus on in our classes, unless it is truly what our particular students need to know. Isn't a test an evaluation instrument, and not a prescriptive force?
Steven Mondy
iBT TOEFL ~ reflecting authenticity...3
Topic: Purposes and Types of Language Test Date: 20 January 2007 11:54 AM
Subject: iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
I can see the value of providing a goal to work towards and providing tasks that will not only develop their abilities to take the test, but through some kind of ‘osmosis’ process in learning, pick up some of the TL (target language). However, this defeats one of the main purposes of an evaluation tool, and it then becomes something else ~ as I said before, a tool to prescribe what some group in society thinks is needed. Isn't it dangerous, and at cross- purposes with ‘authenticity’ to do so? "They need to be able to fulfill certain requirements to succeed in a course," you may ask. Yes they do, and is that the purpose of this kind of test? What is the TOEFL and TOEIC trying to be?
1. A proficiency test (ability in language regardless of ability),
2. or achievement test (how much individuals have been achieving goals of program)
3. or diagnostic test (identify learners' strengths and weaknesses)
4. or placement test (getting info to help position an individuals in a certain place)?
(Hughes, 2005)
Which is the TOEFL or TOEIC, really? In Japan, the TOEIC seems to operate as a placement test for climbing the rungs of the corporate ladder. High score and you get the promotion! TOEFL is closer to the ideals of the proficiency test. And with the new changes is heading towards a closer connection to the real world, and authenticity.
You ask who really cares whether or not they are good speakers? Well, maybe no one does, except the individual who is trying out for the test. Many of my students are still under the illusion that a good score on either of these tests correlates with a good ability in the TL. Probably with the revised TOEFL, they may be right!??? So, do we head down the path of teaching to the test or do we maintain our determination to help these individuals with learning the target language? I think I’ll let you decided on that one for yourself. As for me, I’m still sitting on the fence…
Steven
Subject: iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
I can see the value of providing a goal to work towards and providing tasks that will not only develop their abilities to take the test, but through some kind of ‘osmosis’ process in learning, pick up some of the TL (target language). However, this defeats one of the main purposes of an evaluation tool, and it then becomes something else ~ as I said before, a tool to prescribe what some group in society thinks is needed. Isn't it dangerous, and at cross- purposes with ‘authenticity’ to do so? "They need to be able to fulfill certain requirements to succeed in a course," you may ask. Yes they do, and is that the purpose of this kind of test? What is the TOEFL and TOEIC trying to be?
1. A proficiency test (ability in language regardless of ability),
2. or achievement test (how much individuals have been achieving goals of program)
3. or diagnostic test (identify learners' strengths and weaknesses)
4. or placement test (getting info to help position an individuals in a certain place)?
(Hughes, 2005)
Which is the TOEFL or TOEIC, really? In Japan, the TOEIC seems to operate as a placement test for climbing the rungs of the corporate ladder. High score and you get the promotion! TOEFL is closer to the ideals of the proficiency test. And with the new changes is heading towards a closer connection to the real world, and authenticity.
You ask who really cares whether or not they are good speakers? Well, maybe no one does, except the individual who is trying out for the test. Many of my students are still under the illusion that a good score on either of these tests correlates with a good ability in the TL. Probably with the revised TOEFL, they may be right!??? So, do we head down the path of teaching to the test or do we maintain our determination to help these individuals with learning the target language? I think I’ll let you decided on that one for yourself. As for me, I’m still sitting on the fence…
Steven
Niggily bits
Topic: Assignment 2 discussion and queries Date: 19 January 2007 6:18 PM
Subject: Niggily bits Author: Mondy, Steven
I am really getting into the test creation process and reshaping and reformatting of the test. I'm never satisfied...
I wonder if anyone else is really frustrated with Microsoft word, especially with making multiple choice items ~ it insists on capital letters after the a. b. c. and d.
I was also dealing with a minor issue of using 'not' in a multiple choice item, but decided to redo it on advice from the Hughes book. I was ale to rephrase it so that it wouldn't have the potential of disadvantaging my students.
Actually, it has been quite interesting formulating questions. It's so easy to introduce little things that quickly invalidate an item. In one task, I was going to label an item "Vocabulary:Synonyms" and then ask the students to look for the word that is not a Synonym. I quickly realized that by labeling it "Synonyms" I introduce into the students conscious (or unconscious minds) the expectation to look for synonyms. This in turn may cause a student to make a mistake with what is expected of them, and make the item invalid. I decided to cut the heading, and now it states Section A: Vocabulary.
It's all so easy to introduce unforeseen elements into the test.
Steven Mondy
Subject: Niggily bits Author: Mondy, Steven
I am really getting into the test creation process and reshaping and reformatting of the test. I'm never satisfied...
I wonder if anyone else is really frustrated with Microsoft word, especially with making multiple choice items ~ it insists on capital letters after the a. b. c. and d.
I was also dealing with a minor issue of using 'not' in a multiple choice item, but decided to redo it on advice from the Hughes book. I was ale to rephrase it so that it wouldn't have the potential of disadvantaging my students.
Actually, it has been quite interesting formulating questions. It's so easy to introduce little things that quickly invalidate an item. In one task, I was going to label an item "Vocabulary:Synonyms" and then ask the students to look for the word that is not a Synonym. I quickly realized that by labeling it "Synonyms" I introduce into the students conscious (or unconscious minds) the expectation to look for synonyms. This in turn may cause a student to make a mistake with what is expected of them, and make the item invalid. I decided to cut the heading, and now it states Section A: Vocabulary.
It's all so easy to introduce unforeseen elements into the test.
Steven Mondy
Maybe a stupid question...?
Topic: Assignment 2 discussion and queries Date: 19 January 2007 6:32 PM
Subject: Scoring - Maybe a stupid question...? Author: Mondy, Steven
What is the advantage of scoring a test out of 100, as opposed to dividing by the total and working out a percentage?
I was wondering about this, and can see one advantage of scoring out of 100. That is, it is clearer for the test taker to see the division of scores between the sections, and between various parts. For instance, you may divide up your test into various sections based on a certain taxonomy - such as knowledge vs. application. Also for the untrained eye, it's easier to see the weighting given to various parts, and decide how much effort will be required for each section.
Then there's the marker, who may not be the person who designed the test. They can see the breakdown very clearly after the administration of the test, how much emphasis is placed on various sections...
In spite of this, I'd like to ask everyone if it is necessary to have the total = 100?
Steve
Subject: Scoring - Maybe a stupid question...? Author: Mondy, Steven
What is the advantage of scoring a test out of 100, as opposed to dividing by the total and working out a percentage?
I was wondering about this, and can see one advantage of scoring out of 100. That is, it is clearer for the test taker to see the division of scores between the sections, and between various parts. For instance, you may divide up your test into various sections based on a certain taxonomy - such as knowledge vs. application. Also for the untrained eye, it's easier to see the weighting given to various parts, and decide how much effort will be required for each section.
Then there's the marker, who may not be the person who designed the test. They can see the breakdown very clearly after the administration of the test, how much emphasis is placed on various sections...
In spite of this, I'd like to ask everyone if it is necessary to have the total = 100?
Steve
Testing grammar
Sunday, October 21, 2007
Testing grammar
Topic: Assignment 2 discussion and queries Date: 30 January 2007 1:22 AM
Subject:Testing grammar Author: Mondy, Steven
I think that all speaking tests are quite difficult to administer. There are just so many factors to take into consideration. Our classmate has pointed out a very good point with the testing, or at the very least the acknowledgment of the test-taker’s ability to self-correct. We are often too much caught up in the timing and correctness of procedure to allow students enough time to self-correct (a skill that may become more apparent ‘quite a ways down the track’ in their lang. learning).
From my experience (in Japan), learners tend to do better on written tasks, and have great difficulty producing oral language. There have been those students who the reverse is true, but generally written tests seem to be less invasive, especially if enough time is given. As for the question about giving a written test on paper, well, I think that I read somewhere, maybe in Hughes, maybe Bachman, that we need to maintain face validity ~ testing speaking with a written test somewhat invalidates the procedure.
I would go more for some kind of rubric, that incorporates grammar into one of the sections, along with categories including the one Nico suggested, in that the ability to self-correct is a factor in the level of proficiency that the learner has achieved. Your test seems to be more discreet point testing, and for such a case, then yes, you would have to mark them down on knowing that particular grammar, as long as you have given them enough time to answer the question, and as long as that is your focus within your objectives. A more integrative test, based on a role-play would probably pose more difficulties with rater inconsistencies, but may make grammar less of a focus, and make communication the focal point.
Also remember that, the student’s score is dependent on many factors, and the interpretation of them is relative to the other people taking the same test. Cut off scores don’t remain fixed to a magical 50%.
Steven Mondy
Testing grammar
Topic: Assignment 2 discussion and queries Date: 30 January 2007 1:22 AM
Subject:Testing grammar Author: Mondy, Steven
I think that all speaking tests are quite difficult to administer. There are just so many factors to take into consideration. Our classmate has pointed out a very good point with the testing, or at the very least the acknowledgment of the test-taker’s ability to self-correct. We are often too much caught up in the timing and correctness of procedure to allow students enough time to self-correct (a skill that may become more apparent ‘quite a ways down the track’ in their lang. learning).
From my experience (in Japan), learners tend to do better on written tasks, and have great difficulty producing oral language. There have been those students who the reverse is true, but generally written tests seem to be less invasive, especially if enough time is given. As for the question about giving a written test on paper, well, I think that I read somewhere, maybe in Hughes, maybe Bachman, that we need to maintain face validity ~ testing speaking with a written test somewhat invalidates the procedure.
I would go more for some kind of rubric, that incorporates grammar into one of the sections, along with categories including the one Nico suggested, in that the ability to self-correct is a factor in the level of proficiency that the learner has achieved. Your test seems to be more discreet point testing, and for such a case, then yes, you would have to mark them down on knowing that particular grammar, as long as you have given them enough time to answer the question, and as long as that is your focus within your objectives. A more integrative test, based on a role-play would probably pose more difficulties with rater inconsistencies, but may make grammar less of a focus, and make communication the focal point.
Also remember that, the student’s score is dependent on many factors, and the interpretation of them is relative to the other people taking the same test. Cut off scores don’t remain fixed to a magical 50%.
Steven Mondy
Friday, October 19, 2007
Why does English have a need for pronouns?
13th August, 2006
Recently, I was talking with my wife about the differences between her language (Japanese) and mine (English), and she asked me, “Why does English have the need to keep on restating pronouns?” In Japanese, once the referent is established, there is not much need to keep on using the pronouns, ‘watashi’= I, ‘anata’=you, ‘kare’ =he, ‘kanajo’=she and so on... As an English speaker in my L1 and Japanese in L2, I often find that I lose track of who or what the conversation is about when having a conversation in Japanese, because of the lack of pronouns. My wife often says that the English use of pronouns is redundant, as we have already established what is being discussed (or with a little imagination you’d be able to work it out). I was reading that Japanese is a ‘pro-drop’ language (Wikipedia http://en.wikipedia.org/wiki/Japanese_language), in that it tends to omit words to make an utterance shorter, if the context is known. I think, that English also drops pronouns sometimes, however, not as much as Japanese. So, why in English do we tend to overuse pronouns?
Japanese is said to be a high context culture and English a Low context culture. That is...
High context culture: many things don’t need to be made explicit because of a close connection and understanding.
Low context: societies that need to explicitly spell out things because connections are not as close. (Culture at work: Communicating across cultures, http://www.culture-at-work.com/highlow.html)
Could this explain the need for us to continually check that the listener understands who the referent is? ...And why is necessary for English to paraphrase so often? My wife made a joke and asked, “Don’t people listen the first time?”
Another question that came up was why do some languages (like Romance languages, and even English in some cases) tend to use gender more than others? Why in English, we sometimes refer to a ship as a ‘She’? Why in Italian, Bianca (female) and Bianco (male)? How does culture influence language, and how does language influence culture?
Steven Mondy
Recently, I was talking with my wife about the differences between her language (Japanese) and mine (English), and she asked me, “Why does English have the need to keep on restating pronouns?” In Japanese, once the referent is established, there is not much need to keep on using the pronouns, ‘watashi’= I, ‘anata’=you, ‘kare’ =he, ‘kanajo’=she and so on... As an English speaker in my L1 and Japanese in L2, I often find that I lose track of who or what the conversation is about when having a conversation in Japanese, because of the lack of pronouns. My wife often says that the English use of pronouns is redundant, as we have already established what is being discussed (or with a little imagination you’d be able to work it out). I was reading that Japanese is a ‘pro-drop’ language (Wikipedia http://en.wikipedia.org/wiki/Japanese_language), in that it tends to omit words to make an utterance shorter, if the context is known. I think, that English also drops pronouns sometimes, however, not as much as Japanese. So, why in English do we tend to overuse pronouns?
Japanese is said to be a high context culture and English a Low context culture. That is...
High context culture: many things don’t need to be made explicit because of a close connection and understanding.
Low context: societies that need to explicitly spell out things because connections are not as close. (Culture at work: Communicating across cultures, http://www.culture-at-work.com/highlow.html)
Could this explain the need for us to continually check that the listener understands who the referent is? ...And why is necessary for English to paraphrase so often? My wife made a joke and asked, “Don’t people listen the first time?”
Another question that came up was why do some languages (like Romance languages, and even English in some cases) tend to use gender more than others? Why in English, we sometimes refer to a ship as a ‘She’? Why in Italian, Bianca (female) and Bianco (male)? How does culture influence language, and how does language influence culture?
Steven Mondy
Sapir-whorf
10th August, 2006
What are we to understand from the Sapir-Whorf debate on how a language affects culture, and in turn the individual? I find it quite an interesting discussion, as both sides to the debate give intriguing alternatives...
According to end of the lecture on ‘primitive languages’ (from linguistic voices on campus podcast) the linguist makes a point at the end that a weak version of the Sapir-Whorf hypothesis makes some sense in terms of sexist language (the so called generic ‘he’) as somewhat a warning to be careful of the language we use, as it may influence the way we think. Using ‘he’ or not using ‘he’ can determine how we think about something. When we choose to teach (or not) something in a particular way are we influencing the way our students think, by the language we use? Is Political Correctness a form of ‘linguistic determinism’? How about the movement against euphemisms, that some consider ‘bad language’? How about the elimination of racist terminology in songs (Eminem)? Can we shape society by shaping the language it uses? This taken to the extreme is the use of the so-called ‘Newspeak’ in George Orwell's classic novel Nineteen Eighty-Four, to shape society (Wikipedia -http://en.wikipedia.org/wiki/Sapir-Whorf_hypothesis).
Also the Sapir-Whorf hypothesis assumes that people are limited by the language they speak and they cannot think outside ‘the proverbial box’. In that case, how can anyone be creative?
Does language have an impact on the way we think?
What are we to understand from the Sapir-Whorf debate on how a language affects culture, and in turn the individual? I find it quite an interesting discussion, as both sides to the debate give intriguing alternatives...
According to end of the lecture on ‘primitive languages’ (from linguistic voices on campus podcast) the linguist makes a point at the end that a weak version of the Sapir-Whorf hypothesis makes some sense in terms of sexist language (the so called generic ‘he’) as somewhat a warning to be careful of the language we use, as it may influence the way we think. Using ‘he’ or not using ‘he’ can determine how we think about something. When we choose to teach (or not) something in a particular way are we influencing the way our students think, by the language we use? Is Political Correctness a form of ‘linguistic determinism’? How about the movement against euphemisms, that some consider ‘bad language’? How about the elimination of racist terminology in songs (Eminem)? Can we shape society by shaping the language it uses? This taken to the extreme is the use of the so-called ‘Newspeak’ in George Orwell's classic novel Nineteen Eighty-Four, to shape society (Wikipedia -http://en.wikipedia.org/wiki/Sapir-Whorf_hypothesis).
Also the Sapir-Whorf hypothesis assumes that people are limited by the language they speak and they cannot think outside ‘the proverbial box’. In that case, how can anyone be creative?
Does language have an impact on the way we think?
In what language do deaf people think?
10th August, 2006
In what language do deaf people think? 26 Dec-2003
(http://www.straightdope.com/columns/031226.html)
I have been thinking about this topic recently, as an extension to the ‘wild child’ idea. If a child is brought up in a situation devoid of language what do they think? The article puts forward the argument that deaf people think in signs, if they were lucky enough to learn sign language at an early age. However, what if they didn’t? Do these children develop their own way to interpret and react to the world? For this would support the innatist view? Or do these children not develop any form of device to react to the world? Possibly, their brains just don’t develop the necessary neurons, and they are forced to live in some kind of vegetative state. Having these two options, I would like to think that the human brain is predisposed to make sense of the world, in whatever way.... Steven Mondy
In what language do deaf people think? 26 Dec-2003
(http://www.straightdope.com/columns/031226.html)
I have been thinking about this topic recently, as an extension to the ‘wild child’ idea. If a child is brought up in a situation devoid of language what do they think? The article puts forward the argument that deaf people think in signs, if they were lucky enough to learn sign language at an early age. However, what if they didn’t? Do these children develop their own way to interpret and react to the world? For this would support the innatist view? Or do these children not develop any form of device to react to the world? Possibly, their brains just don’t develop the necessary neurons, and they are forced to live in some kind of vegetative state. Having these two options, I would like to think that the human brain is predisposed to make sense of the world, in whatever way.... Steven Mondy
Secret of the wild child
Secret of the wild child 10th August, 2006
I did some background reading on Genie, at "Secret of the Wild Child"
(http://www.pbs.org/wgbh/nova/transcripts/2112gchild.html)
And I have to say (as a parent of a five year old now) that I am quite appalled at what happened to the poor little girl, not only at her parents, but also by the people who were supposedly helping her. It is quite tragic. I wonder where the line can be drawn between science and humanity...
An interesting thing that occurred to me as I was reading the interview was, why they decided to teach her sign language? Was it that they thought that sign language would give her more opportunities to communicate, a different way to produce language? I am not really sure.
I also started thinking about the difference between input and output. Possibly, many of these scientists were judging her knowledge of language by how much she was ‘producing’...If so, I think that they are only getting half the picture. I often notice the difference between how much I can understand, as opposed to how much I can say in my L2. Poor little Genie may have only got to the stage of being able to produce one, two or three word sentences, but how much was she able to understand? ...And how can we measure the amount of progress in her listening capabilities? I guess, by sticking brain wave probes onto the poor little girl’s head. Science or Humanity? Seriously, I wonder in what ways can we as teachers judge our student’s receptive capabilities, without just making qualitative judgments?
Steven Mondy
I did some background reading on Genie, at "Secret of the Wild Child"
(http://www.pbs.org/wgbh/nova/transcripts/2112gchild.html)
And I have to say (as a parent of a five year old now) that I am quite appalled at what happened to the poor little girl, not only at her parents, but also by the people who were supposedly helping her. It is quite tragic. I wonder where the line can be drawn between science and humanity...
An interesting thing that occurred to me as I was reading the interview was, why they decided to teach her sign language? Was it that they thought that sign language would give her more opportunities to communicate, a different way to produce language? I am not really sure.
I also started thinking about the difference between input and output. Possibly, many of these scientists were judging her knowledge of language by how much she was ‘producing’...If so, I think that they are only getting half the picture. I often notice the difference between how much I can understand, as opposed to how much I can say in my L2. Poor little Genie may have only got to the stage of being able to produce one, two or three word sentences, but how much was she able to understand? ...And how can we measure the amount of progress in her listening capabilities? I guess, by sticking brain wave probes onto the poor little girl’s head. Science or Humanity? Seriously, I wonder in what ways can we as teachers judge our student’s receptive capabilities, without just making qualitative judgments?
Steven Mondy
The Critical Period
The critical period 2 6th August, 2006
I understand that the 'critical Period' hypothesis can be a little easy to hold as an all-encompassing reason for the difficulties that we face in the classroom. Yet, I still feel that we need to have it as a consideration. Especially after discovering an updated version of the ‘critical period hypothesis’
“DeKeyser argues that although it is true that there is a critical period, this does not mean that adults cannot learn a second language perfectly, at least on the syntactic level." (wikipedia,takenAug,6,3:30p.m,
http://en.wikipedia.org/wiki/Language_Acquisition_Device).
Also John. B. Carroll talks of ‘Language learning aptitude’, or the “prediction of how well, relative to other individuals, an individual can learn a foreign language in a given amount of time and under given conditions.” (Wikipedia, Language aptitude, http://en.wikipedia.org/wiki/Language_aptitude)
I think that with enough practice, the right motivation, and the right personality, it is quite possible to achieve a high level of competency. Yet, how many of our students can be said to fulfill all these qualifications? As you have said, most of us out there have various responsibilities that take away our time to study, or restrict us in some way.
I just wanted to point to the Critical Hypothesis as a thing to consider in making judgments about our future plans and actions. If it is possible, as Carrol, B., and Pimsleur, P., suggest, we need to gauge in some way, our student’s capacity to learn. Leading us back to the thing I was talking about earlier – Tests. Arghhh! However, there does need to be some thought given to how much are my students ready for what I’m about to present. Depending on the learner’s capacity to learn, we can therefore make judgments on how to proceed.
I understand that the 'critical Period' hypothesis can be a little easy to hold as an all-encompassing reason for the difficulties that we face in the classroom. Yet, I still feel that we need to have it as a consideration. Especially after discovering an updated version of the ‘critical period hypothesis’
“DeKeyser argues that although it is true that there is a critical period, this does not mean that adults cannot learn a second language perfectly, at least on the syntactic level." (wikipedia,takenAug,6,3:30p.m,
http://en.wikipedia.org/wiki/Language_Acquisition_Device).
Also John. B. Carroll talks of ‘Language learning aptitude’, or the “prediction of how well, relative to other individuals, an individual can learn a foreign language in a given amount of time and under given conditions.” (Wikipedia, Language aptitude, http://en.wikipedia.org/wiki/Language_aptitude)
I think that with enough practice, the right motivation, and the right personality, it is quite possible to achieve a high level of competency. Yet, how many of our students can be said to fulfill all these qualifications? As you have said, most of us out there have various responsibilities that take away our time to study, or restrict us in some way.
I just wanted to point to the Critical Hypothesis as a thing to consider in making judgments about our future plans and actions. If it is possible, as Carrol, B., and Pimsleur, P., suggest, we need to gauge in some way, our student’s capacity to learn. Leading us back to the thing I was talking about earlier – Tests. Arghhh! However, there does need to be some thought given to how much are my students ready for what I’m about to present. Depending on the learner’s capacity to learn, we can therefore make judgments on how to proceed.
individual words/signs are quite arbitrary
I found something quite interesting talking about ASL (American Sign Lang.) that would contribute quite well to an argument about how the individual words/signs are quite arbitrary. It talked of how individual ASL signs have meaning within a system of signs, not to individual pictures. So, I guess the sign for Tokyo doesn't relate directly to a picture of Tokyo.
This may be quite obvious, but how many of us realize that the lexical items we are using, be they spoken or signed, are only understood within the context of the system we are using them, understanding of language, through language. If we have more knowledge of the system, we are better able to understand the parts within a system. I guess it's a 'chicken and the egg' kind of realization I'm making here. However, suddenly it has dawned on me that the way I look at words sometimes, is like I expect that the words immediately create certain connections to pictures within student's minds. Yet as in the Fromkin book, students need to start making connection to other words they have already acquired. Beginners often need more input and understanding of the system that they are learning (time to make those connections). After immersion into the language, and getting comfortable (by making those lexical connections between words) they may be ready to understand. I’m probably going off on another tangent here. But, the arbitrariness of the sign language example set me off thinking about how words, along with grammar (signs and facial expressions) are a tools within a system, called language.
This may be quite obvious, but how many of us realize that the lexical items we are using, be they spoken or signed, are only understood within the context of the system we are using them, understanding of language, through language. If we have more knowledge of the system, we are better able to understand the parts within a system. I guess it's a 'chicken and the egg' kind of realization I'm making here. However, suddenly it has dawned on me that the way I look at words sometimes, is like I expect that the words immediately create certain connections to pictures within student's minds. Yet as in the Fromkin book, students need to start making connection to other words they have already acquired. Beginners often need more input and understanding of the system that they are learning (time to make those connections). After immersion into the language, and getting comfortable (by making those lexical connections between words) they may be ready to understand. I’m probably going off on another tangent here. But, the arbitrariness of the sign language example set me off thinking about how words, along with grammar (signs and facial expressions) are a tools within a system, called language.
“Beam me up Scotty.” The role of context
“Beam me up Scotty.”
According to the textbook(Fromkin et al), an inherent part of an expression's meaning, together with the context can determine its referent.
Without knowing the context of the Star Trek world, we cannot know for certain the true meaning of this phrase. Which of the following three points from the textbook does it fit into~
1. Some expressions have sense, but no reference
2. Others have reference, but no sense
3. Deitic terms have sense, but need context to determine reference.
It has reference. Does it make Sense? Well according to the above definition, no, as it doesn’t have the context. We don’t know what it means to ‘Beam someone up (unless you’re a trekkie!).
Given the 3 points above, we can probably say that a phrase can make complete sense, even though the referent is not known, or may be fiction. The truth of a sentence is determined only if the circumstances are known, not just if a sentence has sense.
What does this mean in real terms? Well, it may mean that I can come up with a logical, well formed argument that makes perfect sense, but could be totally false in the real world. Thus, we need to support our arguments with accurate real life (context dependant) information.
Also, a fanatical religious person can make sense in his/her preaching to us in terms of the paradigm they are working within, but when they go beyond that line of thinking, into someone else’s context, they may run into some difficulty.
Therefore, what makes sense to one person may be totally confusing, or completely wrong, to another.
What do you think?
Steve
According to the textbook(Fromkin et al), an inherent part of an expression's meaning, together with the context can determine its referent.
Without knowing the context of the Star Trek world, we cannot know for certain the true meaning of this phrase. Which of the following three points from the textbook does it fit into~
1. Some expressions have sense, but no reference
2. Others have reference, but no sense
3. Deitic terms have sense, but need context to determine reference.
It has reference. Does it make Sense? Well according to the above definition, no, as it doesn’t have the context. We don’t know what it means to ‘Beam someone up (unless you’re a trekkie!).
Given the 3 points above, we can probably say that a phrase can make complete sense, even though the referent is not known, or may be fiction. The truth of a sentence is determined only if the circumstances are known, not just if a sentence has sense.
What does this mean in real terms? Well, it may mean that I can come up with a logical, well formed argument that makes perfect sense, but could be totally false in the real world. Thus, we need to support our arguments with accurate real life (context dependant) information.
Also, a fanatical religious person can make sense in his/her preaching to us in terms of the paradigm they are working within, but when they go beyond that line of thinking, into someone else’s context, they may run into some difficulty.
Therefore, what makes sense to one person may be totally confusing, or completely wrong, to another.
What do you think?
Steve
Paul Grice and his Maxims
Thank you very much for your help. It does give me a direction to head towards.
Doing a search on the net, I have found many interesting pages that not only define both Pragmatics and Semantics, but also talk about the distinctions between them.
Many of the papers tend to point at 'Paul Grice' contributing very much toward Pragmatics, in particular. I haven't explored the material too well yet, but it seems that according to Grice, there is a social element within pragmatics. Along with this are his so called 'maxims'. (Check out Wikipedia for a definition)
With my Semantics research, there seems to be a lot on various relationships between words, such as synonyms, antonyms, and all the rest that are mentioned within our text.
On the net, and in many of the articles I've accumulated, the writers tend to deal with either Pragmatics/Semantics separately or as I said previously, how they differ.
I’m trying to see the relationship between them both and how they influence each other, but I’ve yet to come to any striking conclusions that add to what I have already said. I guess I need some time to let it all soak in a little.
Yet having said this, I do realize that there is quite a striking difference in perspective, when it comes to dealing with this topic. Firstly, that of how individual phrases and words are interpreted individually, and that of how our social situation affects those words or phrases. Also, I never realized how much context and meaning are so interrelated, yet so defined as particular fields (albeit fairly new according to some writers). I’m sorry if I sound a little confused today. I’m in the process of sorting out what I know from what I read.
Doing a search on the net, I have found many interesting pages that not only define both Pragmatics and Semantics, but also talk about the distinctions between them.
Many of the papers tend to point at 'Paul Grice' contributing very much toward Pragmatics, in particular. I haven't explored the material too well yet, but it seems that according to Grice, there is a social element within pragmatics. Along with this are his so called 'maxims'. (Check out Wikipedia for a definition)
With my Semantics research, there seems to be a lot on various relationships between words, such as synonyms, antonyms, and all the rest that are mentioned within our text.
On the net, and in many of the articles I've accumulated, the writers tend to deal with either Pragmatics/Semantics separately or as I said previously, how they differ.
I’m trying to see the relationship between them both and how they influence each other, but I’ve yet to come to any striking conclusions that add to what I have already said. I guess I need some time to let it all soak in a little.
Yet having said this, I do realize that there is quite a striking difference in perspective, when it comes to dealing with this topic. Firstly, that of how individual phrases and words are interpreted individually, and that of how our social situation affects those words or phrases. Also, I never realized how much context and meaning are so interrelated, yet so defined as particular fields (albeit fairly new according to some writers). I’m sorry if I sound a little confused today. I’m in the process of sorting out what I know from what I read.
Critical Period
I understand that the 'critical Period' hypothesis can be a little easy to hold as an all encompassing reason for the difficulties that we face in the classroom. Yet, I still feel that we need to have it as a consideration. Especially after discovering an updated version of the ‘critical period hypothesis’
“DeKeyser argues that although it is true that there is a critical period, this does not mean that adults cannot learn a second language perfectly, at least on the syntactic level." (wikipedia,takenAug,6,3:30p.m,http://en.wikipedia.org/wiki/Language_Acquisition_Device).
Also John. B. Carroll talks of ‘Language learning aptitude’, or the “prediction of how well, relative to other individuals, an individual can learn a foreign language in a given amount of time and under given conditions.” (Wikipedia, Language aptitude, [http://en.wikipedia.org/wiki/Language_aptitude
I think that with enough practice, the right motivation, and the right personality, it is quite possible to achieve a high level of competency. Yet, how many of our students can be said to fulfill all these qualifications? As you have said, most of us out there have various responsibilities that take away our time to study, or restrict us in some way.
I just wanted to point to the Critical Hypothesis as a thing to consider in making judgments about our future plans and actions. If it is possible, as Carrol, B., and Pimsleur, P., suggest, we need to gauge in some way, our student’s capacity to learn. Leading us back to the thing I was talking about earlier – Tests. Arghhh! However, there does need to be some thought given to how much are my students ready for what I’m about to present. Depending on the learner’s capacity to learn, we can therefore make judgments on how to proceed. Steven Mondy
“DeKeyser argues that although it is true that there is a critical period, this does not mean that adults cannot learn a second language perfectly, at least on the syntactic level." (wikipedia,takenAug,6,3:30p.m,http://en.wikipedia.org/wiki/Language_Acquisition_Device).
Also John. B. Carroll talks of ‘Language learning aptitude’, or the “prediction of how well, relative to other individuals, an individual can learn a foreign language in a given amount of time and under given conditions.” (Wikipedia, Language aptitude, [http://en.wikipedia.org/wiki/Language_aptitude
I think that with enough practice, the right motivation, and the right personality, it is quite possible to achieve a high level of competency. Yet, how many of our students can be said to fulfill all these qualifications? As you have said, most of us out there have various responsibilities that take away our time to study, or restrict us in some way.
I just wanted to point to the Critical Hypothesis as a thing to consider in making judgments about our future plans and actions. If it is possible, as Carrol, B., and Pimsleur, P., suggest, we need to gauge in some way, our student’s capacity to learn. Leading us back to the thing I was talking about earlier – Tests. Arghhh! However, there does need to be some thought given to how much are my students ready for what I’m about to present. Depending on the learner’s capacity to learn, we can therefore make judgments on how to proceed. Steven Mondy
Pragmatics and Semantics
I guess it would be useful at this stage to start thinking about the difference and interdependence between ‘Pragmatics and Semantics’. It appears to be quite an involved undertaking, as there are many facets to either. According to the text, we see that Semantics is concerned with the linguistic meaning of: 1.Lexical items – morphemes (and in turn words) and 2. Phrasal Elements – the meaning of phrases.
Pragmatics on the other hand is concerned with how the context affects meaning.
I understand from the readings that even though a sentence can be grammatically correct, well formed syntactically and have a perfectly understandable structure; it may not be true in certain contexts.
I could say, for instance that, “Yesterday was my wife’s birthday.” Semantically you would understand that yesterday there was a birthday, and it was my wife who was celebrating it. This is perfectly fine, but her actual birthday is in June, and not yesterday. In this context, the statement’s ‘Truth Conditions’ are false. However, if I were in a classroom demonstrating the simple past tense and using this declarative statement as an example of a set phrase, even though the truth conditions are still not true, it is still quite a valid statement to make, as the context has now changed. In this instance, we can see the relationship between ‘Pragmatics and Semantics’, if only in a basic way. Semantics needs context to establish the acceptability of a sentence within a certain environment. Grasping this relationship, it is quite possible to apply our understandings to a language learning environment. We can help students, by setting up meaningful contexts for the phrases they learning.
Context plays an extremely important role in avoiding ambiguity. Take for example the newspaper headline, “Teacher strikes idle kids” (Sanderson, P., (1999), Using Newspapers in the Classroom, Cambridge University Press). Anyone reading this headline (without being given the story that goes along with it) may be a little confused, as it can be perceived in two ways. Ambiguity is a great source of confusion, at least with many of my students. This can in turn lead to negative feelings toward the task at hand.
I believe that context also helps a student make use of existing deep lexical networks. Knowing the context of a unit that we are studying can help us anticipate what lexical items will unveil themselves. Just as in the textbook, knowing the word cat, we start thinking of mouse. Well if we know that today’s lesson is about food, we can use our existing knowledge and experiences to visualize food items and create a receptive framework for new words and phrases related to food. A textbook that does this well is the Tactics for Listening series, by Jack C. Richards. It sets up the context in the beginning of a unit, then slowly and progressively introduces phrases and vocabulary.
Pragmatics helps develop a framework for understanding and along with Semantics allows us to make sense of new and unfamiliar words and phrases. What do you think?
Steven Mondy
Pragmatics on the other hand is concerned with how the context affects meaning.
I understand from the readings that even though a sentence can be grammatically correct, well formed syntactically and have a perfectly understandable structure; it may not be true in certain contexts.
I could say, for instance that, “Yesterday was my wife’s birthday.” Semantically you would understand that yesterday there was a birthday, and it was my wife who was celebrating it. This is perfectly fine, but her actual birthday is in June, and not yesterday. In this context, the statement’s ‘Truth Conditions’ are false. However, if I were in a classroom demonstrating the simple past tense and using this declarative statement as an example of a set phrase, even though the truth conditions are still not true, it is still quite a valid statement to make, as the context has now changed. In this instance, we can see the relationship between ‘Pragmatics and Semantics’, if only in a basic way. Semantics needs context to establish the acceptability of a sentence within a certain environment. Grasping this relationship, it is quite possible to apply our understandings to a language learning environment. We can help students, by setting up meaningful contexts for the phrases they learning.
Context plays an extremely important role in avoiding ambiguity. Take for example the newspaper headline, “Teacher strikes idle kids” (Sanderson, P., (1999), Using Newspapers in the Classroom, Cambridge University Press). Anyone reading this headline (without being given the story that goes along with it) may be a little confused, as it can be perceived in two ways. Ambiguity is a great source of confusion, at least with many of my students. This can in turn lead to negative feelings toward the task at hand.
I believe that context also helps a student make use of existing deep lexical networks. Knowing the context of a unit that we are studying can help us anticipate what lexical items will unveil themselves. Just as in the textbook, knowing the word cat, we start thinking of mouse. Well if we know that today’s lesson is about food, we can use our existing knowledge and experiences to visualize food items and create a receptive framework for new words and phrases related to food. A textbook that does this well is the Tactics for Listening series, by Jack C. Richards. It sets up the context in the beginning of a unit, then slowly and progressively introduces phrases and vocabulary.
Pragmatics helps develop a framework for understanding and along with Semantics allows us to make sense of new and unfamiliar words and phrases. What do you think?
Steven Mondy
The capacity and perserverance to learn
I really do believe that we have the capacity to learn any form of communication, as long as it’s within the limits of our physical and mental capabilities. This may include such things as age, motivation and genetic programming. We still don’t have the ability to use mental telepathy (well some of us, anyway?!), but maybe with the evolution of the species, we may develop that capacity. One of my fellow students at USQ makes quite a persuasive point, in that we may not have even developed our existing abilities (to react to pheromones) yet. Who knows…we have yet to discover many things.
The interesting thing for us teachers is that if we have this capacity (at least at some time), how long will it remain with us? As we have come to understand, there is somewhat a time limit to effectively and comprehensively pick up a language (at least for L1). “The critical period.” This is quite a concern for teachers, as it will undoubtedly make it really difficult for adult learners to acquire a second language. I think that these learners are faced with a hidden barrier to language acquisition, which may discourage them from persisting with the process of learning a second language. As another fellow student stated, many learners don't feel it necessary to attend classes more than only a few times a week. Could part of the problem be that the capacity for learning a second language has diminished in our adult learners (depending upon how much they have gone past the ‘critical period’). A language may seem quite exciting in the beginning, but then the novelty starts fading as reality starts kicking in. Maybe that is one of the reasons that pattern recognition systems are so successful with adult learners. They give people a way to balance the scales back in their favor. What do you think? Steven Mondy
The interesting thing for us teachers is that if we have this capacity (at least at some time), how long will it remain with us? As we have come to understand, there is somewhat a time limit to effectively and comprehensively pick up a language (at least for L1). “The critical period.” This is quite a concern for teachers, as it will undoubtedly make it really difficult for adult learners to acquire a second language. I think that these learners are faced with a hidden barrier to language acquisition, which may discourage them from persisting with the process of learning a second language. As another fellow student stated, many learners don't feel it necessary to attend classes more than only a few times a week. Could part of the problem be that the capacity for learning a second language has diminished in our adult learners (depending upon how much they have gone past the ‘critical period’). A language may seem quite exciting in the beginning, but then the novelty starts fading as reality starts kicking in. Maybe that is one of the reasons that pattern recognition systems are so successful with adult learners. They give people a way to balance the scales back in their favor. What do you think? Steven Mondy
Subscribe to:
Posts (Atom)