Acculturation, extends beyond just the learning of a language, but to the social variables that come along with being immersed within the target language and group. The amount that culture plays within the acculturation process is in my mind quite significant, but somewhat understated. My favorite book to date on culture learning, is Damen's 1987, Culture learning: the fifth dimension in the classroom. The whole process of moving from the transient sojourner (short term traveler) through to the permanent resident within a culture (due to immigration, or other circumstances) is clearly set out within that book. The most interesting aspect for me is the idea that it is a process, dealing with social distance: assimilation/adaptation/adjustment. There seems to be a careful balance between culture shock and integration/positive acceptance of cultural protocols. I believe that the process has no end point, and that the individual continues to go through stages of distancing themselves or being totally accepting of various social aspects, throughout their lives. Life is not static, but ever changing, and with those new situations and circumstances there are chances of new kinds of culture shock. I don't know if you can say that you are totally assimilated to a particular culture. However, people can come to a point in which they are comfortable with the roller coaster ride between minimum and maximum social distance. I guess it would depend on how different the new target culture is, and how passionately an individual feels about their own cultural identity. Acculturation therefore (as I see it) is in the realm of the person who moves to a new culture and must deal with the process of fitting into their surrounding environment. Personally, I believe that once an identity is established within one culture, it is very difficult (if not impossible) to totally assimilate. I think that the ideas of adaptation and adjustment allow the individual opportunities to integrate aspects of the new culture into their own identity, or have tolerance for new ways of doing things.
What is to be learned?
Well, if we want individuals to 'fit in' to a particular culture, we need to make it possible for them to reduce the social distance as much as possible, and create opportunities for them to develop tolerance and understanding of different ways of doing things. We should also create environments that encourage (not discourage) the first culture and language, and that try to reduce the stress of culture shock.
Ciao
Steven
Wednesday, April 16, 2008
Wednesday, April 9, 2008
Compound and Coordinate Bilingualism
My question today is how do we use the measures of compound and coordinate bilingualism?
I was struggling with the concept of Coordinate and Compound bilingualism. Basically, I have a good concept about how they differ: Compound being the individual that has one meaning, with two representations for that meaning, and coordinate being the individual who develops two distinct meanings and respective symbols. Kind of like the following representation:
Compound ~ A = a or b
Coordinate ~ A = a and B=b
At first, I thought of it as the following example:
In English, water is the same whether hot or cold,
But in Japanese, Water is ‘Yu’ if it is hot and ‘Mizu’ if it is cold.
Is this an example to illustrate the difference? I am not quite sure?
Then I talked with some colleagues, and we got into a discussion about cultural protocols influencing the bilingual individual, and how the compound/coordinate difference could also be linked to flexibility between cultural systems (a bit of a tangent, I know). However, the coordinate person would be able to switch between two systems, whereas the Compound person may only be able to reference one cultural protocol, which is used in either language, and in turn culture. The person that can use both of the cultural systems, meanings and symbols will have an easier time in fitting in within either system. However, the person who has one meaning for different symbols will be more efficient, but tend to make inappropriate choices, because of subtle nuances in either language caused by differences in cultural behavior.
I am a little confused, and I may be going off base here, but I would like to know how this concept fits. Hamers and Blanc (2000) seem to only skim over this concept. Maybe it’s not that important?
Ciao
Steve
Hamers and Blanc (2000), Bilinguality and Bilingualism, Cambridge.
I was struggling with the concept of Coordinate and Compound bilingualism. Basically, I have a good concept about how they differ: Compound being the individual that has one meaning, with two representations for that meaning, and coordinate being the individual who develops two distinct meanings and respective symbols. Kind of like the following representation:
Compound ~ A = a or b
Coordinate ~ A = a and B=b
At first, I thought of it as the following example:
In English, water is the same whether hot or cold,
But in Japanese, Water is ‘Yu’ if it is hot and ‘Mizu’ if it is cold.
Is this an example to illustrate the difference? I am not quite sure?
Then I talked with some colleagues, and we got into a discussion about cultural protocols influencing the bilingual individual, and how the compound/coordinate difference could also be linked to flexibility between cultural systems (a bit of a tangent, I know). However, the coordinate person would be able to switch between two systems, whereas the Compound person may only be able to reference one cultural protocol, which is used in either language, and in turn culture. The person that can use both of the cultural systems, meanings and symbols will have an easier time in fitting in within either system. However, the person who has one meaning for different symbols will be more efficient, but tend to make inappropriate choices, because of subtle nuances in either language caused by differences in cultural behavior.
I am a little confused, and I may be going off base here, but I would like to know how this concept fits. Hamers and Blanc (2000) seem to only skim over this concept. Maybe it’s not that important?
Ciao
Steve
Hamers and Blanc (2000), Bilinguality and Bilingualism, Cambridge.
Friday, March 14, 2008
How does a child become bilingual?
This is basically a response reading 2.1, part 2.2 How does a child become bilingual?
The initial answer that was proposed in the reading was by growing up in a bilingual environment.
Being a parent and living abroad, I initially thought that creating a very supportive home environment in one language, can coexist with a different outside environment. When my daughter was born, I imagined creating such a supportive English language environment at home (yet outside the home everything is Japanese), follow the one-person one language maxim, and instill within my daughter a love for both languages, so much so that she would be proud to speak both languages in all situations. Alas, my hopes and expectations were somewhat too idealistic and definitely did not account for all possible affecting variables (psychological and social). Don't get me wrong, I tried my hardest to make those dreams a reality, but if I knew then, what I know now...alas regret, something I try to avoid...
According to the reading ~ exposure, consistency, perceived need, and social support all factor into the mix, however, I feel that the social aspect has a considerable impact.
1. Providing a chance for exposure to take place is sometimes difficult to create. When my daughter was young, it seemed easier, with videos and books, but as she grew older, and especially at the time she began school (regular school), it was harder and harder to create opportunities for her to use and be surrounded by English.
2. Consistency (in staying in English mode, or responding in the same way each time) is difficult to maintain, especially when I am also trying to learn the language of the society I am living in. I find it hard to only respond in English for at least two reasons. One is that I want to use the language I'm learning, and the other is that I want her to be able to really understand what I am talking about. Her Japanese seemed to suddenly take off at around age 5 or 6 and English kinda lagged behind... Sometimes it was easier and faster to speak in Japanese. I also often find myself talking in mixed English and Japanese, using the words that I feel more comfortable with, and that would get the job done.
3. When my daughter started school, her perceived need to speak English suddenly dropped, because she wasn't encountering English that often (I was the only person who spoke to her in English). It wasn't that she didn't understand English, it's just that she had few opportunities to speak it... and slowly but surely, she started reverting back to Japanese, even when speaking with me... My choice was either to encourage her to speak only English, or let her speak Japanese and speak to her in English. In the beginning, I chose the first one, but it didn't work...I eventually moved on to the second option...But now there is nothing motivating her to speak English...
4.It seems sometimes, that all her outside influences are just too strong... That and my wife only speaking Japanese to her...
I am happy with the English she does know. I feel that when she was young, I exposed her to enough English, for a fairly long time, in that so called critical period, that English will stay with her, in some way. She may always have that feeling for the language. My worry now is with her language attrition (loss of English), but I believe that I can support her now and in the future. The kind of support varies as time passes, and she goes through different stages. But I discovered, it's undesirable if you push too much and it's also undesirable if you don't expose your child at all. It's really a fine line...
Somewhat personal today
ciao
Steve
The initial answer that was proposed in the reading was by growing up in a bilingual environment.
Being a parent and living abroad, I initially thought that creating a very supportive home environment in one language, can coexist with a different outside environment. When my daughter was born, I imagined creating such a supportive English language environment at home (yet outside the home everything is Japanese), follow the one-person one language maxim, and instill within my daughter a love for both languages, so much so that she would be proud to speak both languages in all situations. Alas, my hopes and expectations were somewhat too idealistic and definitely did not account for all possible affecting variables (psychological and social). Don't get me wrong, I tried my hardest to make those dreams a reality, but if I knew then, what I know now...alas regret, something I try to avoid...
According to the reading ~ exposure, consistency, perceived need, and social support all factor into the mix, however, I feel that the social aspect has a considerable impact.
1. Providing a chance for exposure to take place is sometimes difficult to create. When my daughter was young, it seemed easier, with videos and books, but as she grew older, and especially at the time she began school (regular school), it was harder and harder to create opportunities for her to use and be surrounded by English.
2. Consistency (in staying in English mode, or responding in the same way each time) is difficult to maintain, especially when I am also trying to learn the language of the society I am living in. I find it hard to only respond in English for at least two reasons. One is that I want to use the language I'm learning, and the other is that I want her to be able to really understand what I am talking about. Her Japanese seemed to suddenly take off at around age 5 or 6 and English kinda lagged behind... Sometimes it was easier and faster to speak in Japanese. I also often find myself talking in mixed English and Japanese, using the words that I feel more comfortable with, and that would get the job done.
3. When my daughter started school, her perceived need to speak English suddenly dropped, because she wasn't encountering English that often (I was the only person who spoke to her in English). It wasn't that she didn't understand English, it's just that she had few opportunities to speak it... and slowly but surely, she started reverting back to Japanese, even when speaking with me... My choice was either to encourage her to speak only English, or let her speak Japanese and speak to her in English. In the beginning, I chose the first one, but it didn't work...I eventually moved on to the second option...But now there is nothing motivating her to speak English...
4.It seems sometimes, that all her outside influences are just too strong... That and my wife only speaking Japanese to her...
I am happy with the English she does know. I feel that when she was young, I exposed her to enough English, for a fairly long time, in that so called critical period, that English will stay with her, in some way. She may always have that feeling for the language. My worry now is with her language attrition (loss of English), but I believe that I can support her now and in the future. The kind of support varies as time passes, and she goes through different stages. But I discovered, it's undesirable if you push too much and it's also undesirable if you don't expose your child at all. It's really a fine line...
Somewhat personal today
ciao
Steve
Tuesday, March 11, 2008
Measures of Bilinguality
I think that I agree with Hamers and Blanc's idea that we should try to maintain a multidimensional view of bilingualism and bilinguality, even when trying to establish criteria to determine competencies that lead to someone being considered bilingual or not. Too often linguistic competence alone is chosen (by the lay-person) as the only criteria. Psychological states and socio-linguistic aspects should also be considered to gain a more holistic perspective.
I know that my own language performance in my L2 is quite often affected by my mood, or the environment I find myself in, or the task I am to perform. I believe that each bilingual person has their own set of behaviors, specific to their situation, culture and languages. My question is that in a measure of bilinguality, how do you account for the 7 measures proposed by Hamers and Blanc in a balanced way, maintaining validity (are we testing what we really want to test), and consistency/reliability over populations of individuals who all have varying and unique needs? In simple terms, how do you make a test that fits each and every bilingual, when cultures and languages and individuals have so many differences?
What does everyone think?
ciao
steve
I know that my own language performance in my L2 is quite often affected by my mood, or the environment I find myself in, or the task I am to perform. I believe that each bilingual person has their own set of behaviors, specific to their situation, culture and languages. My question is that in a measure of bilinguality, how do you account for the 7 measures proposed by Hamers and Blanc in a balanced way, maintaining validity (are we testing what we really want to test), and consistency/reliability over populations of individuals who all have varying and unique needs? In simple terms, how do you make a test that fits each and every bilingual, when cultures and languages and individuals have so many differences?
What does everyone think?
ciao
steve
Friday, March 7, 2008
What is a bilingual 2?
Hi there everyone...
I was doing some research for my homepage, when I came across an interesting slideshow It is made as a listening lesson, but it brings up some interesting comparisons btw elective and circumstantial bilingualism, especially when she talks about her learning English in an English speaking country.
Check it out...
ciao
Steve
I was doing some research for my homepage, when I came across an interesting slideshow It is made as a listening lesson, but it brings up some interesting comparisons btw elective and circumstantial bilingualism, especially when she talks about her learning English in an English speaking country.
Check it out...
ciao
Steve
How you would choose to define your language abilities?
How you would choose to define your language abilities, Steve?
Well, it's hard to say... I consider myself bilingual in some domains, but hopelessly out of my league in other domains. Sometimes I feel that I can speak Japanese really well, especially at an Izakaya (kinda restaurant/pub in Japan) when the language is very casual, but in school meetings (when conducted in Japanese) I struggle for every word.
At the moment, I tend to disagree with the labeling that is done (refer to:Incipient/balanced etc), as it tends to pigeon hole people under headings that may not adequately describe them. But without the jargon, it is very difficult to discuss concepts, 'cause we would need to rely on a lot of pretty heavy description...something I noticed while doing the research subject last semester...
My friend and I (he is Japanese -but fluent in spoken English) were discussing my language ability the other day. He said to me that I don't really know Japanese, 'cause he rarely hears me speak it. I said to him that it is quite difficult to speak to him in Japanese, 'cause I know that he understands English really well. Then I told him to finish the conversation in Japanese...which he did and I could understand his every word, but I was still unable to respond to him in Japanese. I had some kind of invisible wall, that made me clam up in Japanese, and revert back to English replies.
There are many times that people praise me, and a lot of other times when they say my Japanese sucks. So it leads me to wonder how someone can really evaluate another person being bilingual.
I think anyone who has had a reasonable amount of exposure to an L2 (or L3, L4) and communicate or understand communication in that language (through at least one of the macro skills) is bilingual to some extent.
How you would choose to define your language abilities?
ciao
steve
ps. I don't know if I adequately responded to this question, because it seems like such a big one, at least for me, at this time...
Well, it's hard to say... I consider myself bilingual in some domains, but hopelessly out of my league in other domains. Sometimes I feel that I can speak Japanese really well, especially at an Izakaya (kinda restaurant/pub in Japan) when the language is very casual, but in school meetings (when conducted in Japanese) I struggle for every word.
At the moment, I tend to disagree with the labeling that is done (refer to:Incipient/balanced etc), as it tends to pigeon hole people under headings that may not adequately describe them. But without the jargon, it is very difficult to discuss concepts, 'cause we would need to rely on a lot of pretty heavy description...something I noticed while doing the research subject last semester...
My friend and I (he is Japanese -but fluent in spoken English) were discussing my language ability the other day. He said to me that I don't really know Japanese, 'cause he rarely hears me speak it. I said to him that it is quite difficult to speak to him in Japanese, 'cause I know that he understands English really well. Then I told him to finish the conversation in Japanese...which he did and I could understand his every word, but I was still unable to respond to him in Japanese. I had some kind of invisible wall, that made me clam up in Japanese, and revert back to English replies.
There are many times that people praise me, and a lot of other times when they say my Japanese sucks. So it leads me to wonder how someone can really evaluate another person being bilingual.
I think anyone who has had a reasonable amount of exposure to an L2 (or L3, L4) and communicate or understand communication in that language (through at least one of the macro skills) is bilingual to some extent.
How you would choose to define your language abilities?
ciao
steve
ps. I don't know if I adequately responded to this question, because it seems like such a big one, at least for me, at this time...
Wednesday, March 5, 2008
Incipient Vs. Balanced Bilingual
I was just wondering, and it could be quite a stupid question, but is there anything in between Incipient and Balanced bilingual? I realize that these are just labels used by academics to define the boundaries of bilingualism, but I feel I can't say that I am at either the minimal end of incipient bilingualism or the maximum end of being a balanced bilingual.
On the questionnaire attached to the USQ DB, I voted that I was an Incipient bilingual, but I feel there should be further divisions that include people with strong abilities in say, just one receptive skill such as speaking or listening, but weaker with productive skills. I guess this is where the Holistic view of bilingualism plays a part?!
a bit confused...
Steve
On the questionnaire attached to the USQ DB, I voted that I was an Incipient bilingual, but I feel there should be further divisions that include people with strong abilities in say, just one receptive skill such as speaking or listening, but weaker with productive skills. I guess this is where the Holistic view of bilingualism plays a part?!
a bit confused...
Steve
How does context affect bilingualism?
Hamers and Blanc (2000) make the process of defining bilingualism a little more challenging within that first chapter, as now we should also be considering not only cognitive processes, but also social interactions, and socio-psychological aspects as well. That is not only the ability to understand and perform within a second language (L2), but how the context of our situation makes us feel toward a language, and how much we are encouraged to valorise (attach positive value toward) the language systems around us. In a monolingual environment, that may be nil. In a bilingual or multilingual environment it would depend on the prestige attached to the respective languages.
That brings me to question my environment here in Japan. It is essentially a monolingual environment (Japanese), but English seems to be promoted as 'the' language to learn to get ahead. Yet, it seems like it is more said than done, as English levels remain low (according to standardized tests such as TOEIC, etc), at least compared with other Asian countries. Does English have any prestige, or enough to make people value it and internalize it? Are the forces (pretty much everything is Japanese first) at play in this essentially monolingual environment too strong? I know that every time my students (college kids) leave their classes, the pressure to speak Japanese can be seen, because I see them quickly change to their first language (L1). The only exception I see, is with our International High school kids. They seem to continue speaking English in the halls, almost in a kind of defiance or to show themselves capable... Could age and development, as two of the dimensions of bilingualism be strong players in how confident students are in going against societal norms?
I don't know really, but I think that as students get older (here), they start adapting to behavior of the group, and often the nail that sticks up, is hammered down... Kind of a negative view, but Japan tends to favor group behavior rather than showing individualism. And speaking English (well) here is still considered kind of special.
Ciao
Steven thoughtful
That brings me to question my environment here in Japan. It is essentially a monolingual environment (Japanese), but English seems to be promoted as 'the' language to learn to get ahead. Yet, it seems like it is more said than done, as English levels remain low (according to standardized tests such as TOEIC, etc), at least compared with other Asian countries. Does English have any prestige, or enough to make people value it and internalize it? Are the forces (pretty much everything is Japanese first) at play in this essentially monolingual environment too strong? I know that every time my students (college kids) leave their classes, the pressure to speak Japanese can be seen, because I see them quickly change to their first language (L1). The only exception I see, is with our International High school kids. They seem to continue speaking English in the halls, almost in a kind of defiance or to show themselves capable... Could age and development, as two of the dimensions of bilingualism be strong players in how confident students are in going against societal norms?
I don't know really, but I think that as students get older (here), they start adapting to behavior of the group, and often the nail that sticks up, is hammered down... Kind of a negative view, but Japan tends to favor group behavior rather than showing individualism. And speaking English (well) here is still considered kind of special.
Ciao
Steven thoughtful
Monday, March 3, 2008
What is a bilingual?
Hi there everyone,
I have been lazily getting into this stuff on bilingualism, but find it all very fascinating...
I was quite interested in the great variety of perspectives on what we refer to as a bilingual. I have to admit, that before starting my masters program, I used to think in the lines of the popular view as posited by people such as Bloomfied (1935, as cited in Hamers and Blanc 2000): a bilingual being someone who can speak both languages perfectly. Yet, now I am favoring Grosjean (1985, as cited in Hamers and Blanc 2000) or even Macnamara (1994); a bilingual is more than the sum of the two monolinguals and who posses a minimal competence in one of the skills. This seems to be a richer and deeper definition of the complexity of being bilingual. The Japanese expression for someone who is from two cultures is 'half'. I often joke with people here by saying ~not half...'Double'. 'cause I believe (now this may be controversial, and I hope that I don't offend) that true bilinguals are actually smarter...
I have always been struggling with my identity. Throughout my life I have had a deep feeling of being at least partly Maltese, even though I don't have any physical features that show that I am Maltese. Living in Melbourne, Australia during my formative years must have shaped part of that identity. Even though my language proficiency (...having some understanding of the macro skill of listening, but unfortunately not being able to speak so well) did not match my strong feeling of being Maltese, I was able to feel a strong connection to to culture, and to its language. Every time I heard Maltese, my ears would prick up and I would feel warmth toward it, and some disappointment in not being able to be proficient. My identity must have been heavily influenced by my significant other (mother) when I was a child, to develop a bi-cultural identity, but the context of my social networks and community must have been quite powerful, in that the result is a person who cannot effectively communicate in Maltese (at least in a way that I wish to). It's something that my daughter will have to deal with, too. However, I suspect that she will deal with it in different ways, because of her social context....
ciao
Steve
I have been lazily getting into this stuff on bilingualism, but find it all very fascinating...
I was quite interested in the great variety of perspectives on what we refer to as a bilingual. I have to admit, that before starting my masters program, I used to think in the lines of the popular view as posited by people such as Bloomfied (1935, as cited in Hamers and Blanc 2000): a bilingual being someone who can speak both languages perfectly. Yet, now I am favoring Grosjean (1985, as cited in Hamers and Blanc 2000) or even Macnamara (1994); a bilingual is more than the sum of the two monolinguals and who posses a minimal competence in one of the skills. This seems to be a richer and deeper definition of the complexity of being bilingual. The Japanese expression for someone who is from two cultures is 'half'. I often joke with people here by saying ~not half...'Double'. 'cause I believe (now this may be controversial, and I hope that I don't offend) that true bilinguals are actually smarter...
I have always been struggling with my identity. Throughout my life I have had a deep feeling of being at least partly Maltese, even though I don't have any physical features that show that I am Maltese. Living in Melbourne, Australia during my formative years must have shaped part of that identity. Even though my language proficiency (...having some understanding of the macro skill of listening, but unfortunately not being able to speak so well) did not match my strong feeling of being Maltese, I was able to feel a strong connection to to culture, and to its language. Every time I heard Maltese, my ears would prick up and I would feel warmth toward it, and some disappointment in not being able to be proficient. My identity must have been heavily influenced by my significant other (mother) when I was a child, to develop a bi-cultural identity, but the context of my social networks and community must have been quite powerful, in that the result is a person who cannot effectively communicate in Maltese (at least in a way that I wish to). It's something that my daughter will have to deal with, too. However, I suspect that she will deal with it in different ways, because of her social context....
ciao
Steve
Saturday, January 12, 2008
Research Proposal 3
I need to also decide on a paradigm (most likely qualitative or quasi-experimental)---so that means I need to start thinking of questions (rather than hypotheses), and different variables to compare or contrast... This is where attitudes (teacher and student), learning styles, teacher checks and balances, logistics and overall suitability of teaching methodology may come into play...
It'll be most likely action research (but could also make use of case study or grounding theory), so the understandings will come through the research, rather than test or validate hypotheses... I will probably need to propose small incremental changes, that can be implemented within the real system and evaluated on a regular basis...as consistent with action research (Kremmis & McTaggart 1988)
The methods and design I am still playing with. Definitely as an insider, I can offer a unique first hand perspective (but which can also cause bias, which I will need to account for in the design, methods and procedure section for sources of error), but there will be a need for interviews and/or surveys of teacher and student understandings of this kind of teaching and its perceived usefulness. Comparisons could be made within observations which could be undertaken of my own classes and other teachers classes (given that ethical considerations and rater bias are taken into account).
I think it will be difficult to delimit the actual question (but I will need to if I want to keep under the word limit), as there is a lot that can and should be investigated.
Oh well, that's where I am at...
ciao
Steven M
It'll be most likely action research (but could also make use of case study or grounding theory), so the understandings will come through the research, rather than test or validate hypotheses... I will probably need to propose small incremental changes, that can be implemented within the real system and evaluated on a regular basis...as consistent with action research (Kremmis & McTaggart 1988)
The methods and design I am still playing with. Definitely as an insider, I can offer a unique first hand perspective (but which can also cause bias, which I will need to account for in the design, methods and procedure section for sources of error), but there will be a need for interviews and/or surveys of teacher and student understandings of this kind of teaching and its perceived usefulness. Comparisons could be made within observations which could be undertaken of my own classes and other teachers classes (given that ethical considerations and rater bias are taken into account).
I think it will be difficult to delimit the actual question (but I will need to if I want to keep under the word limit), as there is a lot that can and should be investigated.
Oh well, that's where I am at...
ciao
Steven M
Research Proposal 2
I had a problem when trying to decide on a definition for this kind of teacher sharing of a class (which appears to be unique to this particular situation). I think that my lecturers comments in a discussion board post of giving our situation the labels of paired teaching and teaching pairs seem to encapsulate the idea more clearly...
I still want to go ahead with this topic, as
1. It is in operation, and has been for quite a while in my college,
2. It is a system that many of my Japanese administrators favor,
3. It is a system that has morphed from an original team teacher situation in the traditional sense (the reason it has changed I am still not sure of yet),
4. I want to know if there are any advantages of doing it this way, in comparison to the regular one teacher teaching both classes, and
5. I'd like to know how student's feel toward the paired teaching situation,
that's what I've come up with for starters...
I am not sure if the two systems are that different, in that they are offshoots of the same initial rationale, or are modified forms of the same thing. I think, however, that limiting myself to one would be advantageous to eventually getting clearer picture...
As for my proposal I haven't really thought too much into the future. I initially took this course just to get some grounding in research, in case in the future I may want to continue study at a higher level... At the moment I am just concentrating on finishing my masters and have at least two more subjects to do before even thinking about a true research project. At the moment I just want to know how to write a proposal...
This course has been the best so far (it is my 5th), and it has also been the hardest. I am really pushing myself, but getting a lot back...
Ciao
Steven M
I still want to go ahead with this topic, as
1. It is in operation, and has been for quite a while in my college,
2. It is a system that many of my Japanese administrators favor,
3. It is a system that has morphed from an original team teacher situation in the traditional sense (the reason it has changed I am still not sure of yet),
4. I want to know if there are any advantages of doing it this way, in comparison to the regular one teacher teaching both classes, and
5. I'd like to know how student's feel toward the paired teaching situation,
that's what I've come up with for starters...
I am not sure if the two systems are that different, in that they are offshoots of the same initial rationale, or are modified forms of the same thing. I think, however, that limiting myself to one would be advantageous to eventually getting clearer picture...
As for my proposal I haven't really thought too much into the future. I initially took this course just to get some grounding in research, in case in the future I may want to continue study at a higher level... At the moment I am just concentrating on finishing my masters and have at least two more subjects to do before even thinking about a true research project. At the moment I just want to know how to write a proposal...
This course has been the best so far (it is my 5th), and it has also been the hardest. I am really pushing myself, but getting a lot back...
Ciao
Steven M
Research Proposal 1
I have a million and one ideas for a research proposal, and I can't seem to decide on one...
Having said that, I am probably going to look into some action research that would be of direct benefit to the school program I am teaching in at the moment...
I have identified some discontent with the team teaching program that is running within the college I am working.
Team teaching is not unique in itself, as it can be found in many forms, and in Japan it has taken root within a number of niches (AET - assistant English teacher, in the JET program for instance).
However, I wish to look at the effectiveness of team teaching in an adult SLL environment, specifically focusing on the overall effectiveness of native-native English teacher, team teaching situation (which exists in my program) vs. native-Japanese English teacher, (which exists in a parallel program within the same organization). The teachers in both these programs have varying beliefs about the suitability of the native-native team teaching situation.
My limited review of the background thus far has been unsuccessful (can't seem to find anything on the native-native adv/disadv). The situation of two native English teachers (two Americans for instance), teaching on different days, but with the same textbook, doesn't seem to have any sound methodological purpose, yet the administration staff (mainly Japanese) seem to really favor it, as well as some ex-students, who say that it was pretty good.
My initial thoughts were that it was beneficial in terms of learning styles and exposure to varying cultural perspectives... This could be the angle that I could take.
However, for the team teachers it can be a logistic nightmare...which makes it difficult to plan lessons and evaluate student performance effectively...
I haven't quite sussed out the details, but this is the direction I am thinking about taking... I am part of the program, so I am an insider, which makes subjectivity a real problem, as with many kinds of action research, not to mention ethnography etc... Yet, I think that this subjectivity can be balanced with grounding of the findings in some way...
I hope that I am on the right track... Actually, my head is in quite a spin at the moment...
Having said that, I am probably going to look into some action research that would be of direct benefit to the school program I am teaching in at the moment...
I have identified some discontent with the team teaching program that is running within the college I am working.
Team teaching is not unique in itself, as it can be found in many forms, and in Japan it has taken root within a number of niches (AET - assistant English teacher, in the JET program for instance).
However, I wish to look at the effectiveness of team teaching in an adult SLL environment, specifically focusing on the overall effectiveness of native-native English teacher, team teaching situation (which exists in my program) vs. native-Japanese English teacher, (which exists in a parallel program within the same organization). The teachers in both these programs have varying beliefs about the suitability of the native-native team teaching situation.
My limited review of the background thus far has been unsuccessful (can't seem to find anything on the native-native adv/disadv). The situation of two native English teachers (two Americans for instance), teaching on different days, but with the same textbook, doesn't seem to have any sound methodological purpose, yet the administration staff (mainly Japanese) seem to really favor it, as well as some ex-students, who say that it was pretty good.
My initial thoughts were that it was beneficial in terms of learning styles and exposure to varying cultural perspectives... This could be the angle that I could take.
However, for the team teachers it can be a logistic nightmare...which makes it difficult to plan lessons and evaluate student performance effectively...
I haven't quite sussed out the details, but this is the direction I am thinking about taking... I am part of the program, so I am an insider, which makes subjectivity a real problem, as with many kinds of action research, not to mention ethnography etc... Yet, I think that this subjectivity can be balanced with grounding of the findings in some way...
I hope that I am on the right track... Actually, my head is in quite a spin at the moment...
Friday, December 7, 2007
More about Fereyabend...
I don't really prescribe to Popper and his specific rejection of empiricism... I am leaning toward a hermeneutic approach of both holistic and integrative in nature. Rationalism has it's place, but we can certainly rationalize anything, with convincing deductive and inductive arguments. This is it's downfall...
I think that my brother's ice-cream example (great example by the way)
" epistemologies have different flavours and thus - in the same way as strawberry and chocolate are both nice flavours of coloured milk but both are still valid nice flavours of milk - equal right to be called truthful nice flavours of milk if they are based on some internally self consistent socially grounded method, including personal choice." (Mondy, P., 2007)
fits in with what I was reading about objectivity in a critical community, which means that you can have two or more paradigms standing side by side, and both being equally objective and reasonable. This is where one should look for a touchstone...a commonality between a number of paradigms and triangulate certain premises that work in a number of different perspectives. This is where Fereyabend is genuinely great, in that he sees beyond just trying to maintain a single view... But don't you find that if you are too loose in your views, there would be a flip flop from belief in a theory, to rejection then back again...? This can be quite detrimental within the area of high-stakes research...with doctors they may prescribe...but later recant on their previous philosophies...This is socially irresponsible, too.
Hermeneutics has a pyramid structure of Theory (being as objective as one can be)----Philosophy (recognizing reader's perceptions, prejudices and foreknowledge)----and social dynamics (understanding of how context can distort ideas). For educational purposes, this seems pretty sound...
I think that my brother's ice-cream example (great example by the way)
" epistemologies have different flavours and thus - in the same way as strawberry and chocolate are both nice flavours of coloured milk but both are still valid nice flavours of milk - equal right to be called truthful nice flavours of milk if they are based on some internally self consistent socially grounded method, including personal choice." (Mondy, P., 2007)
fits in with what I was reading about objectivity in a critical community, which means that you can have two or more paradigms standing side by side, and both being equally objective and reasonable. This is where one should look for a touchstone...a commonality between a number of paradigms and triangulate certain premises that work in a number of different perspectives. This is where Fereyabend is genuinely great, in that he sees beyond just trying to maintain a single view... But don't you find that if you are too loose in your views, there would be a flip flop from belief in a theory, to rejection then back again...? This can be quite detrimental within the area of high-stakes research...with doctors they may prescribe...but later recant on their previous philosophies...This is socially irresponsible, too.
Hermeneutics has a pyramid structure of Theory (being as objective as one can be)----Philosophy (recognizing reader's perceptions, prejudices and foreknowledge)----and social dynamics (understanding of how context can distort ideas). For educational purposes, this seems pretty sound...
Feyerabend: The case for Methodological Pluralism
I have to say that Feyerabend has a point, when it comes to saying that science has inherent biases, and
that the only way to have an epistemological viewpoint that is free from prejudice, is by having a perspective that is flexible and changeable. But, Berstein (1983) quotes Gadamer by saying there is no knowledge without prejudice, anyway...
I think that Feyerabend's Dadaistic (anything goes approach), and anarchist view of scientific method is a little extreme, especially when it comes to replacing let's say astronomy (a hard science, backed by empirical experience) with astrology (a more touchy feely, yet rational paradigm). The article I was reading, proposed a kind of openness to accepting new ideas, that science should take advantage of other theories and paradigms; such as medicine accepting the help of herbalists and other alternative medicine.
Feyerabend just has far too liberal an interpretation of truth, that he accepts just about any idea (Magic and mysticism). Scientific method may be inconclusive, but it has given us various understandings that are open to verification and falsifiability, but even more importantly they can be practically applied, with reasonable success. Shouldn't we be looking for all those successful applications of theories, and reject those ideas that don't help better our understanding of society. We don't have to hold onto one paradigm, at the expense of beneficial aspects of another (Tibbetts, 1977)
I think that Feyerabend has made it possible to have a more critical view of the nature of scientific method, and probably enabled us to be more open to all potential sources of information...so that we can have a better quality of life, Holistically...
Steven Mondy
Tibbetts, P. (1977) Feyerabend's Against Method: The case for Methodological Pluralism, Sage publications, retrieved Nov, 27, 2007 at http://pos.sagepub.com
that the only way to have an epistemological viewpoint that is free from prejudice, is by having a perspective that is flexible and changeable. But, Berstein (1983) quotes Gadamer by saying there is no knowledge without prejudice, anyway...
I think that Feyerabend's Dadaistic (anything goes approach), and anarchist view of scientific method is a little extreme, especially when it comes to replacing let's say astronomy (a hard science, backed by empirical experience) with astrology (a more touchy feely, yet rational paradigm). The article I was reading, proposed a kind of openness to accepting new ideas, that science should take advantage of other theories and paradigms; such as medicine accepting the help of herbalists and other alternative medicine.
Feyerabend just has far too liberal an interpretation of truth, that he accepts just about any idea (Magic and mysticism). Scientific method may be inconclusive, but it has given us various understandings that are open to verification and falsifiability, but even more importantly they can be practically applied, with reasonable success. Shouldn't we be looking for all those successful applications of theories, and reject those ideas that don't help better our understanding of society. We don't have to hold onto one paradigm, at the expense of beneficial aspects of another (Tibbetts, 1977)
I think that Feyerabend has made it possible to have a more critical view of the nature of scientific method, and probably enabled us to be more open to all potential sources of information...so that we can have a better quality of life, Holistically...
Steven Mondy
Tibbetts, P. (1977) Feyerabend's Against Method: The case for Methodological Pluralism, Sage publications, retrieved Nov, 27, 2007 at http://pos.sagepub.com
Sunday, December 2, 2007
Week 2 of Research Methods in Education
Well, having come to the end of yet another week, I have found that things are starting to make sense... The first week was hard in that there was a lot of terminology... but this week has been hard for me, trying to come to terms with many of the concepts and issues in research...I came to a point in which I needed a time line of these 'thinkers' in the history and philosophy of science.... It is amazing how much of a visual person I am...so I looked up any sites that would give me a good look at all of them in relation to one another…
One site (very simple, I know): http://timelineindex.com/content/view/2031
1. Some of the people I have befriended this week...
Plato...Aristotle...Kant...Hume...Popper…. hahahaha
Actually, it was really good to get some perspective on these guys…
2. Also, I have started to get an idea about what goes into making up good research, but I am still at the very beginning... The readings in module 2 helped, but there were just so many questions (especially in the last reading: Gay, 1992)...and I will need some time to digest the information and sort out what I think...
3. One idea that came out while I was reading the module 2 readings was the idea of allowing time to synthesize and formulate other perspectives and reflect on data (Goetz & Lecompte, 1984). I've often thought to myself, that given enough time between tasks, I am bound to come up with major improvements to whatever I am working on. (That reading kinda consolidated it in my own mind ~ each subsequent revision can slowly give more and more clarity). I try to encourage my writing class students to do exactly that – leave time to reflect. They are often (like many of us) in too much of a hurry to finish things, that they rush through something and not give themselves much time to just sit back and mull things over...
4. I also like the idea that a researcher should say to themselves, “So what”? I think that whenever I start working on something in my teaching job (say some internet based activity), I often get so excited in the task at hand (more so than the students sometimes), that I end up forgetting to answer this fundamental question. What purpose is there in what I am doing, and when much of myself is invested in the project, how much do I wish to be critical (or skeptical) of what I am doing? Am I willing to ask myself, if there are any other ways or even more scope to the approach I am taking? As a researcher, can I afford to be so personally involved in the project? How hard is it to take a step back, and distance myself from the undertaking, so that I can be more responsive to data (or text)? Am I able to modify and possibly change my initial work, if the applications of my initial generalizations yield strange results? I hope that I can be flexible enough to do so…
I hope that we all can…
Steven Mondy
Goetz & Lecompte, (1984) Ethnography and qualitative design in education research, Academic press, Orlando.
One site (very simple, I know): http://timelineindex.com/content/view/2031
1. Some of the people I have befriended this week...
Plato...Aristotle...Kant...Hume...Popper…. hahahaha
Actually, it was really good to get some perspective on these guys…
2. Also, I have started to get an idea about what goes into making up good research, but I am still at the very beginning... The readings in module 2 helped, but there were just so many questions (especially in the last reading: Gay, 1992)...and I will need some time to digest the information and sort out what I think...
3. One idea that came out while I was reading the module 2 readings was the idea of allowing time to synthesize and formulate other perspectives and reflect on data (Goetz & Lecompte, 1984). I've often thought to myself, that given enough time between tasks, I am bound to come up with major improvements to whatever I am working on. (That reading kinda consolidated it in my own mind ~ each subsequent revision can slowly give more and more clarity). I try to encourage my writing class students to do exactly that – leave time to reflect. They are often (like many of us) in too much of a hurry to finish things, that they rush through something and not give themselves much time to just sit back and mull things over...
4. I also like the idea that a researcher should say to themselves, “So what”? I think that whenever I start working on something in my teaching job (say some internet based activity), I often get so excited in the task at hand (more so than the students sometimes), that I end up forgetting to answer this fundamental question. What purpose is there in what I am doing, and when much of myself is invested in the project, how much do I wish to be critical (or skeptical) of what I am doing? Am I willing to ask myself, if there are any other ways or even more scope to the approach I am taking? As a researcher, can I afford to be so personally involved in the project? How hard is it to take a step back, and distance myself from the undertaking, so that I can be more responsive to data (or text)? Am I able to modify and possibly change my initial work, if the applications of my initial generalizations yield strange results? I hope that I can be flexible enough to do so…
I hope that we all can…
Steven Mondy
Goetz & Lecompte, (1984) Ethnography and qualitative design in education research, Academic press, Orlando.
Saturday, November 24, 2007
Falsifiability - Can we use this concept in qualitative research?
As for falsifiability, I think that the underlying assumption of this concept may be at odds with the underlying concepts within qualitative studies. I did a lot of reading on the distinction between quantitative and qualitative theory today and some postulating about the assumptions that both take. In the end, I see a clear distinction being drawn between both, in that quantitative studies relate to explaining or controlling variables; these variances can be double checked in future similar studies, by re-creating similar if not exact conditions within future experiments. Qualitative studies on the other hand try to look at things more holistically. I think that in qualitative studies, it is harder to say something is either right or wrong, as the phenomena under investigation is viewed more loosely and with greater flexibility. In addition, the kinds of things that qualitative studies focus on are reliant on context. That is they are specific to the particular classroom, or based on a specific problem of a smaller group. The theories that may come out (within say, a Funnel approach) are related to smaller and more specific groups, within a certain time frame. That said, it is difficult to come up with conditions that reject the findings of this kind of qualitative study, as it would be more difficult to correlate the findings (not having numerical data) with other similar groups in the future, without validity issues.
From what I gather, falsifiability is about understanding that there may be a case that your theory may be disproved, and in such a case you need to modify your thinking or may even have to reject it. Unless we take a time machine (or utilize historical organization in a qualitative study), we are unable to create the exact conditions necessary to disprove the findings. So, what can we do?
We can look into the validity of the qualitative research by checking the organization of the arguments presented and the usefulness of the descriptive analysis. If the study is logical and presents exceptional descriptions, then how can it be falsified?
One may say that the theory produced for that particular group may not suit present groups, ‘cause the conditions or context is different. Is it now falsified?????
What is the purpose to a qualitative study? How much can we depend on the universals that come out of such studies to make inferences about other groups? Do we use them as only a guide????
Maybe I am way off track, but I thought that the falsifiability idea better suits hard numerical data, which can more easily be disproved…Plus I think more important issues with qualitative studies are with how well the generalizations fit with the context of the study, and what happens soon after with that particular group…I don’t know, I think I m starting to confuse myself….
Steven M.
Wiersma & Jurs (2005) Research Methods in Education, Pearson, USA
From what I gather, falsifiability is about understanding that there may be a case that your theory may be disproved, and in such a case you need to modify your thinking or may even have to reject it. Unless we take a time machine (or utilize historical organization in a qualitative study), we are unable to create the exact conditions necessary to disprove the findings. So, what can we do?
We can look into the validity of the qualitative research by checking the organization of the arguments presented and the usefulness of the descriptive analysis. If the study is logical and presents exceptional descriptions, then how can it be falsified?
One may say that the theory produced for that particular group may not suit present groups, ‘cause the conditions or context is different. Is it now falsified?????
What is the purpose to a qualitative study? How much can we depend on the universals that come out of such studies to make inferences about other groups? Do we use them as only a guide????
Maybe I am way off track, but I thought that the falsifiability idea better suits hard numerical data, which can more easily be disproved…Plus I think more important issues with qualitative studies are with how well the generalizations fit with the context of the study, and what happens soon after with that particular group…I don’t know, I think I m starting to confuse myself….
Steven M.
Wiersma & Jurs (2005) Research Methods in Education, Pearson, USA
Friday, November 23, 2007
Research Philosophical Essay: Where am I?
Hello everyone,
Week 1 coming to an end... where am I? Well, you could use a metaphor and say I was a sponge trying to soak up a pretty big spill, but unfortunately the spill is quite large (content area reading) and my sponge is a little ineffective (or my brain is OS9, trying to deal with i-life applications... bit of a MAC reference there...)...
What am I trying to come to terms with at the moment...and struggling...
The awards go to...
1. In the category of Jargon
~ and the biggest winners so far are Epistemology, Ontology, and Axiology and whateverelse-ology... Actually it is good to be able to put words to the things that I am thinking…but maybe don’t have a word for… I was reading an article dealing with subjectivity (Luhrmann T.M. 2006) that discussed emotions and how different cultures have different ways of expressing various feelings. Some cultures didn’t have a word for ‘sad’, but they expressed the feeling in another way.. It got me thinking about knowing the concepts behind words, but not having the jargon to talk about them in sufficient detail. Now, we have all this new jargon that expresses our ideas even more succinctly;-)
2. In the category of Concepts: The winner is ‘Falsifiability’.
I was particularly struggling with the concept of falsifiability… It is a concept that has come up a number of times in my study, but it is quite a difficult thing for me to grasp. The idea of having to have the logical possibility that something can be shown to be false is quite difficult for me to grasp. And it almost seems defeatist…in that we can only ever accept something until it is proven otherwise, and we can never feel absolute confidence in any decision or outcome… But then again, it allows for improvement, modification, or even change on a large scale…which can make one feel quite satisfied in that we don’t have to stick to traditions that start seeming to be inappropriate within changing societies. Tradition handed down from one generation to the next, is often accepted at face value, due to its long history…even though it could be wrong…case in point: “the world is flat…”
3. In the category of methods and approaches: Qualitative, quantitative and mixed methods win hands down... I see a lot of theory trying to say that these concepts have been thought of as mutually exclusive for a long time, but are now being thought of as complementary. I’m kind of moving toward this later idea. I believe that quantitative data and descriptive analysis can sit side by side…I just need some more information to prove it…When I did ‘Testing’ I noticed you could use both numerical data and qualitative descriptions in analytical scoring. It was quite satisfying, being able to give a number and description of the test takers ability on an item. Yet, when it comes to the student, what will they look at? The number or the description?
4. In the category of self awareness: What kind of researcher am I? Well,
A) Having grown up in Australia with an immigrant mother an Australia born father
B) Culture and language
C) Living in Japan as an Australian
D) Having taught in a variety of contexts.
These among other things are shaping the researcher I am. I am starting to think that the research we do is inextricably linked to our self identity, and what we bring to the research can determine how we conduct the body of our research. Although the paradigm of the research may be determined by the need established by the end users, and/or the needs set out by the inquiry, the approach and methods we use are accented by the person we are. What do I mean by this? Well, even though the goal in scientific method is to be as objective as possible, and try to limit all influences from the researcher, I think that it is extremely difficult to eliminate all subjectivity, even in the data analysis of quantitative studies. The questions we choose and the procedure we follow will have slight variations due to individual researcher bias. Just as no two cooks can make exactly the same meal, no two researchers will have exactly the same influence (or lack there of) on a research study.
This is where I am at the moment… It is not much, and definitely not very logical, but it is an attempt to think aloud and share some thoughts…
Steven Mondy
Luhrmann T.M. (2006) Subjectivity, SAGE publications, University of Chicago, USA
Week 1 coming to an end... where am I? Well, you could use a metaphor and say I was a sponge trying to soak up a pretty big spill, but unfortunately the spill is quite large (content area reading) and my sponge is a little ineffective (or my brain is OS9, trying to deal with i-life applications... bit of a MAC reference there...)...
What am I trying to come to terms with at the moment...and struggling...
The awards go to...
1. In the category of Jargon
~ and the biggest winners so far are Epistemology, Ontology, and Axiology and whateverelse-ology... Actually it is good to be able to put words to the things that I am thinking…but maybe don’t have a word for… I was reading an article dealing with subjectivity (Luhrmann T.M. 2006) that discussed emotions and how different cultures have different ways of expressing various feelings. Some cultures didn’t have a word for ‘sad’, but they expressed the feeling in another way.. It got me thinking about knowing the concepts behind words, but not having the jargon to talk about them in sufficient detail. Now, we have all this new jargon that expresses our ideas even more succinctly;-)
2. In the category of Concepts: The winner is ‘Falsifiability’.
I was particularly struggling with the concept of falsifiability… It is a concept that has come up a number of times in my study, but it is quite a difficult thing for me to grasp. The idea of having to have the logical possibility that something can be shown to be false is quite difficult for me to grasp. And it almost seems defeatist…in that we can only ever accept something until it is proven otherwise, and we can never feel absolute confidence in any decision or outcome… But then again, it allows for improvement, modification, or even change on a large scale…which can make one feel quite satisfied in that we don’t have to stick to traditions that start seeming to be inappropriate within changing societies. Tradition handed down from one generation to the next, is often accepted at face value, due to its long history…even though it could be wrong…case in point: “the world is flat…”
3. In the category of methods and approaches: Qualitative, quantitative and mixed methods win hands down... I see a lot of theory trying to say that these concepts have been thought of as mutually exclusive for a long time, but are now being thought of as complementary. I’m kind of moving toward this later idea. I believe that quantitative data and descriptive analysis can sit side by side…I just need some more information to prove it…When I did ‘Testing’ I noticed you could use both numerical data and qualitative descriptions in analytical scoring. It was quite satisfying, being able to give a number and description of the test takers ability on an item. Yet, when it comes to the student, what will they look at? The number or the description?
4. In the category of self awareness: What kind of researcher am I? Well,
A) Having grown up in Australia with an immigrant mother an Australia born father
B) Culture and language
C) Living in Japan as an Australian
D) Having taught in a variety of contexts.
These among other things are shaping the researcher I am. I am starting to think that the research we do is inextricably linked to our self identity, and what we bring to the research can determine how we conduct the body of our research. Although the paradigm of the research may be determined by the need established by the end users, and/or the needs set out by the inquiry, the approach and methods we use are accented by the person we are. What do I mean by this? Well, even though the goal in scientific method is to be as objective as possible, and try to limit all influences from the researcher, I think that it is extremely difficult to eliminate all subjectivity, even in the data analysis of quantitative studies. The questions we choose and the procedure we follow will have slight variations due to individual researcher bias. Just as no two cooks can make exactly the same meal, no two researchers will have exactly the same influence (or lack there of) on a research study.
This is where I am at the moment… It is not much, and definitely not very logical, but it is an attempt to think aloud and share some thoughts…
Steven Mondy
Luhrmann T.M. (2006) Subjectivity, SAGE publications, University of Chicago, USA
Saturday, November 17, 2007
Contextualizing of Research
Westerners come here in droves, staying a short time, but trying to impart western philosophical norms on a society that either resists or is blatantly opposed (yet not openly) to the ideologies that they try to impart. Many times I hear teacher's complaining about the 'sounds of silence' within the classroom, when not realizing the difference in learning styles that our Japanese learners have.
Communicative teaching and learning is something that we learned about in Methodology...but when it comes to my classroom, in Asia, students find it quite a struggle to change to a learning style that is not introduced to them in elementary and junior/senior high school.
Japanese learners are very good at adapting to new ways of learning, but still have trouble with so called western ways of doing things. True, they are learning English, and with that comes not only language but cultural aspects, too. Yet, one must consider the influence of many years of socialization and learning of cultural norms, within the way one views these learners when it comes to educational research.
Yes, contextualizing of research seems to be quite relevant, especially for more localized studies...
Communicative teaching and learning is something that we learned about in Methodology...but when it comes to my classroom, in Asia, students find it quite a struggle to change to a learning style that is not introduced to them in elementary and junior/senior high school.
Japanese learners are very good at adapting to new ways of learning, but still have trouble with so called western ways of doing things. True, they are learning English, and with that comes not only language but cultural aspects, too. Yet, one must consider the influence of many years of socialization and learning of cultural norms, within the way one views these learners when it comes to educational research.
Yes, contextualizing of research seems to be quite relevant, especially for more localized studies...
Quantitative, Qualitative and mixed methods...
Numbers tend to give a feeling of exactness and rightness (if I can say that), but as I think Hawkins states, you can just choose one model and work out all the predictions according to that particular model (as opposed to other equally valid models)... However, I feel that there will always be a point in which we reach that stops us, 'cause we do not have the capacity to continue...case in point computer technology. In an empirical only framework, our theories and ideas are only based on the next evolution of ideas and the technology that enables us to work out those ideas... How do we make advances in new ways of thinking? One way is being creative and being able to think outside of the box... I think subjectivity can give us a way to do that, but it has to go along with empirical positivism...in tandem...
Also, what is objectivity? How do you reduce emotional states and affective factors for instance to specific data? And once you do reduce it to facts and figures, how do you stop from turning that data into something that disadvantages the individual? I know that within positivism, there is the debate between individual vs. society, but looking at the outcome and usability of data, aren't there many cases where descriptive analysis and holistic expressions may be more useful than just data alone? I guess it depends on what the end user is needing the research to do. Action research on a very specific level may cry out for specific descriptions of the situation at hand, whereas, bigger studies on a national or even world level may need general trends, as the problem as stated in the research affects more people in greater variety. This is possibly where number crunching can be very effective.
At the moment, I see quantitative data sitting along side qualitative observation...And the benefit that comes from both is reflected in the end user...
Also, what is objectivity? How do you reduce emotional states and affective factors for instance to specific data? And once you do reduce it to facts and figures, how do you stop from turning that data into something that disadvantages the individual? I know that within positivism, there is the debate between individual vs. society, but looking at the outcome and usability of data, aren't there many cases where descriptive analysis and holistic expressions may be more useful than just data alone? I guess it depends on what the end user is needing the research to do. Action research on a very specific level may cry out for specific descriptions of the situation at hand, whereas, bigger studies on a national or even world level may need general trends, as the problem as stated in the research affects more people in greater variety. This is possibly where number crunching can be very effective.
At the moment, I see quantitative data sitting along side qualitative observation...And the benefit that comes from both is reflected in the end user...
Hitch hikers guide to the galaxy 2
Arthur Dent's voyage into the unknown somewhat harnesses the tie-in between subjective experience and empirical truths, according to Comte's (see Wikipedia ~ Positivism for very general description) view of Positivism and it's cyclic nature. One man traveling the universe in his pajamas and dressing gown, trying to make sense of things based on the limited experience he has had on an earth that is now non-existent. Arthur struggles to break things down into their simplest, understandable pieces, somewhat in the frame of reductionists trying to reduce everything into an ultimately measurable form.
The series also touches on the idea that we may have trouble understanding the world around us, 'cause we are part of the actual workings of the universe. Arthur, was the last remaining part of the super computer known as earth. If you are part of the search for the solution, you may never fully understand the actual problem. Possibly... Thus, as researchers, how do we view ourselves, within the whole structure of the inquiry? Do our actions actually change the results? Or even our non-participatory role, may influence unforeseen elements???
Accounting for variables is inherent in scientific method, but can those variables truly be eliminated? Viewing methods have their limitations, and often when we try to account for those variables, we rely on hypothesis and theory to create models of behavior that would occur according to rational thought. Back to Comte again and his cyclic view of things.
However, if theorists such as Emile Hennequin (Wikipedia - Positivism) who included subjectivity into the equation, were to look beyond just the empirical data, they mind find answers that are more creative, but still understand that they are still based on certain assumptions...
Do I make any sense?
The series also touches on the idea that we may have trouble understanding the world around us, 'cause we are part of the actual workings of the universe. Arthur, was the last remaining part of the super computer known as earth. If you are part of the search for the solution, you may never fully understand the actual problem. Possibly... Thus, as researchers, how do we view ourselves, within the whole structure of the inquiry? Do our actions actually change the results? Or even our non-participatory role, may influence unforeseen elements???
Accounting for variables is inherent in scientific method, but can those variables truly be eliminated? Viewing methods have their limitations, and often when we try to account for those variables, we rely on hypothesis and theory to create models of behavior that would occur according to rational thought. Back to Comte again and his cyclic view of things.
However, if theorists such as Emile Hennequin (Wikipedia - Positivism) who included subjectivity into the equation, were to look beyond just the empirical data, they mind find answers that are more creative, but still understand that they are still based on certain assumptions...
Do I make any sense?
Hitch hikers guide to the galaxy?
Does anyone remember a little British show called the Hitch hikers guide to the galaxy? In that program, a group of people created a machine to calculate the answer to the ultimate question, of life, the universe and everything. The final answer was 42. Everyone was happy to know the answer, but didn't really know what it meant. So they invented an even bigger computer to work out what the actual question was...
I guess this is my slightly warped, but interesting view of the whole idea of what research is. We may find answers to wonderful questions, but do we actually understand the questions themselves. Or even the methods of obtaining answers to questions? How can we be certain that the processes that we are using are either accurate or valid. When I studied the testing unit here at USQ, I came across the ideas of validity and reliability, and developing consistent and proven methods of evaluating students. Time seemed to factor into the equation, as well as healthy skepticism of ones own creations.
I guess that one thing that the hitch-hikers guide taught me, was that we may develop understandings, but those understandings must remain open, as there may be other interpretations to our theories. And the ideas we have or the questions we ask, may not provide us with satisfactory answers that we can be happy with.
This is not to say that we stop asking, but keep searching for better understanding...
What do you say?
Ciao
Steve M.
I guess this is my slightly warped, but interesting view of the whole idea of what research is. We may find answers to wonderful questions, but do we actually understand the questions themselves. Or even the methods of obtaining answers to questions? How can we be certain that the processes that we are using are either accurate or valid. When I studied the testing unit here at USQ, I came across the ideas of validity and reliability, and developing consistent and proven methods of evaluating students. Time seemed to factor into the equation, as well as healthy skepticism of ones own creations.
I guess that one thing that the hitch-hikers guide taught me, was that we may develop understandings, but those understandings must remain open, as there may be other interpretations to our theories. And the ideas we have or the questions we ask, may not provide us with satisfactory answers that we can be happy with.
This is not to say that we stop asking, but keep searching for better understanding...
What do you say?
Ciao
Steve M.
Friday, November 16, 2007
Technology in Education...
How I would love to take a trip into the future and see what is happening with technology in the classroom, and beyond.
I too, am very interested in technology, but more out of an interest, rather than a particular research focus. I haven't actually done any of the CAL, or technology in Ed. subjects, but I have taken it upon myself to introduce my students (in Japan), to the empowering aspects of utilizing technology in their studies. However, one thing that I have noticed, and in Japan mind you, is the lack of internet access among the student body. In Japan (especially among my students), mobile phone use far surpasses internet use (very subjective, I know...). So, one of the issues that I face is with computer/internet access. Students send e-mail from phones, and portable devices, that although convenient, can limit what they can do. I guess when i-phones come to Japan, at a reasonable price, we will start seeing a change in behavior patterns and greater freedom to do a wider variety of things.
Having said that, I am also a little cautious about going full steam into a situation of complete reliance on technology. A friend of mine has a son who has taken the computer based TOEFL test, and has commented on how different the testing style was. She talked about her son not being able to concentrate as effectively on that test, compared to the paper test, therefore affecting his score. I guess there is still a lot of ironing out of little problems to do, but computer based activities can offer a new scope for teachers and open up new avenues for students in their own learning.
I look on it as giving the best possible access to a great variety of material, to the greatest number of students. In the end, students have to choose to embrace new technologies, and we have to give them the chance to do so.
Ciao
Steven Mondy
I too, am very interested in technology, but more out of an interest, rather than a particular research focus. I haven't actually done any of the CAL, or technology in Ed. subjects, but I have taken it upon myself to introduce my students (in Japan), to the empowering aspects of utilizing technology in their studies. However, one thing that I have noticed, and in Japan mind you, is the lack of internet access among the student body. In Japan (especially among my students), mobile phone use far surpasses internet use (very subjective, I know...). So, one of the issues that I face is with computer/internet access. Students send e-mail from phones, and portable devices, that although convenient, can limit what they can do. I guess when i-phones come to Japan, at a reasonable price, we will start seeing a change in behavior patterns and greater freedom to do a wider variety of things.
Having said that, I am also a little cautious about going full steam into a situation of complete reliance on technology. A friend of mine has a son who has taken the computer based TOEFL test, and has commented on how different the testing style was. She talked about her son not being able to concentrate as effectively on that test, compared to the paper test, therefore affecting his score. I guess there is still a lot of ironing out of little problems to do, but computer based activities can offer a new scope for teachers and open up new avenues for students in their own learning.
I look on it as giving the best possible access to a great variety of material, to the greatest number of students. In the end, students have to choose to embrace new technologies, and we have to give them the chance to do so.
Ciao
Steven Mondy
Friday, November 9, 2007
CLT
CLT is an interesting idea, but there are certainly drawbacks, as well as positives. The ideals of developing communicative competence in a socially integrated way are quite appealing, but very difficult for the average teacher to implement.
I also find it difficult to balance the potential benefits of CLT activities (such as role-play, find someone, and info-gap) with the needs of individuals in a non-English speaking, essentially homogeneous environment.
I also find it difficult to balance the potential benefits of CLT activities (such as role-play, find someone, and info-gap) with the needs of individuals in a non-English speaking, essentially homogeneous environment.
Qualified teacher vs. Inexperienced teacher
1 August 2007 7:56 PM
Having thought about things, I now believe that CLT offers a lot with respect to taking many classroom activities and even textbooks and adapting them to help Ss develop their Communicative language competence... I think it has the flexibility and enough variety to allow for what I was hinting at with different style.
I am not sure whether style can equate with approach, but is closely related. Style might be an accent to approach, and be what each and every teacher brings to the whole process.
I don't know whether having an inexperienced teacher teaching me would be all that bad (as long as they have had enough life experience to offer anecdotal references), as they may bring with that inexperience a fresh, dare I say approach (or should I say style in this case). I understand that the inexperience can be quite inefficient, but the new and inexperienced teacher will be trying their hardest to make it and prove something....at least if there was variety, then it would be interesting...
Having thought about things, I now believe that CLT offers a lot with respect to taking many classroom activities and even textbooks and adapting them to help Ss develop their Communicative language competence... I think it has the flexibility and enough variety to allow for what I was hinting at with different style.
I am not sure whether style can equate with approach, but is closely related. Style might be an accent to approach, and be what each and every teacher brings to the whole process.
I don't know whether having an inexperienced teacher teaching me would be all that bad (as long as they have had enough life experience to offer anecdotal references), as they may bring with that inexperience a fresh, dare I say approach (or should I say style in this case). I understand that the inexperience can be quite inefficient, but the new and inexperienced teacher will be trying their hardest to make it and prove something....at least if there was variety, then it would be interesting...
Having different programs side by side
1 August 2007 4:31 PM
Surely it is possible to have content-based classes side by side (but not necessarily following on from) with other kinds of classes, lets say grammar translation, a small free talking tutorial, or an extensive reading type program. All require different approaches, but could fit within a school program. The students themselves may be able to transfer aspects of one system over to another. Just because English is the commonality btw both, it doesn't mean that the kind of class/approach to the class is/should be the same. It's the transfer of the ball in the football game (as Savignon puts it in "Teaching English as a second or foreign language"), not the actual ball itself, in this case English. We are talking about competence in using English overall. Students just adapt to a new system, transferring knowledge and skills from one activity to another. Variety! Doesn't that more closely resemble real life? Though students may like one style better than another. They may be a little confused in the beginning, but for the sake of individual differences, shouldn't we expose Ss to a variety of learning and teaching styles. If handled right, the program may in fact stimulate students...
Imagine studying 10 subjects with exactly the same teacher in exactly the same style. Wouldn't you get bored?
Do I make any sense? Probably not...
Surely it is possible to have content-based classes side by side (but not necessarily following on from) with other kinds of classes, lets say grammar translation, a small free talking tutorial, or an extensive reading type program. All require different approaches, but could fit within a school program. The students themselves may be able to transfer aspects of one system over to another. Just because English is the commonality btw both, it doesn't mean that the kind of class/approach to the class is/should be the same. It's the transfer of the ball in the football game (as Savignon puts it in "Teaching English as a second or foreign language"), not the actual ball itself, in this case English. We are talking about competence in using English overall. Students just adapt to a new system, transferring knowledge and skills from one activity to another. Variety! Doesn't that more closely resemble real life? Though students may like one style better than another. They may be a little confused in the beginning, but for the sake of individual differences, shouldn't we expose Ss to a variety of learning and teaching styles. If handled right, the program may in fact stimulate students...
Imagine studying 10 subjects with exactly the same teacher in exactly the same style. Wouldn't you get bored?
Do I make any sense? Probably not...
Comprehensible Output ~ Swain
Comprehensible Output ~ Swain 20 July 2007 11:20 PM
I’m am just in the middle of the first reading, and I want to say a big “Hooray” to Swain 1985, as cited in Hadley 2001), for proposing the comprehensible output hypothesis. All through the last subject (principles in language learning), I was reading and wondering about Krashen and his focus on comprehensible input, to the exclusion of a focus on production. Now I have to be careful, as I didn’t choose Krashen to investigate last subject, but to a novice like myself, I feel the need to encourage more outward production of language. Especially being here in Japan, where output is at a premium, and no matter what I do, it is difficult to encourage students to practice with others, and make use of the range of functions needed in whatever we are studying (Hypothesis 2).
One of my colleagues likened Krashen’s input hypothesis to a glass being filled (I’m sure he read it somewhere?!). Only when the glass is filled, will it start overflowing. I understand the logic behind this, but many of our students in the college where I work, tend to stay quiet for pretty much the whole time they are with us (maybe a little bit of an overgeneralization ;-). Why can’t they be encouraged to produce language, at least at levels relative to the student’s abilities? This is rather a big question that would incorporate not only affective variables, but social ones, too. I believe, that essentially students want to start producing in the TL as soon as possible as they wish to communicate, it’s just that they are unable to because they don’t have the communicative competence.
Having said that, I have had times when I gave students the opportunity to contribute, but they have not been ready to respond (think of the glass half full). And it ends up that they give up, or other students decide to respond, cause it takes and aggravatingly long time for the student to answer (I do try to discourage this king of situation, though). This has the potential of raising anxiety, and lifting that affective filter we learned about last semester. But, it’s up to the classroom teacher to read the class and give out opportunities to students to respond at various times in their particular development. Students need opportunities to test the structures they have learned, and the hypotheses they are making about language and the target culture.
So, Output, I say yes…we need to encourage a certain amount of risk taking and foster within students the courage to take chances in the TL.
Steven Mondy
Hadley, A.O. (2001), Teaching language in context 3rd Ed., Heinle and Heinle.
I’m am just in the middle of the first reading, and I want to say a big “Hooray” to Swain 1985, as cited in Hadley 2001), for proposing the comprehensible output hypothesis. All through the last subject (principles in language learning), I was reading and wondering about Krashen and his focus on comprehensible input, to the exclusion of a focus on production. Now I have to be careful, as I didn’t choose Krashen to investigate last subject, but to a novice like myself, I feel the need to encourage more outward production of language. Especially being here in Japan, where output is at a premium, and no matter what I do, it is difficult to encourage students to practice with others, and make use of the range of functions needed in whatever we are studying (Hypothesis 2).
One of my colleagues likened Krashen’s input hypothesis to a glass being filled (I’m sure he read it somewhere?!). Only when the glass is filled, will it start overflowing. I understand the logic behind this, but many of our students in the college where I work, tend to stay quiet for pretty much the whole time they are with us (maybe a little bit of an overgeneralization ;-). Why can’t they be encouraged to produce language, at least at levels relative to the student’s abilities? This is rather a big question that would incorporate not only affective variables, but social ones, too. I believe, that essentially students want to start producing in the TL as soon as possible as they wish to communicate, it’s just that they are unable to because they don’t have the communicative competence.
Having said that, I have had times when I gave students the opportunity to contribute, but they have not been ready to respond (think of the glass half full). And it ends up that they give up, or other students decide to respond, cause it takes and aggravatingly long time for the student to answer (I do try to discourage this king of situation, though). This has the potential of raising anxiety, and lifting that affective filter we learned about last semester. But, it’s up to the classroom teacher to read the class and give out opportunities to students to respond at various times in their particular development. Students need opportunities to test the structures they have learned, and the hypotheses they are making about language and the target culture.
So, Output, I say yes…we need to encourage a certain amount of risk taking and foster within students the courage to take chances in the TL.
Steven Mondy
Hadley, A.O. (2001), Teaching language in context 3rd Ed., Heinle and Heinle.
Approach - How do you incorporate that into a curriculum or Syllabus?
31 July 2007 9:46 AM
How do you incorporate that into a curriculum or Syllabus?
Which approach will give enough freedom of movement?
Do we just allow all teachers the complete freedom to chop and change like the weather, or do we do like ECC (English teaching school in Japan) and prescribe the step-by-step procedures, give all the materials, and expect teachers to adhere to the program?????
Well, the answer to both is yes and no. Complete freedom in teaching technique will be excellent, in that it allows teacher autonomy and creativity, but it doesn't really support the teacher in terms of a suggested course. A dictated curriculum is great for the new or lazy teacher, in that everything is prepared (almost like factory work - does that remind us of any job we've had?), but limits the possible potential of a completely unplanned learning experience.
What I was trying to get at was that teachers need a framework to work by, but be open to the odd side track that will open up those golden opportunities...
For a curriculum planner or coordinator, it would be a headache trying to balance the two. As I was saying in my previous entry, too much freedom makes it difficult to maintain consistency...
Thank you Edith, I too try to never miss an opportunity...
Steven Mondy
How do you incorporate that into a curriculum or Syllabus?
Which approach will give enough freedom of movement?
Do we just allow all teachers the complete freedom to chop and change like the weather, or do we do like ECC (English teaching school in Japan) and prescribe the step-by-step procedures, give all the materials, and expect teachers to adhere to the program?????
Well, the answer to both is yes and no. Complete freedom in teaching technique will be excellent, in that it allows teacher autonomy and creativity, but it doesn't really support the teacher in terms of a suggested course. A dictated curriculum is great for the new or lazy teacher, in that everything is prepared (almost like factory work - does that remind us of any job we've had?), but limits the possible potential of a completely unplanned learning experience.
What I was trying to get at was that teachers need a framework to work by, but be open to the odd side track that will open up those golden opportunities...
For a curriculum planner or coordinator, it would be a headache trying to balance the two. As I was saying in my previous entry, too much freedom makes it difficult to maintain consistency...
Thank you Edith, I too try to never miss an opportunity...
Steven Mondy
Approach 1
21 July 2007 12:01 AM
I see the approach, design and procedure model in the realm of the syllabus designer, who needs to contemplate how the principles of learning and teaching relate to actual practical application and implementation, however that may be stating the obvious??? Each approach requires specific thought to how it is to be presented.
Richards and Rodgers (2006) use the approach, design and procedure model and apply it to the oral, audio-lingual, TPR, silent way, community language learning, Suggestopedia, whole language, MI theory, NLP, lexical, CLBT approach and so on...
It seems that in any situation, thought must be given to the theoretical principles that underlie it, what objectives of each method, the types of learning tasks and the roles of learner and teacher. Once thought has been given to these aspects, then we need to understand which tasks will foster the learning principles and then achieve the particular objectives that are set.
Most of the time, busy or inexperienced teachers will rely heavily on the textbook to decide on these things. Yet, as we become more familiar with teaching approaches, we can sculpt our lessons (design), using activities (procedure) that encourage greater student participation and interaction.
I see the approach, design and procedure model in the realm of the syllabus designer, who needs to contemplate how the principles of learning and teaching relate to actual practical application and implementation, however that may be stating the obvious??? Each approach requires specific thought to how it is to be presented.
Richards and Rodgers (2006) use the approach, design and procedure model and apply it to the oral, audio-lingual, TPR, silent way, community language learning, Suggestopedia, whole language, MI theory, NLP, lexical, CLBT approach and so on...
It seems that in any situation, thought must be given to the theoretical principles that underlie it, what objectives of each method, the types of learning tasks and the roles of learner and teacher. Once thought has been given to these aspects, then we need to understand which tasks will foster the learning principles and then achieve the particular objectives that are set.
Most of the time, busy or inexperienced teachers will rely heavily on the textbook to decide on these things. Yet, as we become more familiar with teaching approaches, we can sculpt our lessons (design), using activities (procedure) that encourage greater student participation and interaction.
Approach 2
30 July 2007 7:40 PM
I think that the approach we take will have a profound impact on the way we go about choosing objectives and in determining activities that will help in achieving them. If one were to have an approach based on behaviorist principles, then the objectives will probably be ones that encourage responses to a stimulus, thus requiring a selection of various drills, for instance. A large language school in Japan (NOVA), works on similar principles, with their use of American Streamline, a text (and some may say an approach) so outdated, it seems laughable that it is still being marketed today. If one were to follow in the steps of the audio-lingual approach, then the objectives will allow for the soaking up time that is required, and be less intrusive, and more teacher centered teaching may take place, until the students are at the point of producing language. Then there is the opposite end of the continuum, with the silent way approach, which has at its heart the idea that a student needs to start producing from the very beginning. So, as you can see, changing the approach will undoubtedly change the way the lesson is taught. In fact, it will shape the way that learners deal with their own learning, while at our institutions. The activities that they will be undertaking have vastly different outcomes, and require different things from students.
My question is:
“If an approach is decided upon within a school, do all teachers, in all classes need to take on that approach?”
I know of a situation in which a coordinator is forcing all teachers to take on communicative language teaching principles within a program, regardless of teacher gripes, or student difficulties with that particular approach. An approach may not necessarily work for all people, all of the time. After all, we need to also take in cultural aspects, or individual differences, for instance that may make one approach ineffective, in some cases, some of the time.
Then there’s what is happening in our college, a kind of free for all with, as many approaches as there are teachers. This too, is not an ideal situation, as there is no consistency (something in fact, that I need in my own learning).
In the end, we have to balance the needs and abilities of the students we teach. We also have to keep in mind that only one approach to teaching may severely disadvantage our students, if it so happens that other approaches are found to be more beneficial. I guess it’s an argument over consistency vs. flexibility. There are merits and demerits in both. I tend to be on the flexibility side, as long as it is based in firm and logical principles. There has to be a reason to change. I remember a lecturer introducing us to the, ‘if a bird were to fly into the room theory…what would you do?’ Carry on with your lesson plan or start talking about the bird (I think he was talking about Herbert Kohl, but I could be wrong…). That idea, although a little outdated (alternative education movement), has had a deep impact on me personally.
We need to establish curriculum's and syllabi that are based on consistent principles, yet they must also have within them some flexibility to allow for the ‘bird flying in the room’ factor.
What do you say?????
Steven Mondy
I think that the approach we take will have a profound impact on the way we go about choosing objectives and in determining activities that will help in achieving them. If one were to have an approach based on behaviorist principles, then the objectives will probably be ones that encourage responses to a stimulus, thus requiring a selection of various drills, for instance. A large language school in Japan (NOVA), works on similar principles, with their use of American Streamline, a text (and some may say an approach) so outdated, it seems laughable that it is still being marketed today. If one were to follow in the steps of the audio-lingual approach, then the objectives will allow for the soaking up time that is required, and be less intrusive, and more teacher centered teaching may take place, until the students are at the point of producing language. Then there is the opposite end of the continuum, with the silent way approach, which has at its heart the idea that a student needs to start producing from the very beginning. So, as you can see, changing the approach will undoubtedly change the way the lesson is taught. In fact, it will shape the way that learners deal with their own learning, while at our institutions. The activities that they will be undertaking have vastly different outcomes, and require different things from students.
My question is:
“If an approach is decided upon within a school, do all teachers, in all classes need to take on that approach?”
I know of a situation in which a coordinator is forcing all teachers to take on communicative language teaching principles within a program, regardless of teacher gripes, or student difficulties with that particular approach. An approach may not necessarily work for all people, all of the time. After all, we need to also take in cultural aspects, or individual differences, for instance that may make one approach ineffective, in some cases, some of the time.
Then there’s what is happening in our college, a kind of free for all with, as many approaches as there are teachers. This too, is not an ideal situation, as there is no consistency (something in fact, that I need in my own learning).
In the end, we have to balance the needs and abilities of the students we teach. We also have to keep in mind that only one approach to teaching may severely disadvantage our students, if it so happens that other approaches are found to be more beneficial. I guess it’s an argument over consistency vs. flexibility. There are merits and demerits in both. I tend to be on the flexibility side, as long as it is based in firm and logical principles. There has to be a reason to change. I remember a lecturer introducing us to the, ‘if a bird were to fly into the room theory…what would you do?’ Carry on with your lesson plan or start talking about the bird (I think he was talking about Herbert Kohl, but I could be wrong…). That idea, although a little outdated (alternative education movement), has had a deep impact on me personally.
We need to establish curriculum's and syllabi that are based on consistent principles, yet they must also have within them some flexibility to allow for the ‘bird flying in the room’ factor.
What do you say?????
Steven Mondy
Sunday, October 21, 2007
What is the linguistic wealth that we bring to this course?
Topic: What is the linguistic wealth that we bring to this course? Date: 14 November 2006 3:06 PM
Subject: Theory into practise... Author: Mondy, Steven
Where do we kick this discussion off? Well, at the moment, our college is in the middle of our mid-term examinations. And I am in the process of thinking about how to test, not only a variety of skill areas, but varying levels and students with different purposes. I have many concerns regarding the appropriateness or validity of all that I do in testing situations. Reading Hughes, I see that testing where possible is to be seen as a group task, and that we should be assessing our own tests and their purposes. I think that essentially this is needed and in theory a given, but I sometimes find that the situations that we find ourselves in make it hard to do. Making our mid-term examinations, while juggling class teaching requirements and college commitments can be quite a handful. I guess what I’m trying to say is that it is not always possible to take time with the test making process, and we may not be able to always give very much attention to evaluation until much later. How do we as teachers start to implement some of the necessary theory into our real life working situations? This is where balance and fortitude is needed in making small changes to the things that we are already doing. We are already using various strategies and techniques, such as suggested in the readings under the different approaches (integrative, communicative etc…). There is definitely a need to understand the effect that our choices of items and complete examinations have on the overall motivation of students within our classes, and to their whole attitude toward language. What do you think?
Steven Mondy
Subject: Theory into practise... Author: Mondy, Steven
Where do we kick this discussion off? Well, at the moment, our college is in the middle of our mid-term examinations. And I am in the process of thinking about how to test, not only a variety of skill areas, but varying levels and students with different purposes. I have many concerns regarding the appropriateness or validity of all that I do in testing situations. Reading Hughes, I see that testing where possible is to be seen as a group task, and that we should be assessing our own tests and their purposes. I think that essentially this is needed and in theory a given, but I sometimes find that the situations that we find ourselves in make it hard to do. Making our mid-term examinations, while juggling class teaching requirements and college commitments can be quite a handful. I guess what I’m trying to say is that it is not always possible to take time with the test making process, and we may not be able to always give very much attention to evaluation until much later. How do we as teachers start to implement some of the necessary theory into our real life working situations? This is where balance and fortitude is needed in making small changes to the things that we are already doing. We are already using various strategies and techniques, such as suggested in the readings under the different approaches (integrative, communicative etc…). There is definitely a need to understand the effect that our choices of items and complete examinations have on the overall motivation of students within our classes, and to their whole attitude toward language. What do you think?
Steven Mondy
Validity and reliability - issues and discussion
Topic: Validity and reliability - issues and discussion Date: 22 November 2006 1:35 PM
Subject: Usefulness??? Author: Mondy, Steven View PeopleLink Options for this User
It should be understood that validity and reliability are but two categories within the general notion of understanding a test’s usefulness. According to Bachman & Palmer, we need to understand a test’s usefulness ~ encompassing reliability, construct validity, authenticity, interactiveness, impact, and practicality (Bachman & Palmer, 1996, Language Testing in Practice, Oxford University Press, p. 18).
They go on to talk about how reliability is a necessary aspect of validity, but it does not necessarily meet the needs of validity, in all cases. What does this mean? Well, it may have something to do with a test being able to fulfill the requirements of getting consistent results over time, however, using the same test in varying situations (such as a proven grammar test, being used to test someone’s speaking ability) may be inappropriate as it does not address all the other aspects to a students speaking ability. It is, as a solitary test for speaking ability, invalid. What is to be learned from this? One thing is that we have to understand the aspects of the skill area that we are testing, to design not only a reliable test, but also a test that retrieves the results that we should be looking for.
We need to also understand that we should to be careful about using tests or test items that we know have worked in the past, with our existing students. There could be many things that invalidate the test. If we aren’t careful of what it is we need to test (our objectives), what we have actually taught in our classes, and our student’s needs, we may end up with results that are actually more harmful, because of bad backwash. Do I make any sense?
Steven Mondy
Subject: Usefulness??? Author: Mondy, Steven View PeopleLink Options for this User
It should be understood that validity and reliability are but two categories within the general notion of understanding a test’s usefulness. According to Bachman & Palmer, we need to understand a test’s usefulness ~ encompassing reliability, construct validity, authenticity, interactiveness, impact, and practicality (Bachman & Palmer, 1996, Language Testing in Practice, Oxford University Press, p. 18).
They go on to talk about how reliability is a necessary aspect of validity, but it does not necessarily meet the needs of validity, in all cases. What does this mean? Well, it may have something to do with a test being able to fulfill the requirements of getting consistent results over time, however, using the same test in varying situations (such as a proven grammar test, being used to test someone’s speaking ability) may be inappropriate as it does not address all the other aspects to a students speaking ability. It is, as a solitary test for speaking ability, invalid. What is to be learned from this? One thing is that we have to understand the aspects of the skill area that we are testing, to design not only a reliable test, but also a test that retrieves the results that we should be looking for.
We need to also understand that we should to be careful about using tests or test items that we know have worked in the past, with our existing students. There could be many things that invalidate the test. If we aren’t careful of what it is we need to test (our objectives), what we have actually taught in our classes, and our student’s needs, we may end up with results that are actually more harmful, because of bad backwash. Do I make any sense?
Steven Mondy
Validity and reliability - issues and discussion 2
Topic: Validity and reliability - issues and discussion Date: 4 December 2006 11:41 PM
Subject: Re:Very testy indeed Author: Mondy, Steven
I kind of resist using anything more than once as I feel, now this is only on a personal level, that it isn't fair to the student who pays X amount of $ to get up to date, well thought out, original, and unique material. Of course, re-inventing the wheel is a laborious and somewhat futile endeavor, yet shouldn't we try to personalize our teaching as much as possible. If students know that a test is being used in basically the same format for years on end, then doesn't that automatically invalidate the test.
I understand that the issue of creating and maintaining reliability is paramount, yet I think that is also essential that we remain vibrant and abreast to the changes in our teaching environment. That may mean whole changes to not only the test, but to the content of the course and the objectives.
Each student is an individual, learning within a certain context. Our programs need to have flexibility in order to respond to that. It’s too easy to expect students to fit our models. Yet, maintaining our formats, will also maintain our teaching objectives, and keep up standards, not to mention strengthen the basis on which we can make judgments about overall ability.
What are we to do?
Steven Mondy
Subject: Re:Very testy indeed Author: Mondy, Steven
I kind of resist using anything more than once as I feel, now this is only on a personal level, that it isn't fair to the student who pays X amount of $ to get up to date, well thought out, original, and unique material. Of course, re-inventing the wheel is a laborious and somewhat futile endeavor, yet shouldn't we try to personalize our teaching as much as possible. If students know that a test is being used in basically the same format for years on end, then doesn't that automatically invalidate the test.
I understand that the issue of creating and maintaining reliability is paramount, yet I think that is also essential that we remain vibrant and abreast to the changes in our teaching environment. That may mean whole changes to not only the test, but to the content of the course and the objectives.
Each student is an individual, learning within a certain context. Our programs need to have flexibility in order to respond to that. It’s too easy to expect students to fit our models. Yet, maintaining our formats, will also maintain our teaching objectives, and keep up standards, not to mention strengthen the basis on which we can make judgments about overall ability.
What are we to do?
Steven Mondy
Reliability in once-off tests
Topic: Validity and reliability - issues and discussion Date: 26 November 2006 5:45 PM
Subject: Reliability in once-off tests Author: Mondy, Steven
How do we ensure reliability in a test we make and use only once? Often I find myself in a situation in which I’m unable to use a test again in the same kind of context, or with students with similar abilities. I often find that I may need to wait 2-3 years before actually being able to teach the same content. By that time, the tests lose validity? Reliability is closely related to having items that are consistent over time. Hughes and Bachman talk about creating item banks, which store items that we know are reliable, in order to reduce the effort that’s needed to make tests. I guess this is one way to be more certain that whether a test is a one-time thing, or to be used again, the content of the test will be reliable. Also, we shouldn’t understand a test only in terms of a complete, unchangeable entity, but a series of well-formulated items that put together correctly, can be reliable and valid for each and every time we use them. So, what I’m trying to say here is, whenever we make a test, we should try to create a test that may have items that are often used again and again, but not necessarily tests as a whole that are used again and again. Is it really possible to use an intact, unchangeable test in exactly the same way with new students anyway?
I’ve also been thinking of how one can develop tests that are specifically used more as a teaching tool, rather than a measurement or evaluation instrument. If students are prevented from seeing a test and analyzing the mistakes that are made in order to solely ensure reliability, then the student is missing a valuable learning experience. I think that tests can be used as a way to have students experience giving an answer, and then checking their answers with a model, or having an instructed lesson following the tests administration (looking over the actual test) in order to clarify the main points of the test within a student’s mind. Of course, we may need to forgo the ability to use the test in the future, but what can be gained may outweigh the possible disadvantages. This may be where the idea of item banks may help?! What do you think?
Steven Mondy
Subject: Reliability in once-off tests Author: Mondy, Steven
How do we ensure reliability in a test we make and use only once? Often I find myself in a situation in which I’m unable to use a test again in the same kind of context, or with students with similar abilities. I often find that I may need to wait 2-3 years before actually being able to teach the same content. By that time, the tests lose validity? Reliability is closely related to having items that are consistent over time. Hughes and Bachman talk about creating item banks, which store items that we know are reliable, in order to reduce the effort that’s needed to make tests. I guess this is one way to be more certain that whether a test is a one-time thing, or to be used again, the content of the test will be reliable. Also, we shouldn’t understand a test only in terms of a complete, unchangeable entity, but a series of well-formulated items that put together correctly, can be reliable and valid for each and every time we use them. So, what I’m trying to say here is, whenever we make a test, we should try to create a test that may have items that are often used again and again, but not necessarily tests as a whole that are used again and again. Is it really possible to use an intact, unchangeable test in exactly the same way with new students anyway?
I’ve also been thinking of how one can develop tests that are specifically used more as a teaching tool, rather than a measurement or evaluation instrument. If students are prevented from seeing a test and analyzing the mistakes that are made in order to solely ensure reliability, then the student is missing a valuable learning experience. I think that tests can be used as a way to have students experience giving an answer, and then checking their answers with a model, or having an instructed lesson following the tests administration (looking over the actual test) in order to clarify the main points of the test within a student’s mind. Of course, we may need to forgo the ability to use the test in the future, but what can be gained may outweigh the possible disadvantages. This may be where the idea of item banks may help?! What do you think?
Steven Mondy
Reliability in once-off tests 2
Topic: Validity and reliability - issues and discussion Date: 30 November 2006 10:07 PM
Subject: Reliability in once-off tests Author: Mondy, Steven
I like to think of a test as something that not only gauge someone's ability, but as a tool that helps a student see where they need to improve. I was doing some speed reading work in a class today that had simple multiple choice recognition items that helped students see a general overall comprehension score in relation to the time it took them to read a passage. By the time we got through 4 passages, students were able to see a pattern being established, which helped them make their own judgements on their reading ability.
In other classes, I tend to buck our school policy a little and let students mark their own tests, or their partners. One disadvantage would be that it could discourage students who are doing badly. On the other hand, it promotes a common mateship within a class, and a little bit of a competative spirit. Handled well, it can be quite a motivating tool, and reduce the negative image toward taking tests. I understand the need to withold tests on large scale apptitude type tests, but for classroom evaluation and learning, there are times when students need to see the test.
Does anyone else agree with this?
Steven
Subject: Reliability in once-off tests Author: Mondy, Steven
I like to think of a test as something that not only gauge someone's ability, but as a tool that helps a student see where they need to improve. I was doing some speed reading work in a class today that had simple multiple choice recognition items that helped students see a general overall comprehension score in relation to the time it took them to read a passage. By the time we got through 4 passages, students were able to see a pattern being established, which helped them make their own judgements on their reading ability.
In other classes, I tend to buck our school policy a little and let students mark their own tests, or their partners. One disadvantage would be that it could discourage students who are doing badly. On the other hand, it promotes a common mateship within a class, and a little bit of a competative spirit. Handled well, it can be quite a motivating tool, and reduce the negative image toward taking tests. I understand the need to withold tests on large scale apptitude type tests, but for classroom evaluation and learning, there are times when students need to see the test.
Does anyone else agree with this?
Steven
Reliability in once-off tests 3
Topic: Validity and reliability - issues and discussion Date: 4 December 2006 9:08 PM
Subject:Reliability in once-off tests Author: Mondy, Steven
Unfortunately, the speed reading is only a unit within a more general course on reading skills, the purpose of which is to increase students ability in gaining information more efficiently. The book I've enlisted is "More Reading Power", which has a series of reading passages in it (all roughly the same length and complexity). Each of the readings can/should be done within a five minute time span. It has lists of reading time to words/minute and a graph for students to plot their own reading time. The onus is on the student themselves to do the comprehension without cheating, mark their own papers, and record all information.
The activity has worked, but I recommend strongly that it be used with high intermediate~advanced students anyway (who are more likely to be motivated to doing the activity).
As for assessment, I'm still struggling on that one. As it is a skill that I want them to improve, I consider their achievements in increasing their comprehension to reading speed a key factor. This is where some kind of criterion referenced list (as a rubric) may come into play to see if each student has 1. been able to improve and 2. If students have achieved a certain level of proficiency.
I'm still juggling what to do, but luckily I still have time (with the approach of our winter vacation).
Hope that I have shed some light on how I approach or plan to approach speed reading.
As for more standardized tests that you are referring to, I can just say that lots of practice may be the key. In those practice tests, you can show students visually on their graphs, and this may give them the motivation to soldier on. You may need to think very carefully about the material you're teaching in close relationship to the students actual abilities, and think about how to test to increase positive results in the students minds. We can't just rely on intrinsic motivation (within the students), but look at giving them an outside reason to be motivated. This may have to be a step by step process, somewhat like a reward system, however, the rewards may need to be made explicit, in terms of concrete visual pictures (graphs and so forth). Do I make any sense? I think I may have just babbled and not really synthesized what I actually mean...!
Steven Mondy
Subject:Reliability in once-off tests Author: Mondy, Steven
Unfortunately, the speed reading is only a unit within a more general course on reading skills, the purpose of which is to increase students ability in gaining information more efficiently. The book I've enlisted is "More Reading Power", which has a series of reading passages in it (all roughly the same length and complexity). Each of the readings can/should be done within a five minute time span. It has lists of reading time to words/minute and a graph for students to plot their own reading time. The onus is on the student themselves to do the comprehension without cheating, mark their own papers, and record all information.
The activity has worked, but I recommend strongly that it be used with high intermediate~advanced students anyway (who are more likely to be motivated to doing the activity).
As for assessment, I'm still struggling on that one. As it is a skill that I want them to improve, I consider their achievements in increasing their comprehension to reading speed a key factor. This is where some kind of criterion referenced list (as a rubric) may come into play to see if each student has 1. been able to improve and 2. If students have achieved a certain level of proficiency.
I'm still juggling what to do, but luckily I still have time (with the approach of our winter vacation).
Hope that I have shed some light on how I approach or plan to approach speed reading.
As for more standardized tests that you are referring to, I can just say that lots of practice may be the key. In those practice tests, you can show students visually on their graphs, and this may give them the motivation to soldier on. You may need to think very carefully about the material you're teaching in close relationship to the students actual abilities, and think about how to test to increase positive results in the students minds. We can't just rely on intrinsic motivation (within the students), but look at giving them an outside reason to be motivated. This may have to be a step by step process, somewhat like a reward system, however, the rewards may need to be made explicit, in terms of concrete visual pictures (graphs and so forth). Do I make any sense? I think I may have just babbled and not really synthesized what I actually mean...!
Steven Mondy
correlation coefficients and true score?
Topic: Validity and reliability - issues and discussion Date: 27 November 2006 11:36 AM
Subject: correlation coefficients and true score? Author: Mondy, Steven
Last night, I was reading about how knowing the correlation coefficient (positive, negative, and no relationship) is quite a useful way to determine the reliability of a test. The study book was talking about how highly correlated scores can mean high reliability. It got me thinking about specific ways of gauging the reliability of our tests and test items. It was suggested that we could do things like give the same test twice (as long as not too much time has elapsed between the administration of the tests), or have such things as split tests, where we compare the two parts of one test. Up until this point, I didn’t really see the value of giving a test more than once in relatively the same, if not exactly the same format. I thought that it might be a waste of time, as I have been always encouraged to believe a test must be held back in extreme secrecy to not influence cheating and studying for specific items. According to the study materials, we see that there may be a change in the scores in that they may be slightly higher, but we can account for this in working out the students overall performance, and correlate the scores to see how that student performed in relation to other students doing the same test (norm referencing?). Then we can see if it has been error from the test format or from other things such as communicative ability, personal attributes, or other random factors (study materials: module 4).
Another interesting thing that I came across was the comparison of the two students with different scores, and being able to say that a student with a slightly higher score in a test may not be better than the student with the lower score, if we take into account their true score (true score=observed score +-measurement error). This was quite striking as I had assumed that ranking students in order of best to worst according to their raw scores was sufficient to get an idea of general performance. This idea of true score has made me rethink how I look at data, especially in terms of what judgments I make when I see the scores of two or more students are very close. What makes the difference between an A and an A- for example, if the range overlaps so significantly?
What do you think about correlation coefficients and true score?
Steven Mondy
Subject: correlation coefficients and true score? Author: Mondy, Steven
Last night, I was reading about how knowing the correlation coefficient (positive, negative, and no relationship) is quite a useful way to determine the reliability of a test. The study book was talking about how highly correlated scores can mean high reliability. It got me thinking about specific ways of gauging the reliability of our tests and test items. It was suggested that we could do things like give the same test twice (as long as not too much time has elapsed between the administration of the tests), or have such things as split tests, where we compare the two parts of one test. Up until this point, I didn’t really see the value of giving a test more than once in relatively the same, if not exactly the same format. I thought that it might be a waste of time, as I have been always encouraged to believe a test must be held back in extreme secrecy to not influence cheating and studying for specific items. According to the study materials, we see that there may be a change in the scores in that they may be slightly higher, but we can account for this in working out the students overall performance, and correlate the scores to see how that student performed in relation to other students doing the same test (norm referencing?). Then we can see if it has been error from the test format or from other things such as communicative ability, personal attributes, or other random factors (study materials: module 4).
Another interesting thing that I came across was the comparison of the two students with different scores, and being able to say that a student with a slightly higher score in a test may not be better than the student with the lower score, if we take into account their true score (true score=observed score +-measurement error). This was quite striking as I had assumed that ranking students in order of best to worst according to their raw scores was sufficient to get an idea of general performance. This idea of true score has made me rethink how I look at data, especially in terms of what judgments I make when I see the scores of two or more students are very close. What makes the difference between an A and an A- for example, if the range overlaps so significantly?
What do you think about correlation coefficients and true score?
Steven Mondy
Validity and Reliability Brainstorm
Topic: Validity and reliability - issues and discussion Date: 30 November 2006 12:51 PM
Subject: Validity and Reliability Brainstorm Author: Mondy, Steven
I've read the Hughes book as well as a lot of both Bachman books and what seems to be the case is that they present a very good definition of both construct validity and reliability and some of the problems associated with them, and give some comparison in the summaries at the end of the respective chapters. Though, I think it may be up to us to come up with a good idea of how they inter-relate, based on what we have learned about the essence of each.
Personally, I've found the Bachman books (Fundamental considerations in testing and Language testing in practice) more helpful for understanding the concepts, as they go into greater detail (a great part of those books is in the material that we were supplied with).
Maybe if we can start a brainstorm on this discussion board of the major issues involved in looking at how they inter-relate, or affect each other. Let me start listing a few ideas that I have come across in the readings…(please correct me if I’m way off base)
1. Reliability and construct validity are concerned with (a) minimizing measurement error (b) maximizing the effects of the language abilities that we are trying to measure. (Bachman, Fundamental considerations in testing, p.161)
2. Reliability is a necessary condition of construct validity, and hence for usefulness. (Bachman, Palmer, Language testing in practice, p.23). Bachman states in the summary on page 289 (Bachman, Fundamental considerations in testing, p.171) that validity is one of the most important qualities to consider in how we use tests, but reliability is necessary to determine potential sources of measurement error and for being able to predict the effects on test scores.
3. Reliability can be defined in terms of CTS theory (classical true score measurement theory) in terms of the correlation of true scores on tests (Bachman, Fundamental considerations in testing, p.171). As long as there is consistency between the scores of various candidates, we can be more certain of item reliability, and therefore of reduced error in testing technique.
4. Please correct me if I’m wrong, but validity seems to be concerned with having a test that closely matches the circumstances of the candidates taking the test, and creating a testing environment that allows for the best possible results to be obtained.
5………
6………
Can anyone add to the list…
Steven Mondy
Subject: Validity and Reliability Brainstorm Author: Mondy, Steven
I've read the Hughes book as well as a lot of both Bachman books and what seems to be the case is that they present a very good definition of both construct validity and reliability and some of the problems associated with them, and give some comparison in the summaries at the end of the respective chapters. Though, I think it may be up to us to come up with a good idea of how they inter-relate, based on what we have learned about the essence of each.
Personally, I've found the Bachman books (Fundamental considerations in testing and Language testing in practice) more helpful for understanding the concepts, as they go into greater detail (a great part of those books is in the material that we were supplied with).
Maybe if we can start a brainstorm on this discussion board of the major issues involved in looking at how they inter-relate, or affect each other. Let me start listing a few ideas that I have come across in the readings…(please correct me if I’m way off base)
1. Reliability and construct validity are concerned with (a) minimizing measurement error (b) maximizing the effects of the language abilities that we are trying to measure. (Bachman, Fundamental considerations in testing, p.161)
2. Reliability is a necessary condition of construct validity, and hence for usefulness. (Bachman, Palmer, Language testing in practice, p.23). Bachman states in the summary on page 289 (Bachman, Fundamental considerations in testing, p.171) that validity is one of the most important qualities to consider in how we use tests, but reliability is necessary to determine potential sources of measurement error and for being able to predict the effects on test scores.
3. Reliability can be defined in terms of CTS theory (classical true score measurement theory) in terms of the correlation of true scores on tests (Bachman, Fundamental considerations in testing, p.171). As long as there is consistency between the scores of various candidates, we can be more certain of item reliability, and therefore of reduced error in testing technique.
4. Please correct me if I’m wrong, but validity seems to be concerned with having a test that closely matches the circumstances of the candidates taking the test, and creating a testing environment that allows for the best possible results to be obtained.
5………
6………
Can anyone add to the list…
Steven Mondy
Two kinds of tests
Topic: Validity and reliability - issues and discussion Date: 4 December 2006 1:45 PM
Subject: Two kinds of tests Author: Mondy, Steven View
Trying to clarify some ideas, I wish to talk about the following case study, that I witnessed.
Case Study:
A teacher for some reason gives students a surprise test (with no previous warning) and then chooses to give them the same test again, but this time with warning.
The student’s are let in on the process of choosing an alternative testing date.
Alternative number 1: One week later (teacher’ choice)
Alternative number 2: Two weeks later (student’s choice)
Surprise tests are not really fair, in that they take students off guard, putting them into a situation in which they are highly resistant to the process at hand and encourage negative attitudes to the test, subject and teacher, which promotes terrible backwash. However, in such a case as above, where the teacher is trying to gauge student’s ability in a spontaneous situation and correlate it with the students prepared-for responses, then I imagine that alternative number 1 would be the better choice to maintain reliability, but would it be invalid, as the students may feel that they need more time to prepare for the test? However, is it really possible to correlate the two tests anyway? Can the formula for working out the correlation between tests be adapted to this scenario (Taking a student’s impromptu ability and comparing it to their prepared answer, I mean)? I think that it would be quite a stretch, without some sort of criterion referencing, and matching to the course objectives.
If alternative 2 is chosen, then there is too much time between the administration of one test and that of the other ~ and the test would become unreliable due to things such as ‘Test Wiseness, ‘ among other factors. (Study book ~ module 4: 4.8)…
The teacher in the above example seems to be in a no-win situation all round. The whole situation seems to be flawed anyway. The teacher will need to consider the reason for giving this test and what information is actually retrieved, and how beneficial it is to the whole learning environment. At the very least, if the tests were to be undertaken, the teacher will need to warn students of the likelihood that this kind of testing will be given.
I wonder, in what situations these kinds of surprise tests could be used in the classroom? There could be a situation, in some speaking classes, where there is a need to know how the student reacts to a new situation, based on their working knowledge of the language. Unfortunately, when in the real world, we cannot prepare for all situations. So, why not have tests that mimic this communicative task?
Steven Mondy
Subject: Two kinds of tests Author: Mondy, Steven View
Trying to clarify some ideas, I wish to talk about the following case study, that I witnessed.
Case Study:
A teacher for some reason gives students a surprise test (with no previous warning) and then chooses to give them the same test again, but this time with warning.
The student’s are let in on the process of choosing an alternative testing date.
Alternative number 1: One week later (teacher’ choice)
Alternative number 2: Two weeks later (student’s choice)
Surprise tests are not really fair, in that they take students off guard, putting them into a situation in which they are highly resistant to the process at hand and encourage negative attitudes to the test, subject and teacher, which promotes terrible backwash. However, in such a case as above, where the teacher is trying to gauge student’s ability in a spontaneous situation and correlate it with the students prepared-for responses, then I imagine that alternative number 1 would be the better choice to maintain reliability, but would it be invalid, as the students may feel that they need more time to prepare for the test? However, is it really possible to correlate the two tests anyway? Can the formula for working out the correlation between tests be adapted to this scenario (Taking a student’s impromptu ability and comparing it to their prepared answer, I mean)? I think that it would be quite a stretch, without some sort of criterion referencing, and matching to the course objectives.
If alternative 2 is chosen, then there is too much time between the administration of one test and that of the other ~ and the test would become unreliable due to things such as ‘Test Wiseness, ‘ among other factors. (Study book ~ module 4: 4.8)…
The teacher in the above example seems to be in a no-win situation all round. The whole situation seems to be flawed anyway. The teacher will need to consider the reason for giving this test and what information is actually retrieved, and how beneficial it is to the whole learning environment. At the very least, if the tests were to be undertaken, the teacher will need to warn students of the likelihood that this kind of testing will be given.
I wonder, in what situations these kinds of surprise tests could be used in the classroom? There could be a situation, in some speaking classes, where there is a need to know how the student reacts to a new situation, based on their working knowledge of the language. Unfortunately, when in the real world, we cannot prepare for all situations. So, why not have tests that mimic this communicative task?
Steven Mondy
Doing Letter Grades on Excel
Topic: Open for Discussion Date: 5 December 2006 1:27 AM
Subject: Doing Letter Grades on Excel Author: Mondy, Steven
If anyone is interested...
There's an easy way of doing your grades on excel.
You can assign letter grades to the results you get for students...
Check out the following website. I managed to finally do it on my computer. It works...
http://www.uwec.edu/help/Excel03/gr-5finalgrad.htm
careful with the formula...
Make the list of grades you want to give then use the formula,
=VLOOKUP(C21,grades,2,TRUE) for example,
~ the c21: is the actual student grade
~ grades: is the list
~ 2: is the column that has A, B, C in the list,
~True: rounds off the grade to the closest figure...
Try it out, it may help you (or may confuse you...)!
Steven Mondy
Subject: Doing Letter Grades on Excel Author: Mondy, Steven
If anyone is interested...
There's an easy way of doing your grades on excel.
You can assign letter grades to the results you get for students...
Check out the following website. I managed to finally do it on my computer. It works...
http://www.uwec.edu/help/Excel03/gr-5finalgrad.htm
careful with the formula...
Make the list of grades you want to give then use the formula,
=VLOOKUP(C21,grades,2,TRUE) for example,
~ the c21: is the actual student grade
~ grades: is the list
~ 2: is the column that has A, B, C in the list,
~True: rounds off the grade to the closest figure...
Try it out, it may help you (or may confuse you...)!
Steven Mondy
Pre-packaged test
Topic: Purposes and Types of Language Test Date: 7 December 2006 7:53 PM
Subject: Pre-packaged test Author: Mondy, Steven
Recently, I was interested to know the effect of a pre-made packaged test on students within our college. I have a class of Chinese students studying within a basic reading course. I’m using the Oxford Dominoes starter series, and they have downloadable book tests (Multiple-choice items) for each of the books. I initially imagined that the tests put out by the publisher had been analyzed and tested (although, here in Japan it’s somewhat a dangerous assumption to make with publishers, as there seems to be frequent mistakes within textbooks). Anyway, after correcting the tests, there seemed to be a definite pattern emerging with relation to the student’s overall results. I saw that what appeared to be a well-formed test that was directly related to the material that we were studying, produced shockingly low results from students. I tried to think about why this was so?
Looking at the test, I couldn’t see any particularly strange or misleading items. The test was divided into sections, “Setting, Characters, Dialog within the book, Vocabulary, and Plot”. All multiple-choice items had four alternatives as suggested in many of our textbooks, which had fairly straightforward distracters. I was fairly confident that it was a reliable test.
However, after getting the results back, I did start to wonder about its validity within the context of this class, and the test’s suitability for our student’s purposes for studying English. The reliability of this test might have been quite high, yet for some reason my students did not do well…
It may have something to do with why the students are actually studying this course in the first place. Chinese students come to Japan, hoping to skip from our course to some university place in Japan. While they are here, they are required to study in (1) Programs that encourage general English learning and (2) Activities that prepare them for university examinations, and other standardized tests.
The reading course above comes under the first heading, and is a required course for them.
Whether right or wrong, I made the following observations:
~ The test may have lacked Validity: in that the objectives of the school, the teacher (me) and the test may not exactly match the goals of the students. These students may not be able to see the purpose of this kind of course, and may be highly resistant to the class and the test itself, therefore invalidating any test, even before it is given.
~ Reliable and well thought-out tests may not produce the same results for different groups of students.
~ Motivation: I observed that many of the students just gave up, and finished the test early because they were either not ready for it, or didn’t have the ability to do it. Either way, they did not feel compelled to continue with the test.
~ Test format: The test that was prepared for these lower level students was solely comprised of written multiple-choice items. There were no pictures or varying techniques such as cloze or matching that would make it possible for students of differing capabilities to have more chances. Also, it was quite clear that although the length of the test did not disadvantage the students (50 items in 50 minutes, which complies with the 1 minute/1question rule), having 50 questions of the same kind did.
Having undertaken this test, and collating the results, I can see that there are some fundamental problems with validity. In the future, I will be wearier of doing tests that aren’t more closely related to the specific needs of my students. That also means I need to be careful of pre-packaged tests that on the surface look fine, but can end up costing in terms of the harmful effects. I will also be careful to provide more of a variety of techniques within a test for differing learning styles.
Steven Mondy
Subject: Pre-packaged test Author: Mondy, Steven
Recently, I was interested to know the effect of a pre-made packaged test on students within our college. I have a class of Chinese students studying within a basic reading course. I’m using the Oxford Dominoes starter series, and they have downloadable book tests (Multiple-choice items) for each of the books. I initially imagined that the tests put out by the publisher had been analyzed and tested (although, here in Japan it’s somewhat a dangerous assumption to make with publishers, as there seems to be frequent mistakes within textbooks). Anyway, after correcting the tests, there seemed to be a definite pattern emerging with relation to the student’s overall results. I saw that what appeared to be a well-formed test that was directly related to the material that we were studying, produced shockingly low results from students. I tried to think about why this was so?
Looking at the test, I couldn’t see any particularly strange or misleading items. The test was divided into sections, “Setting, Characters, Dialog within the book, Vocabulary, and Plot”. All multiple-choice items had four alternatives as suggested in many of our textbooks, which had fairly straightforward distracters. I was fairly confident that it was a reliable test.
However, after getting the results back, I did start to wonder about its validity within the context of this class, and the test’s suitability for our student’s purposes for studying English. The reliability of this test might have been quite high, yet for some reason my students did not do well…
It may have something to do with why the students are actually studying this course in the first place. Chinese students come to Japan, hoping to skip from our course to some university place in Japan. While they are here, they are required to study in (1) Programs that encourage general English learning and (2) Activities that prepare them for university examinations, and other standardized tests.
The reading course above comes under the first heading, and is a required course for them.
Whether right or wrong, I made the following observations:
~ The test may have lacked Validity: in that the objectives of the school, the teacher (me) and the test may not exactly match the goals of the students. These students may not be able to see the purpose of this kind of course, and may be highly resistant to the class and the test itself, therefore invalidating any test, even before it is given.
~ Reliable and well thought-out tests may not produce the same results for different groups of students.
~ Motivation: I observed that many of the students just gave up, and finished the test early because they were either not ready for it, or didn’t have the ability to do it. Either way, they did not feel compelled to continue with the test.
~ Test format: The test that was prepared for these lower level students was solely comprised of written multiple-choice items. There were no pictures or varying techniques such as cloze or matching that would make it possible for students of differing capabilities to have more chances. Also, it was quite clear that although the length of the test did not disadvantage the students (50 items in 50 minutes, which complies with the 1 minute/1question rule), having 50 questions of the same kind did.
Having undertaken this test, and collating the results, I can see that there are some fundamental problems with validity. In the future, I will be wearier of doing tests that aren’t more closely related to the specific needs of my students. That also means I need to be careful of pre-packaged tests that on the surface look fine, but can end up costing in terms of the harmful effects. I will also be careful to provide more of a variety of techniques within a test for differing learning styles.
Steven Mondy
Instructions
Topic: Validity and reliability - issues and discussion Date: 30 December 2006 11:43 PM
Subject: Instructions Author: Mondy, Steven
When we give a test, don't we do so after a long process of introducing the test procedures throughout our teaching? Before attempting the test, the students in many of my classes are often given a pre-test, or warm up test, so that they can become familiar with the kinds of questions and to the types of instructions that they will face.
Also, in the test, each new kind of test item is introduced with an example with answer, or at least the first question is done for them, so that they can work out what to do, if they actually know the content.
I've played with the idea of using the native language of the test takers, but have opted not to, as I feel that a properly presented test (pre-test, test, and post-test activities) can make it possible for any level student to undertake a test solely in the target language.
However, the kinds of instructions are significantly different depending on the level of the student. I find it quite difficult to write clear instructions for lower level students in particular. To resist the urge to use complicated or confusing jargon.
Another thing that I've been pondering is the use of 'please' in instructions. When writing instructions, do we say...Please circle the correct answer, or do we say circle the correct answer? Many would say that we should make our instructions as clear as possible, with little room for misinterpretation. Does that mean we should take the second option?
Steven
Subject: Instructions Author: Mondy, Steven
When we give a test, don't we do so after a long process of introducing the test procedures throughout our teaching? Before attempting the test, the students in many of my classes are often given a pre-test, or warm up test, so that they can become familiar with the kinds of questions and to the types of instructions that they will face.
Also, in the test, each new kind of test item is introduced with an example with answer, or at least the first question is done for them, so that they can work out what to do, if they actually know the content.
I've played with the idea of using the native language of the test takers, but have opted not to, as I feel that a properly presented test (pre-test, test, and post-test activities) can make it possible for any level student to undertake a test solely in the target language.
However, the kinds of instructions are significantly different depending on the level of the student. I find it quite difficult to write clear instructions for lower level students in particular. To resist the urge to use complicated or confusing jargon.
Another thing that I've been pondering is the use of 'please' in instructions. When writing instructions, do we say...Please circle the correct answer, or do we say circle the correct answer? Many would say that we should make our instructions as clear as possible, with little room for misinterpretation. Does that mean we should take the second option?
Steven
oral testing in pairs
Topic: Assignment 2 discussion and queries Date: 3 January 2007 12:16 PM
Subject: oral testing in pairs Author: Mondy, Steven
The point about the stronger student being able to perform in any situation is quite solid. I think that we as teachers can see their ability to handle a situation where the other speaker cannot answer fully (though are we now testing something different). This reflects an aspect of authenticity, in that it more closely resembles real life situations. However, as the proctor of the test, how do we alter our own perceptions to the changes and grading? This is obviously very subjective, as it is quite spontaneous, and we are unable to prepare fully for such a case. We can however, make provisions in our planning for this kind of test, by stating the course of action to take in such a case. This of course is an ideal situation, in which teachers have the time to do so. Still, there will be problems with reliability, especially proctor to proctor consistency, that will certainly need attention.
As for weaker students being paired, well from my experience, it has been rather detrimental as you have pointed out. Students tend to wind down into a state of nervousness about not being able to answer anything. Then there starts to be a problem with reliability (Bachman 1990), in that random factors, such as emotional state and changes to the test environment interfere with the successful completion of the test. Has anyone else had similiar problems?
Steven Mondy
Subject: oral testing in pairs Author: Mondy, Steven
The point about the stronger student being able to perform in any situation is quite solid. I think that we as teachers can see their ability to handle a situation where the other speaker cannot answer fully (though are we now testing something different). This reflects an aspect of authenticity, in that it more closely resembles real life situations. However, as the proctor of the test, how do we alter our own perceptions to the changes and grading? This is obviously very subjective, as it is quite spontaneous, and we are unable to prepare fully for such a case. We can however, make provisions in our planning for this kind of test, by stating the course of action to take in such a case. This of course is an ideal situation, in which teachers have the time to do so. Still, there will be problems with reliability, especially proctor to proctor consistency, that will certainly need attention.
As for weaker students being paired, well from my experience, it has been rather detrimental as you have pointed out. Students tend to wind down into a state of nervousness about not being able to answer anything. Then there starts to be a problem with reliability (Bachman 1990), in that random factors, such as emotional state and changes to the test environment interfere with the successful completion of the test. Has anyone else had similiar problems?
Steven Mondy
oral testing in pairs1
Topic: Assignment 2 discussion and queries Date: 4 January 2007 4:20 PM
Subject:oral testing in pairs Author: Mondy, Steven
I guess I'm talking about practicality here. Pairing up students and using criterion referenced type tests (both discreet and integrated) in the form of mini role-plays and distinct short conversations (both authentic and non). As for the level, I think that anything but true beginner can handle this kind of testing. As for the true beginner, then I would consider straight out student to teacher interviews (even though there may be a problem with performance anxiety in front of a native speaker in the TLU). Oral interviews for speaking type classes are essential to maintain face validity as Hughes points out on page 33. However, the true beginner may not have enough yet to work with to be able to handle true mini-conversation scenarios. Therefore, one may need to balance in favor of less authentic material to make sure validity is maintained…
Does this at all answer your question?
Steven Mondy
Subject:oral testing in pairs Author: Mondy, Steven
I guess I'm talking about practicality here. Pairing up students and using criterion referenced type tests (both discreet and integrated) in the form of mini role-plays and distinct short conversations (both authentic and non). As for the level, I think that anything but true beginner can handle this kind of testing. As for the true beginner, then I would consider straight out student to teacher interviews (even though there may be a problem with performance anxiety in front of a native speaker in the TLU). Oral interviews for speaking type classes are essential to maintain face validity as Hughes points out on page 33. However, the true beginner may not have enough yet to work with to be able to handle true mini-conversation scenarios. Therefore, one may need to balance in favor of less authentic material to make sure validity is maintained…
Does this at all answer your question?
Steven Mondy
oral testing in pairs2
Topic: Assignment 2 discussion and queries Date: 7 January 2007 11:41 AM
Subject: oral testing in pairs Author: Mondy, Steven
It is not always possible to test a wide range of skills within the objectives of your course, but only a selection. Discreet point testing will make it more likely that the skills you are looking for will more clearly be evaluated, whereas integrated tests might make it very difficult to make judgments about students fairly, as students may answer in a variety of ways.
Steven Mondy
Subject: oral testing in pairs Author: Mondy, Steven
It is not always possible to test a wide range of skills within the objectives of your course, but only a selection. Discreet point testing will make it more likely that the skills you are looking for will more clearly be evaluated, whereas integrated tests might make it very difficult to make judgments about students fairly, as students may answer in a variety of ways.
Steven Mondy
iBT TOEFL ~ reflecting authenticity...
Topic: Purposes and Types of Language Test Date: 13 January 2007 5:19 PM
Subject: iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
I was reading about an interesting development in the iBT (internet based TOEFL test) in the Daily Yomuiri, a Japan-based English language newspaper
(Lawrence J. Zwier, TOEFL Booster / Readings get longer, and tougher, Yomiurihttp://www.yomiuri.co.jp/dy/features/language/20070112TDY15001.htm).
The article talks about a change in the test that is an attempt to make the reading section more authentic, has made it more difficult for some test-takers. The most significant change is the move from what the article calls enumeration type readings (where a point is given, then supporting evidence) to a more complex form where several ideas are interlaced ~ a task requiring more understanding of various reading skills and the purpose for obtaining information. The article implies that the change in the test is to encompass, 1. Reading to find information (such things as speed reading, using thinking skills while reading), 2. Reading for comprehension (the more traditional focus), and 3. Reading to learn (being able to summarize, and then use the information).
This was very interesting to me as one of the main focuses in my college has been point number two, comprehension, then followed by point number one, the development of reading skills. Unfortunately, point number 3 has been missed in our curriculum design phase, which will make it quite difficult for our students in this new version of the iBT.
This semester, I have also discovered that this focus has been detrimental to our students, especially within short answer type questions. They are too used to multiple choice type questions, and are not sure what information to include in a written response. I have tried to change this by not only testing, but also teaching various techniques to get away from just reading for comprehension, such things as, discussion, inference making, and summarization. I believe that we need to essentially test how we teach, which means that we at the college need to adopt new curriculum ideas, to reflect changes to ideas out in the real world ~ which I suspect the new version of the iBT is attempting to do?!
Steven Mondy
Subject: iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
I was reading about an interesting development in the iBT (internet based TOEFL test) in the Daily Yomuiri, a Japan-based English language newspaper
(Lawrence J. Zwier, TOEFL Booster / Readings get longer, and tougher, Yomiurihttp://www.yomiuri.co.jp/dy/features/language/20070112TDY15001.htm).
The article talks about a change in the test that is an attempt to make the reading section more authentic, has made it more difficult for some test-takers. The most significant change is the move from what the article calls enumeration type readings (where a point is given, then supporting evidence) to a more complex form where several ideas are interlaced ~ a task requiring more understanding of various reading skills and the purpose for obtaining information. The article implies that the change in the test is to encompass, 1. Reading to find information (such things as speed reading, using thinking skills while reading), 2. Reading for comprehension (the more traditional focus), and 3. Reading to learn (being able to summarize, and then use the information).
This was very interesting to me as one of the main focuses in my college has been point number two, comprehension, then followed by point number one, the development of reading skills. Unfortunately, point number 3 has been missed in our curriculum design phase, which will make it quite difficult for our students in this new version of the iBT.
This semester, I have also discovered that this focus has been detrimental to our students, especially within short answer type questions. They are too used to multiple choice type questions, and are not sure what information to include in a written response. I have tried to change this by not only testing, but also teaching various techniques to get away from just reading for comprehension, such things as, discussion, inference making, and summarization. I believe that we need to essentially test how we teach, which means that we at the college need to adopt new curriculum ideas, to reflect changes to ideas out in the real world ~ which I suspect the new version of the iBT is attempting to do?!
Steven Mondy
iBT TOEFL ~ reflecting authenticity...2
Topic: Purposes and Types of Language Test Date: 15 January 2007 11:10 AM
Subject: Re:iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
It's good to here that others are having similar issues with the TOEFL/TOEIC...My biggest question is whether these new changes are reflective of what is needed by these students. Are we being led down the road in a chicken and egg race, that will end up in our students getting more confused? I think that all the skills that I mentioned before are purposeful and relevant to teaching practice and learning, yet I don't want a situation where it's the test that dictates what we need to focus on in our classes, unless it is truly what our particular students need to know. Isn't a test an evaluation instrument, and not a prescriptive force?
Steven Mondy
Subject: Re:iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
It's good to here that others are having similar issues with the TOEFL/TOEIC...My biggest question is whether these new changes are reflective of what is needed by these students. Are we being led down the road in a chicken and egg race, that will end up in our students getting more confused? I think that all the skills that I mentioned before are purposeful and relevant to teaching practice and learning, yet I don't want a situation where it's the test that dictates what we need to focus on in our classes, unless it is truly what our particular students need to know. Isn't a test an evaluation instrument, and not a prescriptive force?
Steven Mondy
iBT TOEFL ~ reflecting authenticity...3
Topic: Purposes and Types of Language Test Date: 20 January 2007 11:54 AM
Subject: iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
I can see the value of providing a goal to work towards and providing tasks that will not only develop their abilities to take the test, but through some kind of ‘osmosis’ process in learning, pick up some of the TL (target language). However, this defeats one of the main purposes of an evaluation tool, and it then becomes something else ~ as I said before, a tool to prescribe what some group in society thinks is needed. Isn't it dangerous, and at cross- purposes with ‘authenticity’ to do so? "They need to be able to fulfill certain requirements to succeed in a course," you may ask. Yes they do, and is that the purpose of this kind of test? What is the TOEFL and TOEIC trying to be?
1. A proficiency test (ability in language regardless of ability),
2. or achievement test (how much individuals have been achieving goals of program)
3. or diagnostic test (identify learners' strengths and weaknesses)
4. or placement test (getting info to help position an individuals in a certain place)?
(Hughes, 2005)
Which is the TOEFL or TOEIC, really? In Japan, the TOEIC seems to operate as a placement test for climbing the rungs of the corporate ladder. High score and you get the promotion! TOEFL is closer to the ideals of the proficiency test. And with the new changes is heading towards a closer connection to the real world, and authenticity.
You ask who really cares whether or not they are good speakers? Well, maybe no one does, except the individual who is trying out for the test. Many of my students are still under the illusion that a good score on either of these tests correlates with a good ability in the TL. Probably with the revised TOEFL, they may be right!??? So, do we head down the path of teaching to the test or do we maintain our determination to help these individuals with learning the target language? I think I’ll let you decided on that one for yourself. As for me, I’m still sitting on the fence…
Steven
Subject: iBT TOEFL ~ reflecting authenticity... Author: Mondy, Steven
I can see the value of providing a goal to work towards and providing tasks that will not only develop their abilities to take the test, but through some kind of ‘osmosis’ process in learning, pick up some of the TL (target language). However, this defeats one of the main purposes of an evaluation tool, and it then becomes something else ~ as I said before, a tool to prescribe what some group in society thinks is needed. Isn't it dangerous, and at cross- purposes with ‘authenticity’ to do so? "They need to be able to fulfill certain requirements to succeed in a course," you may ask. Yes they do, and is that the purpose of this kind of test? What is the TOEFL and TOEIC trying to be?
1. A proficiency test (ability in language regardless of ability),
2. or achievement test (how much individuals have been achieving goals of program)
3. or diagnostic test (identify learners' strengths and weaknesses)
4. or placement test (getting info to help position an individuals in a certain place)?
(Hughes, 2005)
Which is the TOEFL or TOEIC, really? In Japan, the TOEIC seems to operate as a placement test for climbing the rungs of the corporate ladder. High score and you get the promotion! TOEFL is closer to the ideals of the proficiency test. And with the new changes is heading towards a closer connection to the real world, and authenticity.
You ask who really cares whether or not they are good speakers? Well, maybe no one does, except the individual who is trying out for the test. Many of my students are still under the illusion that a good score on either of these tests correlates with a good ability in the TL. Probably with the revised TOEFL, they may be right!??? So, do we head down the path of teaching to the test or do we maintain our determination to help these individuals with learning the target language? I think I’ll let you decided on that one for yourself. As for me, I’m still sitting on the fence…
Steven
Niggily bits
Topic: Assignment 2 discussion and queries Date: 19 January 2007 6:18 PM
Subject: Niggily bits Author: Mondy, Steven
I am really getting into the test creation process and reshaping and reformatting of the test. I'm never satisfied...
I wonder if anyone else is really frustrated with Microsoft word, especially with making multiple choice items ~ it insists on capital letters after the a. b. c. and d.
I was also dealing with a minor issue of using 'not' in a multiple choice item, but decided to redo it on advice from the Hughes book. I was ale to rephrase it so that it wouldn't have the potential of disadvantaging my students.
Actually, it has been quite interesting formulating questions. It's so easy to introduce little things that quickly invalidate an item. In one task, I was going to label an item "Vocabulary:Synonyms" and then ask the students to look for the word that is not a Synonym. I quickly realized that by labeling it "Synonyms" I introduce into the students conscious (or unconscious minds) the expectation to look for synonyms. This in turn may cause a student to make a mistake with what is expected of them, and make the item invalid. I decided to cut the heading, and now it states Section A: Vocabulary.
It's all so easy to introduce unforeseen elements into the test.
Steven Mondy
Subject: Niggily bits Author: Mondy, Steven
I am really getting into the test creation process and reshaping and reformatting of the test. I'm never satisfied...
I wonder if anyone else is really frustrated with Microsoft word, especially with making multiple choice items ~ it insists on capital letters after the a. b. c. and d.
I was also dealing with a minor issue of using 'not' in a multiple choice item, but decided to redo it on advice from the Hughes book. I was ale to rephrase it so that it wouldn't have the potential of disadvantaging my students.
Actually, it has been quite interesting formulating questions. It's so easy to introduce little things that quickly invalidate an item. In one task, I was going to label an item "Vocabulary:Synonyms" and then ask the students to look for the word that is not a Synonym. I quickly realized that by labeling it "Synonyms" I introduce into the students conscious (or unconscious minds) the expectation to look for synonyms. This in turn may cause a student to make a mistake with what is expected of them, and make the item invalid. I decided to cut the heading, and now it states Section A: Vocabulary.
It's all so easy to introduce unforeseen elements into the test.
Steven Mondy
Maybe a stupid question...?
Topic: Assignment 2 discussion and queries Date: 19 January 2007 6:32 PM
Subject: Scoring - Maybe a stupid question...? Author: Mondy, Steven
What is the advantage of scoring a test out of 100, as opposed to dividing by the total and working out a percentage?
I was wondering about this, and can see one advantage of scoring out of 100. That is, it is clearer for the test taker to see the division of scores between the sections, and between various parts. For instance, you may divide up your test into various sections based on a certain taxonomy - such as knowledge vs. application. Also for the untrained eye, it's easier to see the weighting given to various parts, and decide how much effort will be required for each section.
Then there's the marker, who may not be the person who designed the test. They can see the breakdown very clearly after the administration of the test, how much emphasis is placed on various sections...
In spite of this, I'd like to ask everyone if it is necessary to have the total = 100?
Steve
Subject: Scoring - Maybe a stupid question...? Author: Mondy, Steven
What is the advantage of scoring a test out of 100, as opposed to dividing by the total and working out a percentage?
I was wondering about this, and can see one advantage of scoring out of 100. That is, it is clearer for the test taker to see the division of scores between the sections, and between various parts. For instance, you may divide up your test into various sections based on a certain taxonomy - such as knowledge vs. application. Also for the untrained eye, it's easier to see the weighting given to various parts, and decide how much effort will be required for each section.
Then there's the marker, who may not be the person who designed the test. They can see the breakdown very clearly after the administration of the test, how much emphasis is placed on various sections...
In spite of this, I'd like to ask everyone if it is necessary to have the total = 100?
Steve
Testing grammar
Sunday, October 21, 2007
Testing grammar
Topic: Assignment 2 discussion and queries Date: 30 January 2007 1:22 AM
Subject:Testing grammar Author: Mondy, Steven
I think that all speaking tests are quite difficult to administer. There are just so many factors to take into consideration. Our classmate has pointed out a very good point with the testing, or at the very least the acknowledgment of the test-taker’s ability to self-correct. We are often too much caught up in the timing and correctness of procedure to allow students enough time to self-correct (a skill that may become more apparent ‘quite a ways down the track’ in their lang. learning).
From my experience (in Japan), learners tend to do better on written tasks, and have great difficulty producing oral language. There have been those students who the reverse is true, but generally written tests seem to be less invasive, especially if enough time is given. As for the question about giving a written test on paper, well, I think that I read somewhere, maybe in Hughes, maybe Bachman, that we need to maintain face validity ~ testing speaking with a written test somewhat invalidates the procedure.
I would go more for some kind of rubric, that incorporates grammar into one of the sections, along with categories including the one Nico suggested, in that the ability to self-correct is a factor in the level of proficiency that the learner has achieved. Your test seems to be more discreet point testing, and for such a case, then yes, you would have to mark them down on knowing that particular grammar, as long as you have given them enough time to answer the question, and as long as that is your focus within your objectives. A more integrative test, based on a role-play would probably pose more difficulties with rater inconsistencies, but may make grammar less of a focus, and make communication the focal point.
Also remember that, the student’s score is dependent on many factors, and the interpretation of them is relative to the other people taking the same test. Cut off scores don’t remain fixed to a magical 50%.
Steven Mondy
Testing grammar
Topic: Assignment 2 discussion and queries Date: 30 January 2007 1:22 AM
Subject:Testing grammar Author: Mondy, Steven
I think that all speaking tests are quite difficult to administer. There are just so many factors to take into consideration. Our classmate has pointed out a very good point with the testing, or at the very least the acknowledgment of the test-taker’s ability to self-correct. We are often too much caught up in the timing and correctness of procedure to allow students enough time to self-correct (a skill that may become more apparent ‘quite a ways down the track’ in their lang. learning).
From my experience (in Japan), learners tend to do better on written tasks, and have great difficulty producing oral language. There have been those students who the reverse is true, but generally written tests seem to be less invasive, especially if enough time is given. As for the question about giving a written test on paper, well, I think that I read somewhere, maybe in Hughes, maybe Bachman, that we need to maintain face validity ~ testing speaking with a written test somewhat invalidates the procedure.
I would go more for some kind of rubric, that incorporates grammar into one of the sections, along with categories including the one Nico suggested, in that the ability to self-correct is a factor in the level of proficiency that the learner has achieved. Your test seems to be more discreet point testing, and for such a case, then yes, you would have to mark them down on knowing that particular grammar, as long as you have given them enough time to answer the question, and as long as that is your focus within your objectives. A more integrative test, based on a role-play would probably pose more difficulties with rater inconsistencies, but may make grammar less of a focus, and make communication the focal point.
Also remember that, the student’s score is dependent on many factors, and the interpretation of them is relative to the other people taking the same test. Cut off scores don’t remain fixed to a magical 50%.
Steven Mondy
Friday, October 19, 2007
Why does English have a need for pronouns?
13th August, 2006
Recently, I was talking with my wife about the differences between her language (Japanese) and mine (English), and she asked me, “Why does English have the need to keep on restating pronouns?” In Japanese, once the referent is established, there is not much need to keep on using the pronouns, ‘watashi’= I, ‘anata’=you, ‘kare’ =he, ‘kanajo’=she and so on... As an English speaker in my L1 and Japanese in L2, I often find that I lose track of who or what the conversation is about when having a conversation in Japanese, because of the lack of pronouns. My wife often says that the English use of pronouns is redundant, as we have already established what is being discussed (or with a little imagination you’d be able to work it out). I was reading that Japanese is a ‘pro-drop’ language (Wikipedia http://en.wikipedia.org/wiki/Japanese_language), in that it tends to omit words to make an utterance shorter, if the context is known. I think, that English also drops pronouns sometimes, however, not as much as Japanese. So, why in English do we tend to overuse pronouns?
Japanese is said to be a high context culture and English a Low context culture. That is...
High context culture: many things don’t need to be made explicit because of a close connection and understanding.
Low context: societies that need to explicitly spell out things because connections are not as close. (Culture at work: Communicating across cultures, http://www.culture-at-work.com/highlow.html)
Could this explain the need for us to continually check that the listener understands who the referent is? ...And why is necessary for English to paraphrase so often? My wife made a joke and asked, “Don’t people listen the first time?”
Another question that came up was why do some languages (like Romance languages, and even English in some cases) tend to use gender more than others? Why in English, we sometimes refer to a ship as a ‘She’? Why in Italian, Bianca (female) and Bianco (male)? How does culture influence language, and how does language influence culture?
Steven Mondy
Recently, I was talking with my wife about the differences between her language (Japanese) and mine (English), and she asked me, “Why does English have the need to keep on restating pronouns?” In Japanese, once the referent is established, there is not much need to keep on using the pronouns, ‘watashi’= I, ‘anata’=you, ‘kare’ =he, ‘kanajo’=she and so on... As an English speaker in my L1 and Japanese in L2, I often find that I lose track of who or what the conversation is about when having a conversation in Japanese, because of the lack of pronouns. My wife often says that the English use of pronouns is redundant, as we have already established what is being discussed (or with a little imagination you’d be able to work it out). I was reading that Japanese is a ‘pro-drop’ language (Wikipedia http://en.wikipedia.org/wiki/Japanese_language), in that it tends to omit words to make an utterance shorter, if the context is known. I think, that English also drops pronouns sometimes, however, not as much as Japanese. So, why in English do we tend to overuse pronouns?
Japanese is said to be a high context culture and English a Low context culture. That is...
High context culture: many things don’t need to be made explicit because of a close connection and understanding.
Low context: societies that need to explicitly spell out things because connections are not as close. (Culture at work: Communicating across cultures, http://www.culture-at-work.com/highlow.html)
Could this explain the need for us to continually check that the listener understands who the referent is? ...And why is necessary for English to paraphrase so often? My wife made a joke and asked, “Don’t people listen the first time?”
Another question that came up was why do some languages (like Romance languages, and even English in some cases) tend to use gender more than others? Why in English, we sometimes refer to a ship as a ‘She’? Why in Italian, Bianca (female) and Bianco (male)? How does culture influence language, and how does language influence culture?
Steven Mondy
Sapir-whorf
10th August, 2006
What are we to understand from the Sapir-Whorf debate on how a language affects culture, and in turn the individual? I find it quite an interesting discussion, as both sides to the debate give intriguing alternatives...
According to end of the lecture on ‘primitive languages’ (from linguistic voices on campus podcast) the linguist makes a point at the end that a weak version of the Sapir-Whorf hypothesis makes some sense in terms of sexist language (the so called generic ‘he’) as somewhat a warning to be careful of the language we use, as it may influence the way we think. Using ‘he’ or not using ‘he’ can determine how we think about something. When we choose to teach (or not) something in a particular way are we influencing the way our students think, by the language we use? Is Political Correctness a form of ‘linguistic determinism’? How about the movement against euphemisms, that some consider ‘bad language’? How about the elimination of racist terminology in songs (Eminem)? Can we shape society by shaping the language it uses? This taken to the extreme is the use of the so-called ‘Newspeak’ in George Orwell's classic novel Nineteen Eighty-Four, to shape society (Wikipedia -http://en.wikipedia.org/wiki/Sapir-Whorf_hypothesis).
Also the Sapir-Whorf hypothesis assumes that people are limited by the language they speak and they cannot think outside ‘the proverbial box’. In that case, how can anyone be creative?
Does language have an impact on the way we think?
What are we to understand from the Sapir-Whorf debate on how a language affects culture, and in turn the individual? I find it quite an interesting discussion, as both sides to the debate give intriguing alternatives...
According to end of the lecture on ‘primitive languages’ (from linguistic voices on campus podcast) the linguist makes a point at the end that a weak version of the Sapir-Whorf hypothesis makes some sense in terms of sexist language (the so called generic ‘he’) as somewhat a warning to be careful of the language we use, as it may influence the way we think. Using ‘he’ or not using ‘he’ can determine how we think about something. When we choose to teach (or not) something in a particular way are we influencing the way our students think, by the language we use? Is Political Correctness a form of ‘linguistic determinism’? How about the movement against euphemisms, that some consider ‘bad language’? How about the elimination of racist terminology in songs (Eminem)? Can we shape society by shaping the language it uses? This taken to the extreme is the use of the so-called ‘Newspeak’ in George Orwell's classic novel Nineteen Eighty-Four, to shape society (Wikipedia -http://en.wikipedia.org/wiki/Sapir-Whorf_hypothesis).
Also the Sapir-Whorf hypothesis assumes that people are limited by the language they speak and they cannot think outside ‘the proverbial box’. In that case, how can anyone be creative?
Does language have an impact on the way we think?
In what language do deaf people think?
10th August, 2006
In what language do deaf people think? 26 Dec-2003
(http://www.straightdope.com/columns/031226.html)
I have been thinking about this topic recently, as an extension to the ‘wild child’ idea. If a child is brought up in a situation devoid of language what do they think? The article puts forward the argument that deaf people think in signs, if they were lucky enough to learn sign language at an early age. However, what if they didn’t? Do these children develop their own way to interpret and react to the world? For this would support the innatist view? Or do these children not develop any form of device to react to the world? Possibly, their brains just don’t develop the necessary neurons, and they are forced to live in some kind of vegetative state. Having these two options, I would like to think that the human brain is predisposed to make sense of the world, in whatever way.... Steven Mondy
In what language do deaf people think? 26 Dec-2003
(http://www.straightdope.com/columns/031226.html)
I have been thinking about this topic recently, as an extension to the ‘wild child’ idea. If a child is brought up in a situation devoid of language what do they think? The article puts forward the argument that deaf people think in signs, if they were lucky enough to learn sign language at an early age. However, what if they didn’t? Do these children develop their own way to interpret and react to the world? For this would support the innatist view? Or do these children not develop any form of device to react to the world? Possibly, their brains just don’t develop the necessary neurons, and they are forced to live in some kind of vegetative state. Having these two options, I would like to think that the human brain is predisposed to make sense of the world, in whatever way.... Steven Mondy
Subscribe to:
Posts (Atom)