The Path in Behind and the Challenges in Front: A Methodological Synthesis of Iranian L2 Papers

نوع مقاله : مقاله تحقیق

نویسندگان
Akbar A. Jahanbakhsh 1
Parviz Ajideh 2
1 English Department, Faculty of Persian Literature and Foreign Languages, University of Tabriz, Tabriz, Iran
2 Full professor, English Department, of Persian Literature and Foreign Languages, University of Tabriz, Tabriz, Iran
10.52547/LRR.13.3.15
چکیده
The present study is a methodological synthesis aiming to evaluate the adherence of Iranian L2 papers to the study quality standards. Ten Iranian journals were selected based on the latest ranking of Iran’s Ministry of Science, Research, and Technology (MSRT), and all experimental papers (N = 367) published from their beginnings were explored for study quality with regards to sampling, design, statistical tests, reporting practices and data sharing, and visual presentation. In the evaluation of the papers, the protocols proposed by Gass and Plonsky (2011) and Pagout and Plonsky (2017) were moderated and some recent issues proposed by APA’s (2018) Journal Article Reporting Standards and some scholars (e.g., Hu & Plonsky, 2019; Khany & Tazik, 2019; Larson-Hall, 2017) were added. The results showed that while there were issues, like acceptable sample size, use of pre-testing, reporting descriptive and inferential statistics, and ensuring the reliability of instruments, that were acceptably adhered to quality standards, problematic areas existed in all five facets of quality, and the majority of them stayed constant or changed slightly over time. The shortcomings caused by such lack of adherence are discussed to identify the challenges in the way of improving the papers’ quality. Although the results are obtained from a specific context, the implications may be generalizable to other contexts where English is taught and researched as a foreign language.

کلیدواژه‌ها

موضوعات

Ajideh, P., Zohrabi, M., & Jahanbakhsh, A. A. (in press). Study quality in quantitative L2 studies: A path Analysis on the perceptions of Iranian published authors. Journal of Modern Research in English Language Studies. http://dx.doi.org/10.30479/jmrels.2020.14296.1760.
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 3-25. http://dx.doi.org/10.1037/amp0000191.
Breckler, S. (2009). Dealing with data. American Psychological Association: Science Directions. http://www.apa.org/monitor/2009/02/sd.aspx
Chaudron, C. (2001). Progress in language classroom research: Evidence from the Modern Language Journal, 1916-2000. Modern Language Journal, 85, 57–76. https://doi.org/10.1111/0026-7902.00097.
Cooper, H. & Hedges, L. V. (2009). Research synthesis as a scientific process. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis (2nd ed., pp. 147–158). Russell Sage Foundation.
Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. Routledge.
DeKeyser, R., Alfi-Shabtay, I., & Ravid, D. (2010). Cross-linguistic evidence for the nature of age effects in second language acquisition. Applied Psycholinguistics, 31(3), 413 - 438. https://doi.org/10.1017/S0142716410000056.
Derakhshan, A., Shakki, F. (2021). A meta-analytic study of instructed second language pragmatics: A case of the speech act of request. Journal of Research in Applied Linguistics, 12(1), 15-32. https://doi.org/10.22055/rals.2021.16722.
Gass, S. (2009). A survey of SLA research. In W. Ritchie & T. Bhatia (Eds.), Handbook of second language acquisition (pp. 3–28). Emerald.
Gass, S., Loewen, S., & Plonsky, L. (2020). Coming of age: The past, present, and future of quantitative SLA research. Language Teaching, 1-14. https://doi.org/10.1017/S0261444819000430.
Hu, Y., & Plonsky, L. (2019). Statistical assumptions in L2 research: A systematic review. Second Language Research. https://doi.org/10.1177/0267658319877433.
Hudson, T., & Llosa, L. (2015). Design issues and inference in experimental L2 research. Language Learning, 65(Suppl. 1), 76–96. https://doi.org/10.1111/lang.12113.
Khany, R., & Tazik, K. (2019). Levels of statistical use in applied linguistics research articles: From 1986-2015. Journal of Quantitative Linguistics, 26(1), 48–65. https://doi.org/10.1080/09296174.2017.1421498.
Larson-Hall, J. (2012). Our statistical intuitions may be misleading us: Why we need robust statistics. Language Teaching, 45(4), 460-474. https://doi.org/10.1017/S0261444811000127.
Larson–Hall, J. (2017), Moving Beyond the Bar Plot and the Line Graph to Create Informative and Attractive Graphics. The Modern Language Journal, 101, 244-270. https://doi.org/10.1111/modl.12386.
Larson-Hall, J., & Plonsky, L. (2015). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65(Suppl. 1), 127–159. https://doi.org/10.1111/lang.12115.
Norouzian, R., & Plonsky, L. (2018). Eta- and partial eta-squared in L2 research: A cautionary review and guide to more appropriate usage. Second Language Research, 34(2), 257–271. https://doi.org/10.1177/0267658316684904.
Norris, J. M. (2015). Statistical Significance Testing in Second Language Research: Basic Problems and Suggestions for Reform. Language Learning, 65(Suppl. 1), 97-126. https://doi.org/10.1111/lang.12114.
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning 50(3), 417–528. https://doi.org/10.1111/0023-8333.00136.
Norris, J. M., Plonsky, L., Ross, S. J., & Schoonen, R. (2015). Guidelines for reporting quantitative methods and results in primary research. Language Learning, 65(2), 470-476. https://doi.org/10.1111/lang.12104.
Ortega, L. (2005). Methodology, epistemology, and ethics in instructed SLA research: An introduction. Modern Language Journal, 89(3), 317 - 327. https://doi.org/10.1111/j.1540-4781.2005.00307.x.
Ortega, L. (2009). Understanding second language acquisition. London: Hodder Education.
Paquot, M., & Plonsky, L. (2017). Quantitative research methods and study quality in learner corpus research. International Journal of Learner Corpus Research, 3(1), 61–94. https://doi.org/10.1075/ijlcr3103paq.
Plonsky, L. (2011). Study quality in SLA: A commulative and developmental assessment of designs, analyses, reporting practices and outcomes in quantitative L2 research. [Doctoral dissertation, Michigan State University]. MSU Libraries. https://d.lib.msu.edu/etd/1417/datastream/OBJ/download/Study_quality_in_SLA___a_cumulative_and_developmental_assessment_of_designs__analyses__reporting_practices__and_outcomes_in_quantitative_L2_research.pdf.
Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655–687. https://doi.org/10.1017/S0272263113000399.
Plonsky, L. (2014a). Study quality in quantitative L2 research (1990-2010): A methodological synthesis and call for reform. Modern Language Journal, 98(1), 450–470. https://doi.org/10.1111/j.1540-4781.2014.12058.x.
Plonsky, L. (2014b, February). Sampling, power, and generalizability in L2 research (Or, why we might as well be flipping coins). Keynote presentation at the Second Language Studies Symposium, East Lansing, MI.
Plonsky, L. (2015). Statistical power, p values, descriptive statistics, and effect sizes: A “back-to-basics” approach to advancing quantitative methods in L2 research. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 23-45). Routledge.
Plonsky, L., & Gass, S. (2011). Quantitative research methods, study quality, and outcomes: The case of interaction research. Language Learning, 61(2), 325– 366. https://doi.org/10.1111/j.1467-9922.2011.00640.x.
Plonsky, L., & Gonulal, T. (2015). Methodological synthesis in quantitative L2 research: A review of reviews and a case study of exploratory factor analysis. Language Learning, 65(Suppl. 1), 9-36. https://doi.org/10.1111/lang.12111.
Plonsky, L., & Oswald, F. L. (2015). Meta-analyzing second language research. In L. Plonsky (ed.), Advancing Quantitative Methods in Second Language Research (pp. 106-128). Routledge.
Plonsky, L., Egbert, J., & Laflair, G. T. (2015). Bootstrapping in applied linguistics: Assessing its potential using shared data. Applied Linguistics, 36(5), 591-610. https://doi.org/10.1093/applin/amu001.
Rosenthal, R (1979). File drawer problem and tolerance for null results. Psychol Bull. 86, 638-41. https://doi.org/10.137/0033-2909.86.3.638.
Rosenthal, R. and DiMatteo, M.R. (2001) Meta-Analysis: Recent developments in quantitative methods for literature reviews. Annual Review of Psychology, 52, 59-82. https://doi.org/10.1146/annurev.psych.52.1.59
Wilcox, R. (2005). Introduction to robust estimation and hypothesis testing. Elsevier Academic.
Wilkinson, L., & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604. https://doi.org/10.1037/0003-066X.54.8.594.
Yousefi, M., & Nassaji, H. (2019). A meta-analysis of the effect of instructions and corrective feedback on L2 pragmatics and the role of moderator variables: Face-to face vs. computer-mediated instruction. ITL - International Journal of Applied Linguistics, 170(2), 277-308. https://doi.org/10.1075/itl.19012.you.
Ziegler, N. (2013). Synchronous computer-mediated communication and interaction: A research synthesis and meta-analysis. [Unpublished doctoral dissertation, Georgetown University]. DigitalGeorgetown, https://repository.library.georgetown.edu/bitstream/handle/10822/559497/Ziegler_georgetown_0076D_12341.pdf;sequence=1.
جستارهای زبانی
دوره 13، شماره 3 - شماره پیاپی 69
مرداد و شهریور 1401
صفحه 375-401