Journal Article Review and Critique

Carr-Chellman, Alison A., Hoadley, Christopher M., Kirby, Joshua A. (2005) Instructional Systems Design and the Learning Sciences: A Citation Analysis. Education, Technology, Research, and Design, Vol. 53, No. 1, 37 – 48.

In recent years, a study has emerged that blends educational technology seamlessly into a scientific process and relates learning sciences to instructional design systems. Learning sciences is relatively new, it was established in a conference in the early nineties, since its creation, it has been compared to Instructional Technology. The two fields have significant similarities and differences, however there has been some argument over the past years as to just how similar or different these two fields of study are. The article, “Instructional Systems Design and the Learning Sciences: A citation Analysis”, by Kirby, Hoadley, and Carr-Chellman attempts to compare the two fields of study through a methodical measurement of similar authors and journals.

Instructional systems design rapidly grew in the midst of World War II, based on a need to train and educate the military needs. In the recent decades it has permeated throughout scholastics and organizations. Learning sciences developed out of a need to understand how human beings learn and process information; the field experts were mostly concerned with the cognitive aspects of learning. The two fields of study seem to overlap at times, as if one was the subset of the other, however each has a unique aspect and should be treated independently of the other.

The premise of the study was to determine if experts in each field use similar sources of information, therefore a citation analysis is used to produce links with sources from different journal articles. Citations from various journal articles, within the two fields, were compared. The main publications that were gathered in the field of instructional systems design included two publications by the AECT, Techtrends and Educational Technology Research and Development; another journal, Educational Technology was used as well; articles and citations were pulled from these two journals. Research in Instructional Systems Design has been around since the sixties. However, learning sciences is a fairly new field that began out of a conference that took place in the early nineties, and since then it has been an established field of study with a small amount of scholarly research and periodicals. For the study, three main journals about learning sciences was used, the first conference regarding learning sciences was held at the Northwestern University in 1991, the Journal of the Learning Sciences, and Computer Supported Collaborative Learning.

A citation analysis can be used for many purposes, including measuring the relevance of a journal article as it relates to a particular field of study, showing a correlation between two fields to study, and can be used to examine how data is collect and analyzed. In the citation analysis performed in this study, a total of two thousand and ninety articles were gathered, which produced a total of 2608 independent authors.

In order to calculate the difference between the references, a random sample of ten percent of the publications were selected and the mean average was produced. To determine the cross-field references all of the two thousand and ninety articles were examined. The total number of citations that were examined was estimated at forty four thousand. I felt that the results of the citations were surprising.

Out of all 2,608 individual authors, only sixty-six had published articles in both learning sciences and instructional systems design; thirteen authors published in both peer-reviewed articles in both fields. The data collected raises certain questions about the relationship, if any between the two fields. It is plausible to predict that there would have been a greater number of cross-field authors (given the known similarities between the two fields). However, the data presented is representative of the idea that the two fields are separate and distinct.

When reviewing the citations analysis several questions are raised that the study does not answer but does address post the analysis. The first concern is the awareness of the two fields of study. I believe, at first look, this might not be deduced from the study, however after further analysis and review of the data one can surmise that there exists an overwhelming sense of unawareness between the two fields. This is the logical conclusion one can draw, because of the dearth of cross-field authors. Nevertheless, according to the data collected and the low citation score a concrete determination of the awareness or unawareness of the authors cannot be made. Thus, another question is raised; how accurate is a citation analysis and what can really be learned from the analysis.

There are several inherent risks in utilizing a citations analysis in a study to determine how fields are related. A citations analysis requires a massive amount of front-end work gathering resources and logging references. Not only does one have to be an expert in one field, but also they need to immerse themselves into another field of study. Furthermore, the citations analysis cannot accurately determine how closely related two fields are, because the amount of authorship within a given field is massive. Therefore, the sample needs to be the correct size in proportion to the field of study.

The data collected in this study identifies a difference in the two fields, however it does not take into account each fields strength and weakness; the study simply looks at similar citations. Learning sciences and instructional systems might not have many similarities in teaching and researching, but both fields have the same end goal, to improve education and instruction; each field has its own distinct means to achieve this goal.