Promoting Evidence-Based Practice and Information Literacy through an Undergraduate Nursing Journal Club

In order to strengthen the evidence-based practice and information literacy curriculum in the Bachelor of Science in nursing (BSN) program, a nursing instructor and reference librarian collaborated to pilot an undergraduate nursing journal club as part of a nursing research course. Preand post-test surveys were administered to a cohort of pre-licensure students in their third year of study, who were enrolled in the course. Non-generalizable survey results showed that after students participated in several journal club sessions, they were more confident in their ability to locate and critique research. However, the students’ reading behaviors, workforce preparedness, and familiarity with research related to their clinical areas did not seem to improve at the conclusion of the activity. Additional research is necessary to determine what effect journal clubs have on student learning in the undergraduate nursing classroom.


Brief Review of the Role of the Librarian in Undergraduate Nursing Education
Information literacy has long been acknowledged by both librarians and nursing educators as an essential aspect of evidence-based practice (EBP) in nursing (Adams, 2014;Barnard & Nash, 2005;Forster, 2014;Glasper, 2010;Hopkins, 2011;Klem & Weiss, 2005;Miller, 2010;Phelps, 2013;Shorten & Crookes, 2001).Nurses must be able to access, understand, critically appraise, and consider the value of empirical research in order to implement it in clinical settings.
However, nurses experience significant barriers to EBP, and recent graduates in particular may not be competent in interpreting research and transferring literature to their clinical practice (Wahoush & Banfield, 2014).
In a literature review, Solomons and Spross (2010) identified several barriers to evidence-based practice among nurses including a lack of administrative and organizational support, a lack of information access, and a lack of confidence in locating and evaluating research.In order for nurses to improve patient care and to make decisions in an evidence-based manner, nursing students need more formal educational opportunities designed to strengthen their information literacy skills and to improve their confidence in applying evidence-based decision making in clinical practice.Barnard and Nash (2005) recognized that in order to effectively adopt evidence-based practice on the job, nurses need to further develop their skills in accessing, understanding, evaluating, and synthesizing health research and information.To do this, Barnard and Nash (2005) recommend that nursing educators collaborate with librarians to identify meaningful ways to integrate information literacy into the undergraduate nursing curriculum.
In 2013 in response to nurses' information literacy needs, the Health Sciences Interest Group of the Association of College and Research Libraries (ACRL) created the Information Literacy Competency Standards for Nursing (American Library Association, 2013).The standards are evidence-based and were designed to inform information literacy instruction at all levels in higher education and beyond.Because librarians are knowledgeable about issues related to the gathering, evaluation, organization, and use of information, they are uniquely positioned to help address these needs.Therefore, as long as the adoption of evidence-based practice is a goal for undergraduate nursing education, collaboration between academic librarians and nursing faculty should be a priority.After becoming more informed about the connections between information literacy and EBP in nursing, the authors focused on a program well-established in the medical and nursing fields: the journal club.

Pedagogical Justification for Journal Clubs
A journal club consists of a group of professionals, typically in health-related fields, who meet to discuss and critique research articles (Linzer, 1987).Journal clubs are frequently implemented as a way to help licensed nurses stay current with evidence-based practice, and to understand the decision-making process of applying new research findings to the clinical setting.The link between research and clinical practice occurs at the bedside with direct patient care, and making these connections is an expectation of clinical nursing practice.Critical appraisal of current research is a vital competence for nurses and is regarded by medical professionals as the standard for healthcare (Chang et al., 2013).Despite this, journal clubs are widely underutilized in formal nursing education (Thompson, 2006).
The benefits of journal clubs in both professional and educational settings are extensively documented as a way to raise awareness of research and evidence-based practice, develop professional reading habits and appraising skills, improve critical thinking, and promote a shared knowledge base via interdisciplinary collaboration (Edwards et al., 2001;Goodfellow, 2004;Hunt & Topham, 2002;Thompson, 2006;Valente, 2003).Journal clubs are more likely to be ineffective when articles are chosen randomly, with no clear purpose identified other than to summarize findings (Pitner, Fox, & Riess, 2013).Therefore, when developing a structure for journal clubs, objectives must be identified that assist nurses to seek answers to their clinical questions using critical appraisal skills.A positive correlation exists between the success and longevity of a journal club program and mandatory attendance policies (Pitner et al., 2013).

Pennsylvania Libraries: Research & Practice
Promoting Evidence-Based Practice and Information Literacy through an Undergraduate Nursing Journal Club

palrap.org
Introducing undergraduate nursing students to the expectation of staying current with evidence-based practice is a key reason to implement journal clubs, but the potential for many other positive outcomes exists in formal education.This is a hands-on way to engage learners in literature appraisal skills and to facilitate clinical problem solving (Steenbeek et al., 2009).In addition to using research to identify and solve clinical problems, journal clubs are an effective way to encourage dialogue and promote interaction between nursing students and instructors (Kleinpell, 2002).Moreover, nursing students are frequently tasked with understanding complex and at times abstract concepts that frequently occur in the healthcare setting.Peer instruction, which involves active engagement during discussion with peers of varied knowledge levels, allows students to talk out their thoughts on these concepts, thus resulting in a greater understanding of the topic being discussed (Smith et al., 2009).A barrier to bridging the gap between research and practice that is frequently cited by nurses is uncertainty on how to critique research and difficulty with interpretation of results (Kleinpell, 2002).The peer engagement that occurs during journal clubs is a form of active learning that may effectively address this barrier in the pre-licensure setting.

Nursing Journal Club Pilot
After identifying the connections between information literacy and EBP, the librarian and nursing professor reached out to the student nurse organization on campus and administered an informal survey to the members about where journal clubs would be of greatest benefit in the curriculum.Students and faculty jointly recommended that the journal club would be an ideal fit for the research course in the BSN program.The research course is designed to introduce students to the principles and methods of nursing research and the application of research evidence to practice.The class consists of a cohort of pre-licensure students in their third year of study.
During the course of one semester, we incorporated the journal club into the curriculum as an assigned activity held outside of regularly scheduled class times.The students were graded based on participation; if they attended, showed evidence of preparation, and made a meaningful contribution to the discussion, they earned full points.The grading rubric can be viewed as Appendix A. The emphasis of participation as the primary way students earned full credit was important in order to create a relaxed, low-stakes environment, where students felt free to share their thoughts with their peers.Although students were assessed primarily on participation, the students received immediate feedback at each meeting from the librarian.Because critical appraisal is an essential aspect of the course, students also received feedback from the nursing professor on the development of their critical appraisal skills through other course assessments.
Eight journal club meetings were held in a student lounge, located in the library.Students in the class (n=32) were permitted to select their own groups of four peers, for a total of eight groups.Each group selected a peer-reviewed, empirical research study on a topic of their choice.The student groups served as discussion leaders for their chosen article, and other students from the class were expected to read the article and come prepared to discuss it.In addition to facilitating a journal club with their group, each student was expected to attend two additional journal club meetings as a participant.Journal club meetings were limited to eight student participants and four facilitators.In total, each student in the course participated in three journal club meetings to receive full credit on the assignment.With the discussion leaders, the librarian served as a facilitator for all journal club meetings.To ensure that the meetings were convenient for students to attend, they were scheduled based on the students' and the librarian's availability.A total of approximately six hours for both preparation and participation were required for the activity.Because NURS 200W is a 3-credit course, the instructor determined that the preparation and participation time requirements for the activity were in line with the typical course load.Ultimately, the meetings were designed not to be a burden, but to function as additional opportunities for students to clarify concepts and apply critical appraisal techniques learned in class.

Pennsylvania Libraries: Research & Practice
Promoting Evidence-Based Practice and Information Literacy through an Undergraduate Nursing Journal Club palrap.org

ACRL's Framework in the Journal Club Model
Viewing the journal club as an effective method for fostering information literacy and critical appraisal skill development is strengthened when comparing the journal club discussion questions to the Association of College and Research Libraries' Framework for Information Literacy for Higher Education.Most of the discussion questions can be directly linked to several ACRL frames.For example, most questions asked in journal clubs are ones that relate to the frame, "Information Creation as Process," because they ask readers to thoughtfully read, critique, and identify specific components of the author's empirical research process (ACRL, 2015).Specifically, the question, "How do the results relate to current nursing practice and how might they impact future practice?"emphasizes "Information has Value," because it prompts students to consider how research might impact the patient care process and in what contexts that information should be applied to make changes in a clinical setting.
Other questions encourage students to think about the authors' academic authority; the peer-review and publication process; and the collaborative nature of scholarship in academia.Reviewing each study requires students to identify and describe the author's inquiry process, ideally serving as a model for students who may conduct research in the future.By participating in the discussion itself, the journal club calls upon students to implement several "Scholarship as Conversation" knowledge practices such as, "contribute to scholarly conversation," "critically evaluate contributions made by others in participatory information environments," "identify the contribution that particular articles, books, and other scholarly pieces make to disciplinary knowledge," and ideally, "recognize that a given scholarly work may not represent the only or even the majority perspective on the issue" (ACRL, 2015).By incorporating active learning techniques such as questioning and group discussion, the journal club model provides students with a rich learning experience that includes repeated opportunities to further develop a variety of the ACRL Framework's knowledge practices and dispositions.
Adapted from the American Journal of Critical Care's "Guidelines for Critiquing Research," we developed a list of discussion questions that are common to many journal clubs, but which also included several questions to address the objectives of the undergraduate course (American Journal of Critical Care, 2002).Throughout each journal club session, we discussed the following questions: 1. What is the study's objective/hypothesis/question? 2. What is the methodology of this article?(qualitative/quantitative, as well as descriptive, correlational, experimental, etc.) 3. What were the results of the study?4. Are the results reliable/valid? 5. What were the strengths/weaknesses of the study?6.How do the results relate to current nursing practice and how might they impact future practice?7. Was this article peer-reviewed?How did you determine whether or not this article was peer-reviewed?

Survey
We administered the survey twice to the class of 32 students enrolled in the nursing research methods course: once at the beginning of the semester (pre-journal club) and once at the end of the semester (post-journal club).The students were informed that no personally identifiable information would be collected and that completing the survey was voluntary and would not impact their grade in the course.Twenty-eight students returned the pre-journal club survey and 30 students returned the post-journal club survey, yielding a high response rate range between 88% and 94%.We administered a 14-item Likert-style pre-test survey and an 18-item post-test survey to all students enrolled in the course.Several survey questions were adapted from the "Nursing Evidence-Based Practice Survey" developed by the Department of Nursing Services and Patient Care at the University of Iowa Hospitals and Clinics (Titler, Hill, Matthews, & Reed, 1999).The survey included questions designed to capture students' confidence in their ability to critically appraise research; their self-reported professional reading habits; their attitudes toward EBP; and their attitudes toward participating in a journal club.In the survey administered at the end of the semester, several questions

Results
At the beginning of the semester and before the journal club, the students were initially confident about their ability to critically appraise research, their ability to locate evidence for EBP, their readiness to enter the workforce, and their familiarity with research related to their clinical area (Table 1).For example, at the beginning of the semester, despite more than half (61%) of students reporting that they "rarely" or "never" read nursing literature in their spare time, 65% of students reported that they agreed or strongly agreed with the statement, "I can read a nursing research report and make a sound judgement about its scientific merit."Similarly, at the beginning of the semester 97% of students agreed or strongly agreed that they knew how to locate evidence to guide their clinical practice; 85% of students reported that they agreed or strongly agreed that they were familiar with nursing research related to their clinical area; and 79% of the students agreed that they felt prepared to enter the workforce as a nurse.Even with these initially high perceptions, post-journal club, the students still reported noticeable positive changes in their responses.
At the end of the semester, students appeared to value reading and discussing research as learning techniques more than at the beginning of the semester.86% of respondents agreed or strongly agreed that reading and discussing research helps improve critical appraisal skills in the first survey, and at the end of the semester, 100% of those who responded agreed or strongly agreed with this statement, reflecting a change of 14 percentage points.Student responses to the statement, "I can read a nursing research report and have a general notion about its strengths and weaknesses" showed one of the most noticeable changes from the first survey to the second.Sixty-one percent of students initially agreed or strongly agreed with the statement, and at the end of the semester, 100% of respondents agreed or strongly agreed, reflecting a change of 39 percentage points.Similarly, in the first survey, 65% of students agreed or strongly agreed that they could read a nursing research report and make a sound judgement about its scientific merit, and at the end of the semester 90% of students agreed or strongly agreed with the statement.When comparing the two Pennsylvania Libraries: Research & Practice Promoting Evidence-Based Practice and Information Literacy through an Undergraduate Nursing Journal Club palrap.orgsurveys, the most substantial difference was in response to the statement, "I know where to find evidence (e.g. research findings or evidence-based clinical guidelines) to guide my practice."While the majority of students (61%) initially agreed with this statement, at the end of the semester the percentage of students who strongly agreed with this statement jumped from 36% to 80%, reflecting a difference of 44 percentage points.
Despite some noticeable positive changes in student responses between the two surveys, the students' selfreported reading behaviors, workforce preparedness, familiarity with research in their clinical areas, and likelihood that they would participate in a journal club in the future did not seem to improve by the end of the semester (Table 1).For example, the percentages of students who responded that they "sometimes" or "very often" read nursing research or nursing literature in their spare time only changed between 1 and 4 percentage points between the first survey and the second.In both surveys, zero students reported that they "always" read this content.At the end of the semester, the quantity of students who agreed or strongly agreed that they felt prepared to enter the workforce as a nurse decreased slightly by 2 and 4 percentage points respectively.In the second survey, students who reported that they were undecided or disagreed with the statement about workforce preparedness increased, but only by 3 percentage points.Also, the students seemed to become less sure of their familiarity with nursing research related to their specialty or clinical setting.At the beginning of the semester, 11% of students were undecided about how familiar they were with research in their area, and by the end of the semester, 33% of students reported that they were undecided.Finally, students who reported that it was "very likely" that they would participate in a journal club in the future decreased by 16 percentage points, and those who responded, "somewhat likely," increased by 13 percentage points.Generally, by the end of the semester, the students became less interested in participating in journal clubs in the future.
It should be noted that because this was a self-report study, the students' responses may not accurately reflect their actual critical appraisal abilities and behaviors.Table 1 further details the percentages of student responses on statements relating to their workforce preparedness, professional reading behaviors, EBP beliefs, and critical appraisal self-efficacy.Because this journal club activity was a pilot project, several statements were edited for clarity between the first and second surveys.As a result, "N/A" is listed in fields in Table 1 when participants were not asked to respond to a particular statement.

Discussion
One of the most surprising results of this study was that the cohort of students rated their critical appraisal skills, evidence-gathering abilities, and familiarity with research highly at the beginning of the nursing research methods course.This finding contradicts Solomons and Spross' (2010) claim that a lack of confidence in locating and evaluating research is an EBP barrier for nurses, however, this barrier may be attributed more to nurses in the field instead of undergraduate students.According to their instructor, the students had very limited exposure to nursing research in their previous courses, which suggests that the students may have overestimated their skills, abilities, and familiarity with the subject.When considering this finding and the fact that students reported being less familiar with research in their clinical area at the end of the semester than at the beginning, their initial confidence might be explained by the Dunning-Kruger effect in psychology.The Dunning-Kruger effect describes the phenomenon that unskilled individuals tend to overestimate their actual skills and performance, and because they are unskilled, this prevents them from being aware of their own inaccuracies (Kruger & Dunning, 1999).Because the students reported being less familiar with research related to their clinical area at the end of the semester, the decrease in familiarity may actually be evidence of learning.In order to determine whether or not undergraduate nursing students overestimate their critical appraisal skills, researchers should directly and empirically assess these skills and compare the scores with student perceptions.
Because the students perceived the activities of the journal club (reading and discussing research) as useful learning techniques and because the students reported that their critical appraisal abilities improved by the end of the semester, these findings support the notion that the journal club is a relevant teaching method for nursing students at the undergraduate level.However, our findings did not show evidence of the effectiveness of journal clubs in promoting undergraduate nursing students' professional reading habits, increasing the likelihood of participating in a journal club in the future, or encouraging a sense of workforce preparedness.Because students' participation in the journal club did not positively impact their extracurricular, professional reading habits and because the students became less interested in participating in the journal club, these findings support Alguire's (1998) assertion that the journal club may not be successful as a voluntary activity and may be more successful as a required component of the course.
When considering the results of this case study and the effectiveness of the journal club as a teaching method, it is important to take into account that this project is a single case study, not a two-group experimental design.Based on our results, it is not possible to identify what effect the journal club had on the students, because they were simultaneously enrolled in the nursing research methods course.The course is designed to strengthen students' awareness of research generally and specific to nursing, therefore, the students' critical appraisal and research skills should have improved by the end of the semester even if they had not participated in the journal club.

Reflection
Although the student groups were asked to lead informal discussions about the articles, they tended to meet ahead of time, prepare answers together, and report those answers back to the people attending the journal club meeting.This behavior demonstrates that the students took the journal club seriously and came to class well-prepared, however, the librarian sometimes had to intervene and redirect scholarly conversation.Discussion monopolization by the leaders was an unanticipated consequence of mandatory participation.Without the librarian intervening, some students seemed to experience difficulty joining in the conversation.The discussion leaders likely prepared in this manner because the activity was a graded assignment.In future iterations, the nursing professor and librarian plan to regularly emphasize to the leader groups that their role is to facilitate discussion among their peers.Providing discussion leaders with information or training about how to facilitate group discussions may also be necessary.

Pennsylvania Libraries: Research & Practice
Promoting Evidence-Based Practice and Information Literacy through an Undergraduate Nursing Journal Club

palrap.org
At the conclusion of the journal club and in response to the open-ended questions, multiple students provided feedback that some of the discussion questions were redundant.Although questions vary based on the objectives of particular journal clubs, most questions one would consider when examining a research study would align with the questions posed in this pilot.The students' frustration with the discussion questions could be due to their low selfreported professional reading habits, lack of nursing work experience to compare the research to, or lack of interest in the article topic itself.In order to make the journal club increasingly relatable and valuable for students, instructors and librarians may need to add additional constraints, such as a list of possible topics, to the articles selected for discussion.Successful journal clubs read articles directly tied to their clinical setting (Deenadayalan et al., 2008).To maintain relevance of the activity for students, it may be necessary to assign articles or limit articles closely related to students' clinical assignments.
Another approach that might alleviate this problem would be for the instructor and librarian to choose articles that feature unusual topics or have more surprising results.However, one of the benefits of allowing students to choose the articles was that it gave them an opportunity to practice searching for peer-reviewed research articles using proprietary databases on topics they enjoy learning about.Allowing article choice promotes the objective of using critical appraisal to delve into topics of interest based on clinical experience.In the future, we plan to give students the opportunity to develop and ask additional questions in the meetings. In

Conclusion
Because of this investigation's small sample size and limited methodological approach, the results are not generalizable.A major limitation of this case study is the differences observed in survey responses cannot be attributed to the journal club with any certainty.The authors assumed that as students progressed through the course, their critical appraisal skills and their evidence-based practice self-efficacy would mature and improve regardless of their participation in the journal club activity.
This case study identifies some weaknesses of journal clubs.For example, participation in journal clubs may not impact undergraduate students' professional reading habits, and the assigned activity may not improve students' likelihood of participating in a journal club in the future.Despite these limitations, this case study adds to a small body of literature that shows evidence of the journal club as a viable and appropriate teaching method in undergraduate nursing curriculum.
Although the students' awareness of evidence-based practice and their confidence in critically appraising research seemed to improve, the authors plan to focus on the empirical effectiveness of the journal club in improving undergraduate nursing students' EBP skills and self-efficacy.This case study establishes the use of the journal club as a method for developing students' information literacy skills, and it describes a successful instance of embedded were revised for clarity and to avoid redundancy.Both surveys can be viewed as Appendices B and C. The following four open-ended questions were added to the post-survey to provide students the opportunity to elaborate on their responses and to provide feedback on the experience:1.What field or specialty of nursing are you most interested in? 2. If you read non-assigned nursing materials, please list titles and/or authors below.3.When would participating in the journal club be most helpful and why?Before, during, or after the research course?4. Is there any additional feedback you would like to provide about the journal club experience?
response to the open-ended survey questions, the majority of students (70%) wrote positive statements about participating in the journal club or found the journal club helpful for learning how to critically appraise research.One student commented, "without the helpful hand of the journal clubs, I feel like we still would be under educated about critiquing research."This positive feedback was demonstrated in the classroom setting as well; as the semester progressed and more students had participated in journal clubs, class discussions became more insightful and quiz scores improved.Overall, the primary benefit of the activity seemed to be that it provided students with multiple opportunities to further develop critical appraisal skills learned in class in a social, low-stakes setting.Because the students' open-ended feedback contradicted their survey responses, additional research is necessary to determine factors that influence students' decisions to participate in journal clubs and professional development opportunities in the future.Careful consideration should also be made to minimize response bias in surveys, which may have contributed to this discrepancy.

Table 1 : Pre-and post-journal club behaviors, beliefs, and self-efficacy Question/Statement Never Rarely Sometimes Very Often Always
Pennsylvania Libraries: Research & PracticePromoting Evidence-Based Practice and Information Literacy through an Undergraduate Nursing Journal Club palrap.orglibrarianship by an academic librarian within a baccalaureate nursing program.Ultimately, this case study demonstrates that the journal club activity can serve as a bridge for librarians wanting to make a meaningful contribution to the undergraduate nursing curriculum beyond the freshman level.