Measuring the impact of new library services

Measuring the impact of new library services

اسلاید 1: Measuring the impact of new library servicesDr. Roswitha PollMünster

اسلاید 2: Impact / outcomeOutcomes are the results of library use as affecting the individual userImpact shows not the quality of the service, but the quality of individuals in consequence of using the service

اسلاید 3: Impact / outcome of cultural institutionsknowledgeinformation literacyacademic or professional successsocial inclusionindividual well-being

اسلاید 4: Impact / outcome of library serviceschanges in skills and competenceschanges in attitudes and behaviourchanges in the structure of the librarys clientelehigher social inclusionhigher success in research, study, or profession

اسلاید 5: Impact of a librarys services on the outcomes of its parent institutionrecruitment and retention of studentsrecruitment and retention of excellent academic staffeffective teaching - high graduation rates - high grades in examinations - high employment rates after examinationeffective research - high renown and use of research results and publications - renown of faculties and research groups - high amount of special grants - awards, honoursOutcomes of universitiesThe library can support nearly all of these goals

اسلاید 6: accountabilityReasons for measuring impact of new servicesmanagement of resourcespromotion of the librarys rolespecial funding needed: evidence of positive effectsresults-based budgetingjustifying investment into changerising expenses for new serviceshigher workload when introducing new servicesincreasing demand for new servicesnew allocation of resources necessarycommunication of benefits

اسلاید 7: New services (includes services replacing or complementing traditional services)Electronic services (examples)New non-electronic services (examples)regional/national catalogue databasesportalsonline ordering and deliverypersonalized servicesonline referencedigitized collectionselectronic publishingonline self-paced trainingInternet access via the librarygroup work areasstudy landscapes for problem-based learning

اسلاید 8: .Methods of measuring impact of new servicesProblems of measuringMost tested methods are time-consumingData not available because of data protection rulesResults of projects not comparable because of different data collection methodsServices have different outcome and value for different user groupsIt is difficult to prove that changes in competences or behaviour are indeed due to library useInfluences on individualsare diverse

اسلاید 9: Methods of measuring impact of new servicesUse statistics as measures of impactnew electronic media frequency of use location of use forms of use (sessions, searches, downloads) range of resources used before and after introducing a new service use statistics for a new service (e.g. online delivery)change of use in traditional serviceschange in remote usenumber of new users that had not used the library that had not used the same service in traditional formpercentage of the primary user group using electronic servicesUsers are familiar with the use of electronic resources and services

اسلاید 10: Methods of measuring impact of new servicesQualitative measuresAsking users: print or online surveys, focus groups, interviews What knowledge have they of the new service? Use of services (frequency, location of use) Did they find the new service useful? What problems? Did they get training for a new service? useful? Have they improved skills by using a new service?Exit surveysmost useful

اسلاید 11: Methods of measuring impact of new servicesQualitative measuresAsking academic teachersDo new services support teaching and research? Do they encourage students to use new services? For what purpose do they use new services? Asking library staffWhat help do users need for what service?Was there a change of skills after the introduction of a new service?Anecdotalevidence

اسلاید 12: Methods of measuring impact of new servicesOther methodstests: can assess user skills before and after training on a new service performance monitoring / data mining: can document changes in searching by a new servicemystery shoppers: can report on failures or success and on improvements they perceived in their skillsanalysis of documents and bibliographies compiled by users: Is there a change in - resources used - accuracy of citations - number of electronic resources cited?expected outcomes – according to goalsunexpected outcomes

اسلاید 13: Practical examplesImplementing a regional portal for academic librariesMethods used: use statistics, surveys of academicsStatistical resultsSurvey results:Academics said theyHigher percentage of external usersHigher use of most electronic resources saved time by the integrated search changed their seeking ways found access easier found broader covering of relevant information became acquainted with new resources used more interdisciplinary search integrated the portal into teachingchange ofcompetences, attitudes,behaviour

اسلاید 14: Practical examples Opening a group learning area Methods used: use statistics, survey to studentsStatistical resultsSurvey results:Students said theyPhysical visits to the library have increased work more frequently in the library work more frequently in groups learn from group members who attended library training on electronic services find working in the library more attractivechange in attitudes, behaviour,personal well-being

اسلاید 15: Practical examples Changing the journal collection to electronic form Methods used: use statistics, survey to facultyStatistical resultsSurvey results: online versions were accessed 10 times more In 1998 a number of academics were sceptical as to E-journals In 2002 it was exceptional for faculty staff not to use the electronic version usage of an online article 5.4 times cheaper high decline in print usage remote use increased physical library visits decreasedchange of attitudes and behaviour

اسلاید 16: Practical examplesImplementation of an inquiry-based instruction programMethod used: Analysis of bibliographies in students research papersResults of the analysis increase in citations to scholarly journals no significant differenc in other topics accuracy of citation currency of resources competence of judginghigher awarenes of scholarly journals

اسلاید 17: Impact of new services on the libraryChanges in organization and resource allocationinvestment of funds, room, staff timestaff traininguser trainingchange of workflowschanges in organizational structureReplacement of traditional by electronicservices: possible improvements inspeed of deliveryaccuracy of deliveryease of accessrelevance for usersmarket penetration

اسلاید 18: Impact of new services on the libraryImpact on costsProbably transforming a service from traditional to electronic form will not reduce costsBut: Higher use will reduce cost-per-useImpact on traditional servicespossible decrease in - circulation - copying - physical visitsmay be counterbalanced by - comfortable surroundings in the library - group working areas - good in-house IT equipment

اسلاید 19: Outcome projectseVALUEd: in evidence base, University of Central England: Toolkit for evaluating electronic information services http://www.evalued.uce.uk/index.htmIMLS (Institute of Museum and Library Services): outcome-based evaluation of projects http://www.imls.gov/index.htmARL New Measures Initiative: several projects - Learning outcomes - Higher education outcomes research review - MINES (Measuring the impact of networked electronic services) http://www.arl.org/stats/newmeas/index.html

اسلاید 20: Outcome projectsIBEC: Information School of the University of Washington and University of Michigan School of Information: Toolkit for assessing the impact of information in communities http://ibec.ischool.washington.edu/default1024.aspxSCONUL and LIRG (Library and Information Research Group): impact initiative and mailing list http://www.jiscmail.ac.uk/archives/lis-impact.htmlIFLA Section Statistics and Evaluation: working group on outcome/impact; bibliography of literature and projects worldwide http://www.ulb.uni-muenster.de/outcome.html

اسلاید 21: Dont throw away the old bucket until you know whether the new one holds water. Swedish Proverb A new broom sweeps clean, but an old broom knows the corners. Virgin Islander Proverb Men learn little from success, but much from failure. Arabian Proverb To change and to improve are two different things. German Proverb No matter how much the world changes, cats will never lay eggs. Bambara Proverb