NSW.net Consortium Content Evaluation Working Party
Background
The Consortium Content Evaluation Working Party was established to review database products for potential use by public libraries. The resources must be web based, include full text and include a significant amount of Australian content. The capacity of the product and licence to allow remote access and IP authentication is also important.
Electronic Database Evaluation form
Electronic Database Evaluation & Comparison form
Working Party Membership
Membership of the CEWP comprises a representative from each of the nine public library zones across the state, together with representatives from NSW.net and the State Library (Reference Services, and PLS Division).
The current zone members of the working party can be found at the following location together with a list of library services in the Zone which they represent. ZONE MEMBERS
Guidelines for Delivery and Assessment of Resources
The guidelines are subject to ongoing modification in response to market and industry changes and the interests of NSW.net and its users.
To suggest a database, information resource or service that may be suitable for purchase by NSW.net users, please complete the NSW.net Inquiry Form
Contents
- Content
- Statistics
- Access Model
- Navigation
- Searching
- Functionality
- Media
- Accessibility
- Support
- Licence Agreements & Legalities
- Price
- Other
- Consortium Purchase Desirables
Evaluation, Selection, Negotiation and Delivery Processes
- Database, information resource or service is submittted to the mailing list by either a NSW.net user, member of the public or vendor.
- The NSW.net Coordinator is advised.
- The Coordinator ensures that all working party members are informed via the mailing list, and initial opinions are invited.
- Opinions are assessed. If there is a unanimous opinion that the content is not appropriate then the submission will not proceed. The person who made the submission will be informed of the decision.
- The Coordinator will inform the members that the resource is available for assessment either via the list or at a meeting and invite a member to take ownership of the assessment process.
- The assessor/s may draw on any available resources or people with relevant expertise for the assessment of the product.
- A trial should be arranged by the Coordinator and / or the assessor/s. The trial may be made available to any member of the party or other interested NSW.net users and include support from the vendor to ensure that the scope and the use of the database is understood.
- In assessing the product all the Selection Criteria (see below) must be addressed. The recommendation to make or not make a consortium purchase offer to NSW.net users should be based on these criteria.
- The assessor/s of the product should produce a report which provides a response to the Selection Criteria and recommends that NSW.net either enter into negotiations with the vendor to offer the product or that the product be rejected. To ensure reasonable consideration is given to the product the report should highlight contentious issues, and shortcomings.
- This report will be circulated to the Coordinator and all working party members via the mailing list.
- While any member may still raise further concerns which will be considered, it will be assumed that the report is the result of full and informed consideration and consultation, and worthy of proceeding.
- On receipt of the report the Coordinator will accept or decline the recommendation.
- If the recommendation is accepted, the Coordinator will negotiate a price with the vendor, and may attempt to improve the delivery of the product according to the recommendation.
-
If negotiations are successful offers will be made in July and December. Offers over $2000 should be made in December to allow for inclusion in the next year's budget, and with the expenditure to occur in the following financial year. Smaller offers can be made in either July and December. If an opportunity arises outside of these occasions the Working Party can recommend that an offer be made. The offers should include:
- a brief synopsis of the product
- a summary or complete list of titles included
- the licence agreement
- the price - this may be an appoximation due to the nature of the offer made by vendors ie dependent on a certain number of subcribers
- contact details of the vendor
- the URL of the product or vendor, as well as details of any additional trials
- the email address of the mailing list for any further questions
- a deadline to respond
- any other relevant information
- On receipt of acceptances to subscribe to the offer the coordinator will arrange with the vendor for payment, connection to the service, support and training. The NSW.net website will inform users about access to ongoing support and training services.
- The Coordinator may not proceed with the purchase if the number of subcribers is less than 10. Users will be informed if this decison is made and alternative arrangements suggested, where possible.
- Is the content already available through another NSW.net consortium purchase? If so, is this resource significantly less expensive or does it contain significant additional features?
- Is there a significant amount of Australian content?
- Is the content reliable and authoritative?
- What percentage is full text?
- Is the content frequently and comprehensively updated?
- Will the content meet the client needs of NSW.net users?
- Is this content available elsewhere and would it be better obtained by other means (eg another provider or in another media)
- How is the content archived? What happens if a title in the database goes out of print? Does the vendor archive back editions or keep these titles in the database?
- Are usage statistics available and accessible by each site independently? Do they show relevant details for assessing the value of the product to users?
- Do the statistics comply with the requirements of the ePLEG guidelines?
What kind of access model does the product permit?
E.g:
- unlimited users, by geographical location
- finite users, by geographical location
- concurrent users across the network
- remote access
- charge by searches or articles retrieved
- Is navigation intuitive and user friendly?
- Do pages and items load quickly?
- Can searches be narrowed by media type, source etc?
- Is advanced searching available?
What features are provided?
- persistent URLS (storable searches)
- saved searchs
- Z39.50 implementation
- printing and emailing of search results
- integration with library systems
What media does the database provide, and what file formats does it come in?
- full text (.html, .txt)
- abstracts (.html, .txt)
- images (.jpg, .gif, .png)
- sound (.mp3, .wav)
- video (.mpg)
Are these file formats appropriate and platform independent?
Does the system meet W3C accessibility guidelines?
For validating HTML:
http://validator.w3.org/
For checking accessibility:
http://webxact.watchfire.com
Also useful for testing the appearance to colour-blind people is:
http://www.vischeck.com
Note that Vischeck is still being developed and is not yet reliable.
Issues to consider:
- Is it readable by screen readers?
- Do the images have ALT tags?
- Could there be problems for the colour-blind?e.g: color coded icons should also be differentiated by shape.
- Is technical support available for NSW.net users?
- What form does it take e.g 1800 number, email, etc?
- Will an account manager be nominated for NSW.net?
- Is online context sensitive help available?
- Will the vendor conduct training sessions on a regular basis?
- Has the product experienced any problems in the last 12 months? (eg Servers down etc)
- Are there any compatibility problems with client machines and software e.g Netscape, Internet Explorer, etc?
Licence Agreements and Legalities
- Does the provider have a suitable model for access to meet the needs of NSW.net users e.g branch library access?
- Is the licence agreement acceptable i.e. it does not leave NSW.net users open to unwarranted liabilities?
- Is the content of the product copyright cleared?
- For more information about licence agreements consult the following resources:
- Does the product provide value for money?
- Does the price differ substantially depending on the form e.g. magazine, book, CD-Rom?
- Are there any extra charges in addition to the subscription price?
- What other issues may need to be considered about this product?
Consortium Purchase Desirables:
- Is the product available with IP authenticated access?
- Does it allow unlimited concurrent users?
- Are users billed directly?
- Are there any unreasonable indemnification clauses in the licence agreement?
- Is remote access available?
- Are usage statistics available and accessible by each site independantly, to demonstrate the value of the product to users? Do they comply with the requirements of the ePLEG guidelines?
- There should not be penalties or blockages to access for excess usage. If use of the product exceeds the terms of the agreement, the vendor should notify NSW.net before imposing penalties or blocking access.
- Consideration should be given to resources which overlap to avoid small consortium purchases.

