ELRA builds on extensive experience in national and European HLT projects in playing its role as a major contributor to the creation of a European infrastructure for HLT evaluation. Long term involvement in the field helps ELRA to provide a forum for evaluation activities as well as a warehouse of software packages and resources for various kinds of evaluation. ELRA’s over-riding aim is to facilitate communication between the various stakeholders in the provision of adequate evaluation: funding agencies, the scientific academic community, the research and development communities in industry and the user community wishing to profit from accurate evaluation of human language technology products, as well as all other interested parties.

Examples of ELRA’s experience in HLT evaluation include:

• Editing and developing evaluation guidelines, reports

• Providing an online information service with knowledge sources on HLT evaluation

• Launching and carrying out evaluation campaigns

• Producing evaluation packages for a wide range of technologies

• Providing online automatic evaluation interfaces

• Conducting customized evaluations (protocol design, data collection, scoring software implementation, etc)

• Performing market hlt software evaluations (hlt software assessment and/or selection, confidential)