Semantic artefact assessment

One of the main objectives of the FAIR-IMPACT project is to extend FAIR assessment beyond data. Based on existing best practices and recommendations by the community (see Further information), in FAIR-IMPACT we have addressed semantic artefact assessment from two different perspectives, further described below.

FAIR assessment methodology

We have proposed a methodology for assessing semantic artefacts (vocabularies and ontologies) by extending the Linked Open Terms Methodology (LOT) for ontology development. LOT has been widely for ontology development, and we extended it adding two new phases for FAIR assessment: FAIR pre-assessment (i.e, when a semantic artefact is not yet ready public in the Web) and FAIR assessment (i.e. when a semantic artefact is ready to be published on the Web). In both cases, we have distilled existing FAIR assessment guidelines and recommendations in seven steps, ranging from checking the implementation language of the target semantic resource to checking the community best practices that apply to it (see image 'Overview of the proposed FAIR assessment methodology'). For each step, the methodology indicates which FAIR principles apply, as well as how two different tools for semantic artefact FAIR assessment (FOOPS! and O’FAIRe) implement tests for assessing that step. You can already check out the milestone report about this here.

The image below is an overview of the proposed FAIR assessment methodology. There are six main steps to be carried out for FAIR assessment when developing the ontology, with a transversal step depending on the best practices issued by specific communities. Four of the proposed activities may be carried out without the ontology being published online, and hence they are part of the pre-assessment activity. The two remaining activities (i.e., looking for an ontology in existing registries and checking the ontology publication) can only be addressed once a release of the ontology is available online (some registries require online availability to store an ontology).

Image: Overview of the proposed FAIR assessment methodology. Citation: Garijo, D., Poveda-Villalón, M., Flohr, P., Gonzalez-Beltran, A., le Franc, Y., & Verburg, M. (2023). M5.3 Semantic artefact assessment methodology (Version 1). 

FAIR by design methodology

While our FAIR assessment methodology addresses the assessment of semantic artefacts that may pre-exist, the future deliverable D4.2 will focus on a methodology to generate FAIR semantic artefacts and their context by design. The methodology identifies gaps in current practices for ontology engineering and proposes solutions to address them within the semantic artefact development process. For example, the requirement elicitation with competency questions (CQs) is key for validating a semantic artefact. However, CQs are rarely linked to the ontology itself, and therefore a key element of semantic artefact rationale and provenance is lost (resulting in a gap). You can already check out the milestone here.

Semantic artefacts can be assessed using automated FAIR assessment tools. You can find the tools that FAIR-IMPACT works with here.

Further information

• Best practices for implementing FAIR Vocabularies and Ontologies on the Web
• D2.8 FAIR Semantics Recommendations Third Iteration (V1.0)
• Coming to Terms with FAIR Ontologies
• Features of a FAIR vocabulary
• Ten simple rules for making a vocabulary FAIR
• O'FAIRe makes you an offer: Metadata-based Automatic FAIRness Assessment for Ontologies and Semantic Resources. Amdouni E., Bouazzouni S., Jonquet C. International Journal of Metadata, Semantics and Ontologies, 2022, 16 (1), pp. 16-46

Community feedback

We are very happy to invite community feedback on this work. You can do this in different ways:

• Commenting on this webpage. Please in your comment below, aim to be as specific as possible as to what element of the methodology you are referring to.
• Providing direct comments on the full report. You can leave suggestions and comments on specific parts or ask for clarification from the authors.
• Would you like more personal contact on the topic of metrics for data? Get in touch with us by sending an email to metrics@fair-impact.eu. 

As you read and comment on these metrics please bear in mind the following: 

• M5.3 is still a work in progress, and we are working on a second iteration to address all the received community feedback. 
• If you detect any issues with the documents or assessment tools, please let us know. Please open issues in the respective repositories of O’FAIRe (https://github.com/agroportal/fairness) or FOOPS! (https://github.com/oeg-upm/fair_ontologies) when appropriate.
• If you have new types of semantic artefacts not covered by the presented methodologies, please do let us know. We are in the process of building a semantic artefact benchmark that may benefit from heterogeneous semantic artefacts.