Contents

1. FAIR Evaluator tool




Recipe Overview
Reading Time
30 minutes
Executable Code
Yes
Difficulty
FAIR Evaluator tool
FAIRPlus logo
Recipe Type
Hands-on
Audience
Principal Investigator, Data Manager, Data Scientist
Maturity Level & Indicator
[F+MM-1.1C] [F+MM-1.2C]
hover me Tooltip text
Cite me with FCB049

1.1. Ingredients

Ingredient

Type

Comment

HTTP1.1 protocol

data communication protocol

guidance on persistent resolvable identifiers

policy

Persistent Uniform Resource Locators - PURL

redirection service

Archival Resource Key

identifier minting service; identifier resolution service

Handle system

identifier minting service; identifier resolution service

DOI

identifier minting service

based on Handle system

identifiers.org

identifier resolution service

EZID resolution service

identifier resolution service

name2things rsolution service

identifier resolution service

FAIREvaluator

FAIR assessment

FAIRShake

FAIR assessment

RDF/Linked Data

model

Actions.Objectives.Tasks

Input

Output

1.2. Objectives

  • Perform an automatic assessment of a dataset against the FAIR principles 1 expressed as nanopublications using the FAIREvaluator 2.

  • Obtain human and machine-readable reports highlighting strengths and weaknesses with respect to FAIR.

1.3. Step by Step Process

1.3.1. Loading FAIREvaluator web application

Navigate the FAIREvaluator tool, which can be accessed via the following 2 addresses:

the FAIREvaluator Home page

Fig. 1.3 the FAIREvaluator Home page

1.3.2. Understanding the FAIR indicators

In order the run the FAIREvaluator, it is important to understand to notion of FAIR indicators (formerly referred to as FAIR metrics). One may browse the list of currently community defined indicators from the Collections page

Select a 'FAIR Maturity Indicator - Collections'

Fig. 1.4 Select a ‘FAIR Maturity Indicator - Collections’

1.3.3. Preparing the input information

To run an evaluation, the FAIREvaluator needs to following 5 inputs from users:

  1. a collection of FAIR indicators, selected from the list described above.

  2. a globally unique, persistent, resolvable identifier for the resource to be evaluated.

  3. a title for the evaluation. Enforce a naming convention to make future searches easier as these evaluations are saved.

  4. a person identifier in the form of an ORCID.

Running the FAIREvaluator - part 1 - setting the input

Fig. 1.5 Running the FAIREvaluator - part 1: setting the input

1.3.4. Running the FAIREvaluator

Hit the ‘Run Evaluation’ button from ‘https://fairsharing.github.io/FAIR-Evaluator-FrontEnd/#!/collections/new/evaluate’ page

Running the FAIREvaluator - part 2 - execution

Fig. 1.6 Running the FAIREvaluator - part 2: execution

1.3.5. Analysing the FAIREvaluator report

Following execution of the FAIREvaluator, a detail report is generated.

FAIREvaluator report - overall report

Fig. 1.7 FAIREvaluator report - overall report

Time to dig into the details and figure out the reasons why some indicators are reporting a failure:

FAIREvaluator error report

Fig. 1.8 apparently a problem with identifier persistence if using DOI, which are URN rather than URL sensu stricto

1.4. Conclusion

Using software tools to assess FAIR maturity constitutes an essential activity to ensure processes and capabilities actually deliver and claims can be checked. Furthermore, only automation is able to cope with the scale and volumes of assets to evaluate. The software-based evaluations are repeatable, reproducible and free of bias (other than those that may be related to definitions of the FAIR indicators themselves). These are also more demanding in terms of technical implementation and knowledge. Services such as the FAIRevaluator are essential to gauge improvements of data management services and for helping developers build FAIR services and data.

1.5. Reference

References

1

M. D. Wilkinson, M. Dumontier, I. J. Aalbersberg, G. Appleton, M. Axton, A. Baak, N. Blomberg, J. W. Boiten, L. B. da Silva Santos, P. E. Bourne, J. Bouwman, A. J. Brookes, T. Clark, M. Crosas, I. Dillo, O. Dumon, S. Edmunds, C. T. Evelo, R. Finkers, A. Gonzalez-Beltran, A. J. Gray, P. Groth, C. Goble, J. S. Grethe, J. Heringa, P. A. ‘t Hoen, R. Hooft, T. Kuhn, R. Kok, J. Kok, S. J. Lusher, M. E. Martone, A. Mons, A. L. Packer, B. Persson, P. Rocca-Serra, M. Roos, R. van Schaik, S. A. Sansone, E. Schultes, T. Sengstag, T. Slater, G. Strawn, M. A. Swertz, M. Thompson, J. van der Lei, E. van Mulligen, J. Velterop, A. Waagmeester, P. Wittenburg, K. Wolstencroft, J. Zhao, and B. Mons. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data, 3:160018, Mar 2016.

2

M. D. Wilkinson, M. Dumontier, S. A. Sansone, L. O. Bonino da Silva Santos, M. Prieto, D. Batista, P. McQuilton, T. Kuhn, P. Rocca-Serra, M. Crosas, and E. Schultes. Evaluating FAIR maturity through a scalable, automated, community-governed framework. Sci Data, 6(1):174, 09 2019.

1.6. Authors

Authors

Name

ORCID

Affiliation

Type

ELIXIR Node

Contribution

Philippe Rocca-Serra

University of Oxford

Writing - Original Draft

Dominique Batista

University of Oxford

Writing - Review & Editing
Assessment 2. FAIRshake tool