I had a first look at the EEA checklists available on ChecklistBank.Thanks Markus for putting in these initial checklists.
Ideally, I would like to filter GBIF occurrences for one particular policy eg ‘Habitats annex IV’ or any other policy.
For this, it would be good to tag species lists of ‘policy relevance’ in ChecklistBank(how?), not only at EU level, but also at country or even more local level. Maybe we need others dwC/colDP fields for such policy relevant lists.
European Environmental Agency should publish all European policy lists, give accurate metadata, fix issues eg EEA redlist, give access to original data(not only the mapped DarwinCore/CoLDP terms, how?) and maintain the different versions of the checklists.
At the country/local level, GBIF nodes should publish all sorts of policy relevant checklists.
Only then will GBIF be able to deliver ‘policy relevant’ occurrence data and reporting mechanisms using GBIF workflows might become be put in place.
There is a lot of work to be done before that, more on the standards and the content, than on the tools.
But being able to deliver occurrence data relevant for certain policies would be invaluable.
How could we address this with the Secretariat and the GBIF Community?
Maybe with a CESP project? Any volunteer?
Querying GBIF occurrences by a given policy list or more general, by an existing (and maintained) checklist would indeed be very valuable.
Marking checklists as policy relevant lists can be done in 2 ways:
Every checklist dataset has a single type field which is based on this enumeration currently: http://api.checklistbank.org/vocab/datasettype
There is an entry “legal” for lists appearing in legal texts, which in most cases would fit.
Every dataset can also be tagged with free keywords. We need to provide guidance on which keywords to use, but policy, legal, national, regional etc would be good keywords. And in case of national or regional lists the covered area. That already is covered by the geographicCoverage field in the metadata, but I can see this still be useful to have “Netherlands” or “NL” as a keyword. It just needs some guidance to which keywords to use.
original raw files can already be bundled in any dwc or coldp archives. Personally I think this is good practice as long as the archive size isnt going through the roof. A simple folder “sources” or “raw” should also be recommended.
After all it is a lot of best practices to document.
As suggested by Markus Döring, the original files are provided in a ‘raw’ folder in addition to the colDP compliant files.
Also, a github repo was created on our GitLab.
It contains all data files(original, intermediate and colDP) plus the SQL code that transform EEA files into CoL files.
This is just the beginning of the process… colDP standards are fine for taxonomic checklists but not necessarily adequate for legal checklists. That is why the more complete raw files are also published in Frictionless Data Package format.
Linking legal checklists to the corresponding legal texts and dealing with the applicability of these lists (often hidden in obscure annexes) is just the top of the iceberg.
Looking forward to refining this process and adding several more European legal checklists together with Belgian/Flemish/Walloon checklists in the CLB pipe.
Dear GBIF community,
You’ll find my third attempt to publish legal checklists on CLB as CETS 104 Bern Convention.
It contains nameUsage of all three appendixes of the Convention on the Conservation of European Wildlife and Natural Habitats (ETS No. 104) a.k.a. the Bern Convention.
All suggestions are welcome by email or via an issue on our Gitlab repository.
Looking forward to further this topic with you.
These are the lessons I learned after publication of four European legal checklists.
Early choices
The exercise focuses on key European legal checklists that are not yet available yet in a machine readable format
CoLDP format was retained over DwCA as it offers more entitities and attributes
Frictionless data was choosed to package the original (or intermediate data)
Data wrangling mostly consists of SQL statements and/or regular expressions to transform texts
Positive points
The existing exchange standards allow anyone to publish legal checklists
Checklist Bank online tools greatly simplify the publication process
Original data can be added to the ColDP publication. This was added as Frictonless Tabular data package
CLB Data/metadata validation and issues feedbabck helps the publication and give enough feedback to improve the quality of the checklists.
Negative points
Most of the legislations are only available in PDF format. Extracting the data is tedious and difficult process.
Legal texts come with annexes that lists all species affected by the law. Current standards do not support this annex concept. Annexes often related to each others.
In most cases, remarks apply to certain species which is difficult to capture.
Population of species often appear in the legal text, which is difficult to express.
The legal texts that apply to a species or group of species mentionned in the annex are not easy to capture as the legal texts are disconnected from the annex.
Suggestions of improvements
All checklists embedded in legislation should be published in machine readable format, e.g. ColDP. PDF is suitable for printing, not for data analysis.
All legal checklists should receive a DOI to allow simple citation by users.
As these lists evolve, keeping track of the different versions of the lists is crucial.
Checklists publication shall be done directly by the legal authority, in parallel of the publication of the legal texts.
Publication shall clearly state which legal items apply to which species or species populations.
Current CoLDP covers very well the taxonomic checklists. Legal checklists require different entities and attributes. Annexes, references to legal texts, restrictions, population of species, etc… should be covered.
Lawyers, and decision makers, shall be consulted before adapting the CoLDP standards to legal checklists.
Publication of the original data, eg in Frictionless format, shall be encouraged. This original data often retains more information specific to the legislation.