COL XR needs to be referenced via dataset identifier, not just `taxonID`

Taxonomy
,
1

I worked through the problem below, I’m leaving it in case it’s helpful for anyone. My ultimate question is can there be a better mechanism at GBIF for resolving a taxon in COL XR that has been archived and is returning 404?

-–

I ran into something unexpected about referencing COL XR taxa via scientificNameID. After reading documentation and posts [1], I thought referencing https://www.gbif.org/taxon/SOMETAXONID was a reasonable thing to do because I thought underneath that was {Some COL XR PID}/SOMETAXONID .

Then, a few days ago I happened to check one of the matches I aligned to on July 3rd:

https://api.gbif.org/v2/species/match?scientificName=Ophioglossaceae&checklistKey=7ddf754f-d193-4cc9-b351-99906754a03b&taxonRank=FAMILY&kingdom=Plantae

Originally it returned: https://www.gbif.org/taxon/DMT

but now it returns: https://www.gbif.org/taxon/KT8B2

And the original identifier (https://www.gbif.org/taxon/DMT) returns a 404. So I did some snooping…

cave explorer GIF and found both Checklistbank and COL mark it as deprecated:

I also found that I wasn’t the only one who had run into this:

And I learned COL XR is issuing a new version with a new DOI every few months, which is great!

GBIF is usually a release behind what is on CLB. Not a big deal, but it means that if I name match on the CLB API, there is a chance I’ll be misaligned with GBIF until GBIF updates the COL XR dataset. I don’t know if this is critical, but I’m trying to get all the pieces to line up.

The Hack

Ultimately, I need to be able to anchor my scientificName to a scientificNameID. And I don’t know of anyway to append an identifier to a DOI to resolve to the taxon instead of the COL XR dataset. So my new plan is to continue using the GBIF API, (which works well for name matching and returns convenient metadata, like IUCN status). But now I also:

  1. Ask GBIF what version of COL XR they’re using: https://api.gbif.org/v1/dataset/7ddf754f-d193-4cc9-b351-99906754a03b/identifier
  2. Ask CLB which dataset key corresponds to that key: https://api.checklistbank.org/dataset?q=10.48580/dgy8b&limit=5
  3. then, I update the scientificnameID to a checklistbank url, which includes the COL XR version identifier: https://checklistbank.org/dataset/315834/taxon/DMT

But in general, I’m feeling:

Should I:

A. RTFM again?
B. Keep calm and carry on?
C. Use a silver bullet that a kind stranger offers me on discourse?

  1. Footnotes

    Taxonomy interpretation :: Technical Documentation

    How to migrate from the legacy GBIF Backbone Taxonomy to Catalogue of Life Extended Release - GBIF Data Blog

    Update on the Catalogue of Life eXtended Release (COL XR) - Technical Support Hour for Nodes

    How will the occurence parquet files change with the move to COL taxonomy identifiers?

    API changes with CoL XR ↩︎

1 Like
Obtaining GBIF's Current CoL Backbone Version
2

One more note. I have also tried https://identifiers.org/col:DMT but it resolves to the COL landing page that says ‘404’ but also includes a note about it being archived. As far as I can tell, there is no content negotiation that would alert a machine that there is more information here: api.checklistbank.org/dataset/3LR/taxon/DMT . But even that is confusing because I’m pretty certain most machines would discard body because it returns a 404 as well.

…still, col:DMT is probably the easiest thing for me to place in scientificNameID for now.

I guess my ultimate question is can there be a better mechanism at GBIF for resolving a taxon in COL XR that has been archived and is returning 404?

3

There probably can. We archive ids and track which releases they span over time. This allows COL to provide tombstone pages like https://www.catalogueoflife.org/data/taxon/DMT that also suggest new identifiers for similar names.

4

Thanks @markus. Tombstone page is a good name for it, I hadn’t heard of that before. Have y’all considered using 410 as the response code from COL? At least it would be something the GBIF build could use to differentiate and pull in the archival message, instead of returning a generic 404 page.

5

there is no COL API, it is all done from CLB and GBIF can do the same - we just need to decide what/when to build it. I’ll look more into this next week

6

that is great for sharing with GBIF in scientificNameID indeed!

1 Like
7

Thank you for this!

Your hack helped me out with a similar issue.

It would be helpful if GBIF made it more clear which version of the CoL backbone is currently in use, but for now this is a thoughtful solution!

8

Hi @mihtmo, you can check the version we are using when you are making a download and it takes you to the version on ChecklistBank:

image
image1632×1134 78.6 KB

It is not super obvious, so I have created an issue to see where else we can display that info in a more visible way, feel free to add comments if you have more ideas where it can be useful to have that info

1 Like
9

@estebanmhGBIF
Thank you, that’s appreciated!
When working with downloads manually, this seems like an great way to check.

In my case, my workflow needs to be able to do this check in an automated fashion, so the earlier solution is still the way I’ll have to do it. As far as I can tell, downloads that have been made through the downloads API don’t seem to display/return an associated XR version? I might be wrong about that.

If there’s any way approved way to retrieve the current version using the GBIF API, I’d love to know! Otherwise, I’ll keep using this DOI → key → download solution and keep an eye out for updates!