The International Barcode of Life project (iBOL) dataset seems to have a few problems. I know that this dataset is constructed by GBIF rather than being published directly by BOLD (indeed I was involved in the first releases of this dataset), which speaks to an unfortunate apparent lack of collaboration between GBIF and BOLD.
There are at least two issues. The first is that the images for BOLD specimens do not seem to have been updated since 21 July 2024, whereas the occurrence data in the latest release is more recent (18 March 2026). I realise that BOLD has changed it’s API, and its data packages do not include images, but are there plans to update the image data?
The second issue is that many of the reported event dates for the barcode data are clearly wrong. I discovered this by filtering records for the year 2026 and seeing lots of records with images show up, when as I note above, images haven’t been added since mid-2024.
See for example 3441767615 which is BOLD PLICL155-20 (note that GBIF has out of date URLs to the record in BOLD). GBIF says this has an event date of 2025-01-18, yet in the BOLD data dump I have, the collection date is 2018-01-25. Bizarrely no collection date is displayed on the BOLD web site, but you can use their API to retrieve the details: https://portal.boldsystems.org/api/documents/eAHLTCm2KijKT04tLs5MsQrw8XT2MTQ11TUysM7JzM0sSU0BAMZOC0I= (BOLD’s API is, um, less than user friendly). The relevant part of the result is:
"collection_date_start": "2018-01-25",
"collection_date_end": "2018-02-01",
I was looking through GBIF’s GitHub repo to see if I could find the code used to create the iBOL dataset, but to no avail.
1 Like
@rdmpage, it’s disappointing to hear this, as I’ve generally given a free pass to IBOL data when auditing for Pensoft data papers.
Here’s another example: https://www.gbif.org/occurrence/2251413881. Yes, the BOLD systems page ( BOLD Data Portal - SSDA367-06 ) likewise has the garbage date “0197-10-03”, but note that the BOLD record has exact site “Mbalmayo Forest Reserve” and country “Cameroon (CM)”. The locality has made it into the GBIF record, but not the country. Note also that the DwC typeStatus information “Holotype of Orthogonioptilum neoprox” is put in “disposition” by GBIF and the assigned scientificName is “BOLD:ACF3155”, despite the BOLD record having “Species: Orthogonioptilum neoprox” and “Scientific Name Authorship: Darge, 1992”.
1 Like
@datafixer Unlike you to give someone a free pass 
In GBIF’s defence, there are issues mapping BOLD’s data fields to Darwin Core (see also BCDM/mapping_BCDM_to_DWC.tsv at main · DNAdiversity/BCDM · GitHub). The voucher_type field in BOLD is full of all sorts of values, including type status, is it in a museum, etc. (see iPhylo: Exploring BOLD's DNA barcode data releases: there's a fraction too much friction) so dwc:disposition is probably safest. BOLD uses country/ocean and country_iso, it looks like GBIF doesn’t use either (Darwin core expects doc:country) and attempts to infer country from geocraphic coordinates. No coordinates, no country. Regarding the scientific name, BOLD BINs make sense to me as they are the clusters of barcodes regarded as belonging to the same taxon, and if you look at BOLD data overall, individual barcodes can have all sorts of taxonomic names attached to them.
Thanks for the feedback, it is appreciated!
-
Images: We are now generating the DwC from the BOLD data packages. We are no longer able to create the full dataset of the BOLD APIs, and as you know, image uris are not (yet?) available in the data packages. Rather than skipping the the image extension, I have included the last version of the multimedia extension that we were able to crawl of the APIs. Once image URIs are available in the data packages, we will update asap.
-
Odd dates: I investigated process id PLICL155-20 in the file BOLD_Public.16-Jan-2026.tsv from which the current version was generated (see full tsv line below). The date you found through the API, 2018-01-25, is not present in the tsv. It also looks like there is a column shift there.
processid sampleid fieldid museumid record_id specimenid processid_minted_date bin_uri bin_created_date collection_code inst sovereign_inst taxid kingdom phylum class order family subfamily tribe genus species subspecies species_reference identification identification_method identification_rank identified_by identifier_email taxonomy_notes sex reproduction life_stage short_note notes voucher_type tissue_type specimen_linkout associated_specimens associated_taxa collectors collection_date_start collection_date_end collection_event_id collection_time collection_notes geoid country/ocean country_iso province/state region sector site site_code coord coord_accuracy coord_source elev elev_accuracy depth depth_accuracy habitat realm biome ecoregion sampling_protocol nuc nuc_basecount insdc_acs funding_src marker_code primers_forward primers_reverse sequence_run_site sequence_upload_date bold_recordset_code_arr
PLICL155-20 BIOUG59119-A07 GMP#20575 BIOUG59119-A07 PLICL155-20.COI-5P 11936254 2020-07-08 BOLD:ACE8075 2013-05-07 BIOUG Centre for Biodiversity Genomics 365800 Animalia Arthropoda Insecta Diptera Phoridae Metopininae None Megaselia None None None Megaselia BIN based genus Renee Miskie None Malaise trap PL12-7E Malaise trap PL12-7E Vouchered:Registered Collection inv whole voucher None D.Janzen, W.Hallwachs 2025-01-18 2032-01-05 None None None 504 Costa Rica CR Guanacaste Province Area de Conservacion Guanacaste Sector Pailas, Pailas Dos PL12-7 CRI|NO01_458|PL12_7 [10.7612, -85.3353] None None 791.0 None None None 1. Forest & Woodland|1.6. Subtropical/Tropical Moist Lowland Forest Neotropic Tropical_&_Subtropical_Moist_Broadleaf_Forest Costa_Rican_seasonal_moist_forests Malaise Trap TTTATATTTCATTTTTGGAGCTTGAGCCGGTATAGTAGGAACATCTTTAAGTATTATAATTCGAGCCGAATTAGGTCACCCAGGTGCTCTAATTGGTGATGATCAAATCTATAATGTTATTGTTACTGCACATGCTTTTATTATAATTTTTTTTATAGTTATACCTATTATAATAGGAGGATTTGGTAATTGATTAGTTCCCTTAATATTAGGAGCTCCTGATATAGCATTTCCTCGTATAAACAACATAAGTTTTTGAATATTACCTCCTTCTTTAACACTTTTATTAGCCAGAAGTATAGTAGAAAATGGGGCTGGAACAGGATGAACTGTTTATCCTCCTCTTTCATCTAGTATTGCTCACAGAGGAGCTTCTGTAGATTTAGCTATTTTTTCATTACATTTAGCTGGAATTTCTTCCATTTTAGGAGCGGTAAATTTCATTACAACAATTATTAATATACGATCTTCAGGAATTACCTTTGATCGTATACCTTTATTTGTTTGATCTGTTGGAATTACAGCTTTATTATTACTTCTTTCATTGCCAGTATTAGCAGGAGCTATTACAATATTATTAACTGATCGAAATTTTAATACTTCTTTTTTTGATCCTGCTGGTGGAGGTGATCCAATTCTTTATCAACATTTATT 654 None COI-5P Canadian Centre for DNA Barcoding 2020-09-25 ['PLICL']
I will download the latest data package and see if the problem is still there.
2 Likes
@thomasstjerne BOLD data is a bit of a mess. I see that in the data for PLICL155-20 that you show there is a date 2032-01-05 for collection_date_end! (assuming that the missing two column offset occurs before this field). I have been collecting successive BOLD data packages so at some point I could probably trawl back through them and see when this error was introduced.
Maybe @datafixer can be persuaded to do an audit
@thomasstjerne This is the output of a DuckDB query across several snapshots of the BOLD database (I’ve not grabbed any snapshots for 2026 yet). The date vanishes, then reappears, then changes to a new date. And we have collection_date_end now being in the future. I wish BOLD took more care with their data exports.
┌───────────────────────────┬───────────────────────┐
│ Snapshot │ collection_date_start │
├───────────────────────────┼───────────────────────┤
│ 30-Mar-2022 │ 2018-01-25 │
├───────────────────────────┼───────────────────────┤
│ 06-Jul-2022 │ *(empty)* │
├───────────────────────────┼───────────────────────┤
│ 28-Sep-2022 │ *(empty)* │
├───────────────────────────┼───────────────────────┤
│ 30-Dec-2022 │ *(empty)* │
├───────────────────────────┼───────────────────────┤
│ 31-Mar-2023 │ *(empty)* │
├───────────────────────────┼───────────────────────┤
│ 30-Jun-2023 │ 2018-01-25 │
├───────────────────────────┼───────────────────────┤
│ 29-Sep-2023 → 26-Sep-2025 │ 2018-01-25 │
├───────────────────────────┼───────────────────────┤
│ 26-Dec-2025 │ 2025-01-18 │
└───────────────────────────┴───────────────────────┘
@thomasstjerne I’ve got a list of just over 7 million BOLD images which I think is most of the images linked to barcodes in BINs. There are rpocessids not in BINs that I haven’t got images for, I’m trying to fill those in. Let me know if you are interested in the image metadata file.
Thanks @rdmpage
Yes, I would be interested in the images. The current stale version of images I use, is based on API crawls up until some time in 2024. It contains images for a bit over 7 million processids.
I generated a new version of the DwC-A based on the 22nd May Data Package, it is ingested in GBIF here.
The code for the data transformation is now here: GitHub - gbif/bold-dwca-pipeline · GitHub. Suggestions and improvements are welcome.
@thomasstjerne Just to keep this topic alive for a moment, I am currently up to 8.2 million images, aiming for ~10 million before I call it a day. Will let you know once harvest is complete.
1 Like
I’ve emailed you a link to retrieve a TSV file with ~10.4 million image links, harvested from the BOLD API.