# Field disagreements in Darwin Core

**URL:** <https://discourse.gbif.org/t/field-disagreements-in-darwin-core/3370>\
**Category:** Data Publishing\
**Created:** [June 23, 2022, 5:34am UTC](https://discourse.gbif.org/t/field-disagreements-in-darwin-core/3370 "2022-06-23T05:34:10Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![datafixer](https://sea1.discourse-cdn.com/gbif1/user_avatar/discourse.gbif.org/datafixer/32/956_2.png) [@datafixer](https://discourse.gbif.org/u/datafixer)\
**Post date:** [June 23, 2022, 5:34am UTC](https://discourse.gbif.org/t/field-disagreements-in-darwin-core/3370/1 "2022-06-23T05:34:10Z")

</div>

When you’re preparing a Darwin Core table, checking individual fields can be pretty straightforward. As a minimum, you make sure that

1. entries are in the correct field
2. entries are valid for the field
3. entries are appropriate for the field
4. entries are formatted as recommended for Darwin Core
5. entries are consistently formatted

Examples:

(1) “holotype” belongs in _typeStatus_, not _type_

(2) “Beetle sp. 3” is not a valid entry in _scientificName_

(3) “F. Smith, Dec 1968” in _identifiedBy_ should be split between “F. Smith” in _identifiedBy_ and “1968-12” in _dateIdentified_

(4) “2012-10-05” in _eventDate_, not “10/5/12”

(5) “Forest Service workers” and “For. Ser. workers” in _recordedBy_ should both be “Forest Service workers”

The next step in checking isn’t so simple. Individual fields might be perfect on their own, but do they agree with each other in individual records?

For example, you might have a record with “Chrysomela scripta Fabricius, 1801” in _scientificName_ but “Linneaus, 1758” in _scientificNameAuthorship_. Or “France” in _country_ and “Asia” in _continent_. Oops.

Below are some of the Darwin Core disagreements I’ve seen in my work as a data auditor.

_ **Missing-but-expected** _

Missing-but-expected (MBE) disagreements can appear in any group of related fields. Blank entries are perfectly valid in an individual field, but they constitute errors if related fields have relevant entries. Here are some examples:

&nbsp;

| No. of records | decimalLatitude | decimalLongitude | stateProvince | country |
| --- | --- | --- | --- | --- |
| 14 | 52.1934 | -105.4405 | Saskatchewan | |

&nbsp;

| No. of records | scientificName | genus | specificEpithet | infraspecificEpithet |
| --- | --- | --- | --- | --- |
| 26 | Aus bus cus Smith, 1900 | Aus | | cus |

&nbsp;

| No. of records | minimumElevationInMeters | occurrenceRemarks |
| --- | --- | --- |
| 5 | | found just below treeline at ca 2100’ |

&nbsp;

_ **That’s-not-right** _

That’s-not-right (TNR) means that the valid entry in one field simply does not agree with the valid entry in a related field. There are many possible TNRs. Some of the types I’ve seen include:

- “Aus bus cus Smith, 1900” in _scientificName_ but “species” instead of “subspecies” in _taxonRank_
- “Aus bus cus Smith, 1900” in _scientificName_ but “Jones, 1910” in _scientificNameAuthorship_
- “Aus bus cus Smith, 1900” in _scientificName_ and _acceptedNameUsage_ but “Dus” in _genus_
- “2012-10-05” in _eventDate_ but “5.vii.2012” in _verbatimEventDate_
- “2012-10-05” in _eventDate_ but “2010” in _year_
- “2012-10-05/30” in _verbatimEventDate_ but “289” in _startDayOfYear_ (should be 279)
- “2012-10-05” in _eventDate_ but “2009” in _dateIdentified_
- “France” in _country_ but “Asia” in _continent_
- _decimalLatitude_ and _decimalLongitude_ are “63.6762” and “13.6875” but _country_ is “Norway” (should be “Sweden”)
- _decimalLatitude_ and _decimalLongitude_ are given to 2 or 3 decimal places, but _coordinateUncertaintyInMeters_ is “10” (See [this Wikipedia table](https://en.wikipedia.org/wiki/Decimal_degrees#Precision))
- “400” in _minimumElevationInMeters_ but “300” in _maximumElevationInMeters_
- “25” in _individualCount_ but “absent” in _occurrenceStatus_

_ **one-to-many** _

Overlapping with TNR, I very often see a valid entry in one field and more than one valid entry in another field. Which of the latter is correct? It’s usually not possible to answer that question from within the Darwin Core table, and either an external data reference or the data compiler needs to be consulted.

&nbsp;

| No. of records | genus | family |
| --- | --- | --- |
| 63 | Aus | Improbabilidae |
| 17 | Aus | Probabilidae |

&nbsp;

| No. of records | scientificName | taxonID |
| --- | --- | --- |
| 63 | Aus bus Jones, 1910 | 167243 |
| 17 | Aus bus Jones, 1910 | 158046 |

&nbsp;

| No. of records | decimalLatitude | decimalLongitude | stateProvince |
| --- | --- | --- | --- |
| 63 | 50.2809 | 10.5383 | Bavaria |
| 17 | 50.2809 | 10.5383 | Thuringia |

&nbsp;

| No. of records | locality | decimalLatitude | decimalLongitude |
| --- | --- | --- | --- |
| 63 | 5 km W of Babinda | -31.8535 | 146.4446 |
| 17 | 5 km W of Babinda | -31.6535 | 146.4446 |

&nbsp;

_ **Finding disagreements** _

With small Darwin Core datasets (not too many records and a small number of fields), it’s possible to check for field disagreements in a spreadsheet by sorting on one field, putting the field to be checked next to that first field (by freezing panes or hiding columns) and reading carefully down the paired columns.

For both small datasets and very large ones, it’s easier and faster to do checking [on the command line](https://www.datafix.com.au/cookbook/) and you are less likely to miss a disagreement. A useful prerequisite, though, is to do field-by-field checks (above). In that last “one-to-many” example, it wouldn’t help if some of the _locality_ entries were “Babinda, 5 km W of”.

Some field disagreements will be detected programmatically by the [GBIF Data Validator](https://www.gbif.org/tools/data-validator/about) and flagged as issues, such as “Country coordinate mismatch”, “Elevation min max swapped” and “Recorded date mismatch”. However, there are many more possible disagreements, and in the case of “one-to-many” disagreements GBIF does not provide context. The Validator is a good place to start, though!

_Robert Mesibov (“datafixer”); [robert.mesibov@gmail.com](mailto:robert.mesibov@gmail.com)_

---

<div class="post-metadata">

**Author:** ![mgrosjean](https://avatars.discourse-cdn.com/v4/letter/m/97f17d/32.png) [@mgrosjean](https://discourse.gbif.org/u/mgrosjean)\
**Post date:** [June 23, 2022, 11:47am UTC](https://discourse.gbif.org/t/field-disagreements-in-darwin-core/3370/2 "2022-06-23T11:47:42Z")

</div>

Thank you datafixer, it illustrates very well the different type of field disagreements that can be encountered.  
I think this post will be useful for many publishers!

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/gbif1/original/1X/d2525d53d815eb0c992cc44730ed8a6a1be1dbb5.png) [@system](https://discourse.gbif.org/u/system)\
**Post date:** [July 23, 2022, 9:48pm UTC](https://discourse.gbif.org/t/field-disagreements-in-darwin-core/3370/3 "2022-07-23T21:48:22Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.