# Data publishing counts distorted by eBird

**URL:** <https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483>\
**Category:** Miscellaneous\
**Created:** [September 7, 2022, 9:25am UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483 "2022-09-07T09:25:19Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![rdmpage](https://sea1.discourse-cdn.com/gbif1/user_avatar/discourse.gbif.org/rdmpage/32/25_2.png) [@rdmpage](https://discourse.gbif.org/u/rdmpage)\
**Post date:** [September 7, 2022, 9:25am UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483/1 "2022-09-07T09:25:19Z")

</div>

The “Data Publishing” tab in country reports gives what I think is a misleading impression of publishing activity by individual countries. For example, for India GBIF reports over 30 million occurrences published.

 ![Screenshot 2022-09-07 at 09.56.22](https://global.discourse-cdn.com/gbif1/original/1X/1545a6f6669c5123993bfdeedbe9f28a16328d0b.png)

But if I look at the list of [25 published datasets from India](https://www.gbif.org/dataset/search?publishing_country=IN) the total occurrences comes to 217,237. Browsing the data it seems that the bulk of the 30 million records counted as published by India are from eBird. While the eBird observations are **in** India, it seems a stretch to argue that they are published **by** India.

It also seems to overshadow efforts such as the [India Biodiversity Portal](https://indiabiodiversity.org/) which has 82,462 occurrences representing 38% of the data for datasets published by India. Yet as a percentage of the data GBIF regards as published by India this is less than 0.3%.

Note that this issue isn’t unique to India. I picked that country because I was curious about the impact of the India Biodiversity Portal. Other countries may well find their estimates of publishing activity distorted by eBird (and other?) data.

Obviously there are various ways to approach this problem, but as it stands the headline figure can give a seriously misleading indication of the direct contribution of a country to GBIF.

---

<div class="post-metadata">

**Author:** ![rui.figueira](https://avatars.discourse-cdn.com/v4/letter/r/49beb7/32.png) [@rui.figueira](https://discourse.gbif.org/u/rui.figueira)\
**Post date:** [September 9, 2022, 9:11am UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483/2 "2022-09-09T09:11:55Z")

</div>

This topic was discussed some years ago among the GBIF Secretariat, some nodes and publishers, before the decision to attribute eBird records as a publication of the country of origin of the records. I took part on the discussion reporting the case of Portugal, where:

- the community of portuguese birdwatchers is very active, many of which use eBird to manage their daily bird observations;
- [SPEA](https://spea.pt/en/), the Portuguese Society for the Study of Birds and a BirdLife’s partner in Portugal, agreed with eBird to use this platform to manage all observations records. SPEA even migrated their existing databases to eBird. This agreement resulted in the regional eBird portal [PortugalAves](https://ebird.org/portugal/home).

Therefore, in the case of Portugal, I think it is fair to say that eBird records in Portugal are published **by** Portugal. One issue that never got a solution by eBird was to identifying records published by SPEA, and correctly attribute them to SPEA as a GBIF publisher, without generating duplications.

---

<div class="post-metadata">

**Author:** ![rdmpage](https://sea1.discourse-cdn.com/gbif1/user_avatar/discourse.gbif.org/rdmpage/32/25_2.png) [@rdmpage](https://discourse.gbif.org/u/rdmpage)\
**Post date:** [September 9, 2022, 9:28am UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483/3 "2022-09-09T09:28:48Z")

</div>

I suspected this was the explanation. The sheer scale of eBird can be overwhelming at times, making it difficult to see underlying patterns.

---

<div class="post-metadata">

**Author:** ![Timhirsch](https://avatars.discourse-cdn.com/v4/letter/t/f9ae1b/32.png) [@Timhirsch](https://discourse.gbif.org/u/Timhirsch)\
**Post date:** [September 12, 2022, 12:53pm UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483/4 "2022-09-12T12:53:38Z")

</div>

I would add that @rui.figueira 's point could equally apply to India. The huge number of eBird records in India is closely related to the activities of a very proactive eBird partner, Bird Count India [https://birdcount.in/](https://birdcount.in/) Under the old way of counting ‘data from’ a country, all of these records would have been shown as being published from the United States as the headquarters of eBird. I would say it is more meaningful to group them as coming from the country of observation as this is primarily the output of Indian volunteer observers. There is never a perfect way to slice data by country, but if eBird overwhelms other data it’s because of a real difference in scale, not an artefact.

---

<div class="post-metadata">

**Author:** ![pieter](https://avatars.discourse-cdn.com/v4/letter/p/82dd89/32.png) [@pieter](https://discourse.gbif.org/u/pieter)\
**Post date:** [September 20, 2022, 12:46pm UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483/5 "2022-09-20T12:46:19Z")

</div>

I got curious what would happen if we were to slice the observations by the country of the source dataset, This is what I found out:

### source

Based on the parquet occurrence dump of august. (looking at the country of origin of the publishing organizations that host the datasets).

### table

| continent | basisofrecord | n |
| --- | --- | --- |
| Africa | PRESERVED\_SPECIMEN | 3472891 |
| Africa | OCCURRENCE | 94676 |
| Africa | LIVING\_SPECIMEN | 57510 |
| Africa | HUMAN\_OBSERVATION | 29296091 |
| Africa | MATERIAL\_SAMPLE | 56655 |
| Africa | MACHINE\_OBSERVATION | 31327 |
| Asia | OCCURRENCE | 530982 |
| Asia | LIVING\_SPECIMEN | 94678 |
| Asia | FOSSIL\_SPECIMEN | 86486 |
| Asia | HUMAN\_OBSERVATION | 5730359 |
| Asia | OBSERVATION | 586884 |
| Asia | MACHINE\_OBSERVATION | 86981 |
| Asia | MATERIAL\_SAMPLE | 609 |
| Asia | PRESERVED\_SPECIMEN | 14102859 |
| Asia | MATERIAL\_CITATION | 772 |
| Europe | PRESERVED\_SPECIMEN | 70357904 |
| Europe | HUMAN\_OBSERVATION | 623233689 |
| Europe | OBSERVATION | 8237730 |
| Europe | MATERIAL\_CITATION | 1086411 |
| Europe | OCCURRENCE | 5604377 |
| Europe | MATERIAL\_SAMPLE | 3361261 |
| Europe | FOSSIL\_SPECIMEN | 3167174 |
| Europe | LIVING\_SPECIMEN | 1241597 |
| Europe | MACHINE\_OBSERVATION | 5983418 |
| North America | FOSSIL\_SPECIMEN | 6771940 |
| North America | OCCURRENCE | 2584175 |
| North America | MATERIAL\_CITATION | 526 |
| North America | MATERIAL\_SAMPLE | 541395 |
| North America | OBSERVATION | 9422865 |
| North America | HUMAN\_OBSERVATION | 1138294213 |
| North America | PRESERVED\_SPECIMEN | 83156613 |
| North America | MACHINE\_OBSERVATION | 2441401 |
| North America | LIVING\_SPECIMEN | 61432 |
| Oceania | MATERIAL\_SAMPLE | 1377494 |
| Oceania | FOSSIL\_SPECIMEN | 5359 |
| Oceania | OBSERVATION | 1560379 |
| Oceania | PRESERVED\_SPECIMEN | 13122144 |
| Oceania | OCCURRENCE | 1170654 |
| Oceania | MACHINE\_OBSERVATION | 1852449 |
| Oceania | HUMAN\_OBSERVATION | 25500411 |
| Oceania | LIVING\_SPECIMEN | 85423 |
| South America | FOSSIL\_SPECIMEN | 40631 |
| South America | LIVING\_SPECIMEN | 62125 |
| South America | MATERIAL\_SAMPLE | 121422 |
| South America | HUMAN\_OBSERVATION | 3746238 |
| South America | PRESERVED\_SPECIMEN | 13929306 |
| South America | OCCURRENCE | 3570246 |
| South America | OBSERVATION | 6287 |
| South America | MATERIAL\_CITATION | 8383 |
| South America | MACHINE\_OBSERVATION | 350041 |
| NA | OBSERVATION | 98996 |
| NA | MATERIAL\_SAMPLE | 35478750 |
| NA | LIVING\_SPECIMEN | 275272 |
| NA | MATERIAL\_CITATION | 601959 |
| NA | MACHINE\_OBSERVATION | 3263962 |
| NA | FOSSIL\_SPECIMEN | 4 |
| NA | OCCURRENCE | 3785904 |
| NA | HUMAN\_OBSERVATION | 93696982 |
| NA | PRESERVED\_SPECIMEN | 10159784 |

I wasn’t able to match up all datasets to a continent. But it still shows that most GBIF occurrences are human observations, and most are hosted in Europe or North America.

### plot

 ![image](https://global.discourse-cdn.com/gbif1/original/1X/a023b8f67e9a54d6fa64f2b3b2bff7cfb5161c40.png)

If you think it would be interesting I could do the same but for class instead of basisofrecord, then we’ll probably see lots of those human observations are birds.

This query took about 20 mins on my machine. It can probably be optimized a bunch.

---

<div class="post-metadata">

**Author:** ![rdmpage](https://sea1.discourse-cdn.com/gbif1/user_avatar/discourse.gbif.org/rdmpage/32/25_2.png) [@rdmpage](https://discourse.gbif.org/u/rdmpage)\
**Post date:** [September 20, 2022, 1:05pm UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483/6 "2022-09-20T13:05:23Z")

</div>

It feels a little like if you’re not doing citizen science on birds, you are now in the long tail of biodiversity data.

---

<div class="post-metadata">

**Author:** ![vijaybarve](https://sea1.discourse-cdn.com/gbif1/user_avatar/discourse.gbif.org/vijaybarve/32/212_2.png) [@vijaybarve](https://discourse.gbif.org/u/vijaybarve)\
**Post date:** [September 23, 2022, 6:57pm UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483/7 "2022-09-23T18:57:09Z")

</div>

And looks like iNaturalist will be next in line.

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/gbif1/original/1X/d2525d53d815eb0c992cc44730ed8a6a1be1dbb5.png) [@system](https://discourse.gbif.org/u/system)\
**Post date:** [October 24, 2022, 4:57am UTC](https://discourse.gbif.org/t/data-publishing-counts-distorted-by-ebird/3483/8 "2022-10-24T04:57:10Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.