全球生物量数据产品（python）

项目级资产

数据集项目包含以下资产。

列

每个表包括以下列。

Columns

Each table includes the following columns.

Name

Description

Type

Gbifid

GBIF’s identifier for the occurrence

Int64

Datasetkey

GBIF’s UUID for the dataset containing this occurrence

Byte_array

Occurrenceid

See dwc:occurrenceID.

Byte_array

Kingdom

See dwc:kingdom. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Phylum

See dwc:phylum. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Class

See dwc:class. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Order

See dwc:order. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Family

See dwc:family. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Genus

See dwc:genus. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Species

See dwc:species. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Infraspecificepithet

See dwc:infraspecificEpithet. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Taxonrank

See dwc:taxonRank. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Scientificname

See dwc:scientificName. This field has been aligned with the GBIF backbone taxonomy.

Byte_array

Verbatimscientificname

The scientific name as provided by the data publisher

Byte_array

Verbatimscientificnameauthorship

The scientific name authorship provided by the data publisher.

Byte_array

Countrycode

See dwc:countryCode. GBIF’s interpretation has set this to an ISO 3166-2 code.

Byte_array

Locality

See dwc:locality.

Byte_array

Stateprovince

See dwc:stateProvince.

Byte_array

Occurrencestatus

See dwc:occurrenceStatus. Either the value PRESENT or ABSENT. Many users will wish to filter for PRESENT data.

Byte_array

Individualcount

See dwc:individualCount.

Int32

Publishingorgkey

GBIF’s UUID for the organization publishing this occurrence.

Byte_array

Decimallatitude

See dwc:decimalLatitude. GBIF’s interpretation has normalized this to a WGS84 coordinate.

Double

Decimallongitude

See dwc:decimalLongitude. GBIF’s interpretation has normalized this to a WGS84 coordinate.

Double

Coordinateuncertaintyinmeters

See dwc:coordinateUncertaintyInMeters.

Double

Coordinateprecision

See dwc:coordinatePrecision.

Double

Elevation

See dwc:elevation. If provided by the data publisher, GBIF’s interpretation has normalized this value to metres.

Double

Elevationaccuracy

See dwc:elevationAccuracy. If provided by the data publisher, GBIF’s interpretation has normalized this value to metres.

Double

Depth

See dwc:depth. If provided by the data publisher, GBIF’s interpretation has normalized this value to metres.

Double

Depthaccuracy

See dwc:depthAccuracy. If provided by the data publisher, GBIF’s interpretation has normalized this value to metres.

Double

Eventdate

See dwc:eventDate. GBIF’s interpretation has normalized this value to an ISO 8601 date with a local time.

Byte_array

Day

See dwc:day.

Int32

Month

See dwc:month.

Int32

Year

See dwc:year.

Int32

Taxonkey

The numeric identifier for the taxon in GBIF’s backbone taxonomy corresponding to scientificname.

Int32

Specieskey

The numeric identifier for the taxon in GBIF’s backbone taxonomy corresponding to species.

Int32

Basisofrecord

See dwc:basisOfRecord. One of PRESERVED_SPECIMEN, FOSSIL_SPECIMEN, LIVING_SPECIMEN, OBSERVATION, HUMAN_OBSERVATION, MACHINE_OBSERVATION, MATERIAL_SAMPLE, LITERATURE, UNKNOWN.

Byte_array

Institutioncode

See dwc:institutionCode.

Byte_array

Collectioncode

See dwc:collectionCode.

Byte_array

Catalognumber

See dwc:catalogNumber.

Byte_array

Recordnumber

See dwc:recordNumber.

Byte_array

Identifiedby

See dwc:identifiedBy.

Byte_array

Dateidentified

See dwc:dateIdentified. An ISO 8601 date.

Byte_array

License

See dwc:license. Either CC0_1_0 or CC_BY_4_0. CC_BY_NC_4_0 records are not present in this snapshot.

Byte_array

Rightsholder

See dwc:rightsHolder.

Byte_array

Recordedby

See dwc:recordedBy.

Byte_array

Typestatus

See dwc:typeStatus.

Byte_array

Establishmentmeans

See dwc:establishmentMeans.

Byte_array

Lastinterpreted

The ISO 8601 date when the record was last processed by GBIF. Data are reprocessed for several reasons, including changes to the backbone taxonomy, so this date is not necessarily the date the occurrence record last changed.

Byte_array

Mediatype

See dwc:mediaType. May contain StillImage, MovingImage or Sound (from enumeration, detailing whether the occurrence has this media available.

Byte_array

Issue

A list of issues encountered by GBIF in processing this record. More details are available on these issues and flags in this blog post.

Byte_array

数据集资产

import pystac_client

catalog = pystac_client.Client.open(
    "https://planetarycomputer.microsoft.com/api/stac/v1",
)
search = catalog.search(collections=["gbif"])
items = search.get_all_items()
items = {x.id: x for x in items}
list(items)

['gbif-2021-09-01',
 'gbif-2021-08-01',
 'gbif-2021-07-01',
 'gbif-2021-06-01',
 'gbif-2021-04-13']

item = list(items.values())[0]
item

import planetary_computer

signed_asset = planetary_computer.sign(item).assets["data"]

import dask.dataframe as dd

df = dd.read_parquet(
    signed_asset.href,
    storage_options=signed_asset.extra_fields["table:storage_options"],
    dataset={"require_extension": None},
)
df

chunk = df.get_partition(0).compute()
chunk

chunk.groupby(["kingdom", "phylum", "class", "family", "genus"])[
    "species"
].value_counts().sort_values(ascending=False).head(15)

kingdom   phylum    class           family       genus          species                    
Animalia  Chordata  Aves            Sulidae      Sula           Sula variegata                 143568
                    Actinopterygii  Percidae     Perca          Perca fluviatilis               48700
                                    Cyprinidae   Rutilus        Rutilus rutilus                 39743
                                    Esocidae     Esox           Esox lucius                     37786
                                    Percidae     Gymnocephalus  Gymnocephalus cernua            23030
                                    Cyprinidae   Abramis        Abramis brama                   17225
                                    Lotidae      Lota           Lota lota                       15668
                                    Cyprinidae   Alburnus       Alburnus alburnus               14580
                                    Salmonidae   Coregonus      Coregonus maraena               14381
                                                                Coregonus albula                13104
                    Mammalia        Phocoenidae  Phocoena       Phocoena phocoena               12250
                    Actinopterygii  Osmeridae    Osmerus        Osmerus eperlanus               11782
                                    Salmonidae   Salmo          Salmo trutta                    10384
                                                 Salvelinus     Salvelinus alpinus              10340
                                    Cyprinidae   Scardinius     Scardinius erythrophthalmus      9120
Name: species, dtype: int64

species_per_country = chunk.groupby("countrycode").species.nunique()
species_per_country

countrycode
AD      6
AE     55
AF     32
AG     51
AI     37
     ... 
YE     19
YT      5
ZA    688
ZM     82
ZW     91
Name: species, Length: 最后，我们可以通过将species_per_country连接到有国家边界的数据集上，用geopandas在地图上绘制计数。243, dtype: int64

import geopandas
import cartopy

countries = geopandas.read_file(
    "https://raw.githubusercontent.com/datasets/geo-countries/master/data/countries.geojson"
)
gdf = geopandas.GeoDataFrame(
    countries.merge(species_per_country, left_on="ISO_A2", right_index=True)
)
crs = cartopy.crs.Robinson()
ax = gdf.to_crs(crs.proj4_init).plot(
    column="species", legend=True, scheme="natural_breaks", k=5, figsize=(15, 15)
)
ax.set_axis_off()

from dask_gateway import GatewayCluster

cluster = GatewayCluster()
cluster.scale(16)
client = cluster.get_client()
#然后使用dask.dataframe.read_parquet来读入这些文件。为了进一步加快速度，我们将指定一个要读入的文件子集。

df = dd.read_parquet(
    signed_asset.href,
    columns=["countrycode", "species"],
    storage_options=signed_asset.extra_fields["table:storage_options"],
)