Gap Analysis of Agrobiodiversity data in GBIF and the NAL Thesaurus
<p>This dataset contains all documents, the text and the pdf files, as well as the code that was used to carry out the term analysis of agriculturally relevant organisms in GBIF. </p> <p><a href="https://www.gbif.org/">The Global Biodiversity Information Facility (GBIF)</a> is an international network and research infrastructure funded by the world's governments and aimed at providing anyone, anywhere, open access to data about all types of life on Earth. </p> <p><a href="https://agclass.nal.usda.gov/">The National Agricultural Library Thesaurus (NALT)</a> has online vocabulary tools of agricultural terms. </p> <p>My task was to use the agricultural terms from the NALT and analyze the agriculturally relevant organisms in GBIF. Some of the goals were:</p> <ul> <li>Get descriptive statistics about Agrobiodiversity Data (AgData) in GBIF</li> <li>Create visualizations to view occurrence trends of the GBIF corpus and AgData in GBIF to determine gaps or biases.</li> <li>Provide examples of and code for how agricultural researchers can work with GBIF data.</li> </ul> <p><br></p> <p><strong>Details about the process and the methodologies used to carry out this analysis</strong></p> <p>I started off with trying to extract names from the Agricultural Thesaurus. I encountered some problems trying to extract names using the RDF format in the Thesaurus. An employee at the Library later provided me with the names in the Thesaurus in a text file. </p> <p>I then proceeded to extract the scientific names from that text file to run them through the GBIF API. Since there were so many of the names, the API would throw a connection error. The API can handle only so many requests in a particular interval of time. To handle this, I leveraged exception handling in Python. Every time the API threw an error, I told the script to wait for 5 seconds and then resume sending requests. Although this took a lot of time, it allowed me to get data such as year of occurrence, coordinate values about the ag relevant data from the API.</p> <p><br> <strong>Technology</strong> <br></p> <p>I used Python because it is has support for both web scraping and data analysis, both of which were needed for this project. I used Jupyter notebooks, run through Anaconda. Project Jupyter is a non-profit, open-source project that supports interactive data analysis and scientific computing. It allows users to code right in our browser and eliminates the need to install any other Integrated Development Environment, and also makes it very convenient to share our code. </p> <p>The main packages used in this project are pandas for data manipulation, requests and json to interact with the GBIF API, NumPy which adds support for array and matrix operations and more.</p> <p>Tableau and matplotlib has been used to create visualizations after performing the analysis in Python. </p><div><br>Resources in this dataset:</div><br><ul><li><p>Resource Title: Code.</p> <p>File Name: Code.zip</p><p>Resource Description: This zip file contains multiple Jupyter notebooks that contain the code for all the analysis.</p><p>Resource Software Recommended: Jupyter notebook,url: <a href="http://jupyter.org/">http://jupyter.org/</a> </p></li><br><li><p>Resource Title: Visualizations.</p> <p>File Name: Visualizations.zip</p><p>Resource Description: This zip file contains Tableau workbooks for the visualizations.</p><p>Resource Software Recommended: Tableau,url: <a href="https://www.tableau.com/">https://www.tableau.com/</a> </p></li><br><li><p>Resource Title: Corpus.</p> <p>File Name: Corpus.zip</p><p>Resource Description: This zip file contains the two datasets of family Apidae and Reduviidae.</p></li></ul><p></p>
| Main Author: | |
|---|---|
| Format: | Dataset biblioteca |
| Published: |
2018
|
| Subjects: | Animal production, Crop and pasture production, NAL-KSD, GBIF, Global Biodiversity Information Facility, data visualization, |
| Online Access: | https://figshare.com/articles/dataset/Gap_Analysis_of_Agrobiodiversity_data_in_GBIF_and_the_NAL_Thesaurus/24662769 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|