G. Agtrials collaboration

Second Proposal for a use case - sites integration

Cassava example: http://www.agtrials.org:8080/tbtrial/61

Cassava variety name in agtrials linked back to a Passport data from the CIAT database, showing the fingerprinting

Objective of our collaboration: Accessing evaluation summary data across sites (GxE) based on a query on 'Popular Crop variety name x trait name'.  To make a use case for Common bean :  we need the reference trait lists (Alberto) and a reference variety list  (Steve - list of most evaluated varieties)  to enable access to the genebank/variety record in CIAT genebank to get the  fingerprinting and other information with the gene of interest  and use Agtrials to assess the stability across environment - we need an integrated way of showing this like in our examples above.

1. First Use case on Rice  : http://www.agtrials.org:8

Agtrials.org (http://www.agtrials.org/)is an information portal developed by the CGIAR Research Program on Climate change, Agriculture and Food Security (CCAFS) which provides a

Access to a database of the performance of agricultural technologies at sites across the developing world. Agtrials is a web repository for evaluation data and a registry for evaluation sites. 

Objectives: 1. Share data and information on evaluations of agricultural technology. 2. Acquire agricultural evaluation data sets for your own research. 3. Explore the geographic dimensions of agricultural evaluation

Present situation

  1. The Crop ontology contains ontology for presently 8 crops
  2. a new web site that enables consulting, commenting, modifying and uploading trait names.  http://www.cropontology-curationtool.org/
  3. Web service API is available to be used by programmes to search, load, delete terms .
  4. The new web site is hosted on Google App Engine (Cloud)
  5. The code is being versioned and hostedon GitHub
  6. The Crop ontology DB is noSQL DB, not relational
  7. CIAT downloaded once the crop ontology terms to store this in the agtrials database which is relational
  8. Agtrials web site uses the terms in dynamic pick lists in the evaluation site registration form (metadata) and  when users query the web site.
  9. Users uploading data or registering sets are not crop specialist and do not really understand the trait names sometimes.


  1. Agtrials needs a dynamic  way of getting the terms from the new Crop ontology site into the registration form and query feature so it is constantly updated – perhaps a real time search of the trait names into the crop ontology ?
  2. Agtrials could insert URLs so users can open the term definition, photos, recommended methods etc, directly on crop ontology site 
  3. Crop ontology needs to get back new trait names created by Agtrials users to follow up on their validation into the ontology

Immediate Actions

  1. Herlin will send the structure of the Db to Milko and Luca – I do have a copy but this may be not the latest version?
  2. With the support of Luca and Milko, Herlin will test the ontology API (http://www.cropontology-curationtool.org/api ) for the various functions running a trial for dynamically getting the terms from the Crop ontology site – use case developed before 10th of September

Skype - 06/09 - Andrew, Herlin, Milko, Luca, ELizabeth

What was done by Luca
1. All necessary Search functions required by Herlin were added to the API and instructions sent to Herlin : http://www.cropontology-curationtool.org/api 
2. Examples are attached to each feature
3. the static HTML page for each trait is available under the link 'download' : http://www.cropontology-curationtool.org/terms/CO_321:0000118/Stem%20rust/static-html
4. Milko indicated the the Agtrials DB needs to store the ontology identifiers

What's next? - to be done before 12/09


  • to add one field for the Ontology Identifier in the Agtrial DB 
  • with the ontology ID, Herlin can develop make sure the agtrials automatically connect the crop ontology site and retrieve all children terms of an ontology and their attributes using the API features in the order as listed hereunder:
    1.  http://www.cropontology-curationtool.org/get-ontology-roots/CO_330
    2. http://www.cropontology-curationtool.org/get-children/CO_330:0000003
    3. http://www.cropontology-curationtool.org/get-attributes/CO_330:0000003


  • Identify the variables to which the ID should be attached to in the Agtrials DB
  • Select the use case to show at the GRM in 2 weeks  - ANDREW & GLENN

Skype - 12/09

1. Andrew mapped the trait names (called variables in Agtrials)  and selected Rice as a use case for our demo on a dynamic retrieval of ontology terms and access to static pages.

2. Elizabeth indicated that Luca is finalizing the annotation tool that will enable to map trait names to the ontology terms and add metadata to the data files

2. Herlin will work on it the coming days and can count on Luca's support

3. The traits list only contains the variables but no relationships or attribute data - some attribute data will be in Agtrials and need to be acquired by the Crop ontology

4. The new trait upload  for Excel files is being finalized by Luca so this could be use for  the demo with Common beans traits

5. For  validating terms, adding relationships and add attributes information(definition,photos)  - curator check is needed - Glen will contact Alberto Fabio Guerrero (CIAT) to inform him about the planned use case and see if he has a TRait list ready as per the request of Rosemary (homework for GCP)

6. Demo can be made during the crop group session and mentioned during the plenary presentation

Next Skype - Friday 15 - 3:00p.m. rome, 8:00 a.m., 8:00 Mexico, ? Kenya

Proposal of a use case:

1.      A researcher is on CropOntology.org examining traits that might be useful for his/her analysis.

2.      They go into the rice ontology and find “drought recovery”http://www.cropontology-curationtool.org/terms/CO_320:0000058/Stem%20rust/static-html(note: Let’s find one with a photograph for the presentation)

3.      Then on Crop Ontology, they might have 2 options (this is what would need to be programmed into the API):

a.      Go to the table in AgTrials to see a list of all trials where this trait was measured -http://www.agtrials.org:85/tbvariablesmeasured - though this capability has not been constructed yet.

b.      Go to a map showing all trial sites where this trait was measured – see http://gismap.ciat.cgiar.org/agtrials/ - again, not yet developed.

4.      Download the evaluation data, analyze it and write up a paper….


Even though these have not yet been developed, we could take screen shots and explain them in the presentation. The AgTrials_to_Crop ontology use case is well defined and this will be big help in documenting the data in AgTrials. Let’s talk about this further in the poster session tomorrow. …..one question about all this is to what extent the user switches sites versus the extent that we just develop it with data available from both efforts, but all in the background…


Last Skype in October 2011 Bioversity/ CIAT-DAPA

1. it was agreed that scientists should 1st enter their trait names into the TRait dictionary so when they fill in the trait form in Agtrials, definition etc will available - example of Musa - Trait dictionary template shared

2. Agtrials needs to be able to pull out the traits for each crop - use of API - Herlin and Luca

3. The feature to send candidate terms from Agtrials to CO needs still to be developed - Herlin and Luca

4. we need to build access to the list of varieties and attribute information like we have for the TRait names - would be variety cards with PaD and charcterization data, observations, photos etc - Luca will show a prototype on scraping online DBs and will start getting Beans data


Skype 4 November

Agenda: discuss the prototype developed by Luca for genebank accession and the way of linking this information back to Agtrials

1.Accessing evaluation sites and summary data based on a query on 'Popular Crop variety x trait'.  To make a use case for Common bean accessions & varieties and  the trait:  we need the reference trait lists (Alberto) and a reference variety list  (Steve) to annotate data with it and that uses  data from the scraping made by Luca - Objective is to enable access to the genebank/variety record to get the  fingerprinting with the gene of interest  and use Agtrials to assess the stability across environment - we need an integrated way of showing this like in our examples above.

2. Herlin and Luca will link Agtrials with the API of the draft genebank web site to implement access from the variety name to the variety/accession card, similar to what was done for the variables measured & ontology terms.

3. Luca will make the Bean accessions more accessible on the draft genebank site and  particularly the varieties that have been the most evaluated (> to 50 sites) - cf Jagath Excel File for the CLimate change project on mapping the Bean diversity - FIGS type approach

4. One objective is to be able to map the varieties where they were evaluated and look for other regions having agroclimatic conditions in which the variety would perform

5. Can we map the accessions on the GENERATION atlas using this API?  (comment= Shall we get the GCP reference sets to make a demo? Do we have reference sets list ready for beans?)

2. List of traits for beans is about to be ready and the Excel file, if under the Trait template format, will be uploaded on the Crop ontology on the web site by Alberto : http://www.cropontology-curationtool.org/add-ontology

Skype of 9 January 2012

Glen informed us about the Agnips project and links with Agtrials. 

Standardization needed 

Interested in the fact that the CO cross reference to  Gramene- Jim jones 

Getting trait names from Agtrials:

1. luca pull info from agtrials through herlin API and get matching/non matching through an automatic annotation

2. Discuss a temporary space to validate the submitted trait names

3. where to display the data with an indication of Agtrials as source

4. when the term enters the ontology, then the link is created

Getting Variety Passport linked to Agtrials

- FRom the pro type of Luca get the gene bank passport

-from EPHYS, see if the varieties are in and if yes, get the data through web harvesting

- Luca could up to now not look at the downloadable db structure but now EPHYS os online and we an indication from Jugath onwhere the get the information from , Luca could get probably the varieties.

Reference sets to be mapped to the atlas Beans ICRISAT = Get the reference sets for beans - discuss that with GCP team get the reference set data add a filter on GCP ref sets on the generation atlas - GCP team not yet interested - approach CIRAD

GCP workshop in PAG Meeting in 2 weeks with a presentation form David Bergvison

Skype of 30 January - Glenn, Luca, Milko, Herlin, Elizabeth

Getting trait names from Agtrials:
1. API on Agtrials from Herlin works fine and downloads from Agtrials is effective. Only the display of the terms is not yet in place and need discussions -Luca, Milko, Elizabeth
2. the full Cycle of trait name exchange flow  (matching Agtrials names to the CO and getting the join matching ones for submission) is not yet achieved - matching Agtrials variable names to the Ontology is not yet dynamic. It is still based on a download but Herlin and Glenn are  committed to get it automated -Herlin & Glenn

Getting Variety Passport linked to Agtrials

1. Herlin did the link to Luca's prototype obtain by scraping- Matching of names with the genebanks data


2.  Luca will improve the search through API to add flex ability of searching without including spaces, dash,etc like google in google search 
3. The questions is : Can we find all the varieties evaluated by breeders in the genebanks data? particularly for cassava? -  Glenn to ask in CIAT

4. the Crop ontology traits for cassava was developed by IITA but will be validated with CIAT in 2012- Elizabeth to ask Peter who to contact in CIAT and ask Bakare to search ICCAS for the variety names. 

GRSS cont. - sorghum - prepare a use case - Slides for JC

GENERATION Atlas - Maps of all ICRISAT sorghum accessions , need to show the accessions that compose the reference set

Get the passport for the sorghum ref sets through  an API is not possible on SINGER so was suggested to use a simple file downloaded from SINGER

How to get the list? 

Show a Figs type search or a simple mapping of accessions against the GRSS.

Question to debate next time : When can we do a prototype showing more integration of agtrials data/ontology/scraped data store?