Pure Harvest™ Software Data Import 2024-07-02 Pure Harvest can take an Uploaded data file (must be a .csv spreadsheet) and use it to create a temporary table. The purpose of this is to allow users to quickly load large data-sets into Pure Harvest without having to manually add each row. We are still working on what types of data-sets we can import, but we will be supporting Fields, Crops, and Varieties. There are many circumstances that will need to be worked out, and we will resolve them if and when customers have a data-file they would like to upload. Creating the Data File When creating the spreadsheet, make sure that all columns are formatted as text, not as numbers or dates or any other type of column. Many times in spreadsheets, the value displayed in the formatted column may not be the actual value in the column. For instance, column formatted to show 2 decimals may display “2.00” but the actual value of the column would just be “2” which means when the file is loaded the decimals would not be included. To save the spreadsheet as a CSV file, there is usually an option when you choose “Save as” to set the file type to csv. Column Headers In order for the file to be uploaded correctly, each column must have a column header. Do not use any special characters in column headings such as ?, #, !, &, or $. The upload will fail since the system cannot use those characters when creating a temporary table. "" A nasty “” invisible character may be introduced to the csv by Excel. You can burn up a lot of time trying to figure out why your spreadsheet won't load if this char is in the file. https://www.freecodecamp.org/news/aquick-tale-about-feff-the-invisible-character-cd25cd4630e7/ Uploading a File From the Workflow → Data Import page, select the type of data file you are uploading, then click the choose file button After choosing the file to import, press the Upload button, PH will use the uploaded csv file to create a temporary table. Accepting the imported fields Fields will be accepted into the main fields table based on the app-id in the file. This can be done multiple times because the previous set is deleted before it is reloaded. The upload script modifies the names of the columns in the temporary table to match the columns in Pure Harvest. You can export the data back to your PC to see what we did. You can use these column names for your next import. [Export to CSV] Immediate download of your corrected data file. Upon Successful Import You will see feedback like this if your upload was successful. SUMMARY REMARKS Validate Fields Join okay: Table contains 55 rows and 55 valid rows were found. insert into fields okay: Inserted 55 of 55 show_rejected_fields All rows loaded. Inserted 0 rows to web_shared_fields. Field Data In order for the fields to be created, the Application (given by app_id) must already exist, with no current fields on it. All crops & varieties must already exist in your database. Required Columns These columns are currently required for a successful upload: $csv_header=“by_producer_assoc_id,app_id,field_number,seed_class,county,state, acres_applied_for,crop_name,variety_name,date_planted,gps_location field_zone,producer_field_name,insp_code, inspections_needed,previous_usage,crop_year,grower_name, by_grower_assoc_id,identity,shared_with”; We automatically change some column names to match the Pure Harvest tables: Column name provided Automatically changed to PH column name crop crop_name crop_produced is_crop_produced field_name producer_field_name grower grower_name applicant producer submittor producer_name variety variety_name plant_date date_planted assoc_id by_producer_assoc_id grower_id by_grower_assoc_id acres_applied acres_applied_for gps_coordinates gps_location Extra Columns Additional columns can be included, but are not required to load the field data Just a few of the additional columns are listed below, contact support for a full list. • • • • • • date_planted grower_name remark insp_code (meaning Inspection Plan Code) inspections_needed do_send_receipt If you choose to include the insp_code, the Inspection plan for that code must already exist. Variety Data Only new varieties can be uploaded, If any variety or synonym already exists in the system, it will be skipped by the import. A variety csv file must have the following columns: • • • crop_name variety synonym Additional columns can be included, but are not required to load the field data Just a few of the additional columns are listed below, contact support for a full list. • • • owner_id originator_id is_inactive • • • • • • • • • • year_available description additional_notes print_name print_description print_producer print_acres is_title_v protection_status pvp_94_status If you track Varieties by program or class, you can include the columns: • • seed_class program If you track variety characteristics in Pure Harvest, you can include one column for each characteristic name. The column names MUST match the names of the characteristics you have defined in Pure Harvest. One column per characteristic. Once you have uploaded your data file, contact support to have the data transferred from the temporary table to the actual varieties table(s) Internal Processing Each uploaded file must be validated before accepting to the internal tables. See tech notes imp(fields,varieties).notes.txt elsewhere. Since each incoming file is unique, we have done the import manually.