rowid,dataset_title,publisher,author,dataset_issued,dataset_modified,dataset_description,source,info_url,start_date,end_date,file_title,download_url,format,file_description,file_created,file_modified,file_size,licence
832,Passengers in History,History Trust of South Australia,History Trust of South Australia,2016-06-28T00:52:30.371967,2021-09-08T23:20:52.626089,"_Passengers in History_ brings together two wonderful resources:
A passengers data base developed by the SA Maritime Museum,
A list of ships’ logs and diaries, titled Log of Logs.
The passengers data base was produced by staff and volunteers at the South Australian Maritime Museum over 20 years. It includes entries for 328,000 passengers and 20,000 voyages to South Australia between 1836 and 1961.
This data set is presented as a search index that can expose most open data formats.",data.sa.gov.au,https://data.sa.gov.au/data/dataset/ffa75eb8-475a-460b-91d1-714f269016d5,1836-01-01,1961-01-01,Passengers in History Search Index,https://data.history.sa.gov.au:8983/solr/passengers/select,API,"#Passengers in History API Documentation#
The Passengers in History data endpoint has been provided using an Apache Solr index (see https://lucene.apache.org/solr/) that uses a Drupal Solr 4.x schema (see https://www.drupal.org/project/search_api_solr). See [https://wiki.apache.org/solr/QueryParametersIndex](https://wiki.apache.org/solr/QueryParametersIndex) for more detail surrounding the query syntax.
An example of a basic implementation can be viewed at https://data.history.sa.gov.au/gallery_pih.php?type=vessel&startindex=0
Replace the type parameter with any of the types listed below to return associated images eg. https://data.history.sa.gov.au/gallery_pih.php?type=passenger&startindex=0
The example code can be downloaded from https://data.history.sa.gov.au/gallery_pih.php.txt
The open_data index contains 7 different types of content:
* voyage
* passenger
* vessel
* master
* port
* source
* organisation
Additionally, the photo index (photo_index) can be used to retrieve photos (specify the index using the index_id query parameter).
A voyage is centric to most other content types contains references to passengers, vessels and ports. A join query should be used to get information about related content (see https://wiki.apache.org/solr/Join for information surrounding the query syntax for joins).
The is_nid field is the unique identifier.
The number of rows per query is limited to 999 so the start and rows query parameters would need to be used to iterate over the entire result set.
There are many response formats available (see https://cwiki.apache.org/confluence/display/solr/Response+Writers). Just add the parameter 'wt' to the query and select the response format eg. json or csv. The default is xml.
##Examples##
A typical workflow might be:
Get the voyage/s that arrived in 1836:
[https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:open_data AND is_field_voyage_year:1836](https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:open_data AND is_field_voyage_year:1836)
Use a join to get the passengers for one of the voyages (use the voyage is_nid field)
[https://data.history.sa.gov.au:8983/solr/passengers/select?start=0&rows=50&q={!join from=im_field_passengers to=is_nid}index_id:open_data AND is_nid:944632](https://data.history.sa.gov.au:8983/solr/passengers/select?start=0&rows=50&q={!join from=im_field_passengers%20to=is_nid}index_id:open_data AND is_nid:944632)
Use a join to find the origin and destination of a particular voyage:
[https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_origin to=is_nid}index_id:open_data AND is_nid:944632](https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_origin to=is_nid}index_id:open_data AND is_nid:944632)
[https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_destination to=is_nid}index_id:open_data AND is_nid:944632](https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_destination to=is_nid}index_id:open_data AND is_nid:944632)
Use a join to find the vessel for the voyage:
[https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_vessel to=is_nid}index_id:open_data AND is_nid:944632](https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_vessel to=is_nid}index_id:open_data AND is_nid:944632)
Use a join to find photos of the vessel (using the vessel is_nid):
[https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:photo_index AND im_file_usage_list:920026](https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:photo_index AND im_file_usage_list:920026)
Use a join to find out where a vessel was built (using the vessel is_nid):
[https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_built_where to=is_nid}index_id:open_data AND is_nid:923958](https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_built_where to=is_nid}index_id:open_data AND is_nid:923958)
Use a join to find the built by organisation for a particular vessel:
[https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_field_built_by to=is_nid}index_id:open_data AND is_nid:923958](https://data.history.sa.gov.au:8983/solr/passengers/select?q={!join%20from=is_field_built_by%20to=is_nid}index_id:open_data%20AND%20is_nid:923958)
Get organisation details (using the organisation is_nid):
[https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:open_data AND ss_type:organisation AND is_nid:916199](https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:open_data%20AND%20ss_type:organisation%20AND%20is_nid:916199)
Or find all vessels built by the organisation (using the organisation is_nid):
[https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:open_data AND is_field_built_by:916199](https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:open_data%20AND%20is_field_built_by:916199)
##Other Useful Queries
Find all photos that are attached to a passenger:
data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=is_nid to=im_file_usage_list}ss_type:(passenger OR image)
Or find all passengers that have a photo attached (the reverse):
data.history.sa.gov.au:8983/solr/passengers/select?q={!join from=im_file_usage_list to=is_nid}ss_type:(passenger OR image)
All fields are searchable (see listing below). There are many search options using the Apache Solr query syntax. For instance to find all vessels that have the word ‘Adelaide’ in their tm_title field but not the word ‘port’:
data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:open_data AND ss_type:vessel AND tm_title:(adelaide -port)
In this case ss_type is specifically set to be ‘vessel’. The – represents negation of the term. There are many more options – refer to a good blog here [https://yonik.com/solr/query-syntax/](https://yonik.com/solr/query-syntax/)
Facet search on the voyage destination for a date range between 1834 and 1836:
https://data.history.sa.gov.au:8983/solr/passengers/select?rows=0&q={!join%20from=is_nid%20to=is_field_destination}index_id:open_data&fq=ds_field_depart_date:[1834-01-01T00:00:00Z%20TO%201836-12-31T12:59:59Z]&facet=on&facet.field=is_field_destination
In this case use a port search by node to get the titles of the ports eg.(using the first facet):
https://data.history.sa.gov.au:8983/solr/passengers/select?q=index_id:open_data%20AND%20is_nid:887339
The answer is most likely to be Sydney!
##Content Type Fields##
###Passenger
* is_nid
* ss_type
* tm_field_occupation
* tm_field_surname
* tm_field_firstname
* is_field_age
* is_arrival_year
###Voyage
* is_nid
* ss_type
* ds_field_depart_date
* is_field_destination
* tm_field_master
* is_field_origin
* is_field_vessel
* is_field_voyage_year
* tm_title
###Vessel
* is_nid
* ss_type
* tm_title
* tm_field_services
* im_field_sources
* is_field_year_end
* is_field_year_start
* is_field_built_by
* is_field_built_where
* is_field_built_year
* tm_field_demise
* tm_field_rig
* tm_field_dimensions
* tm_field_tonnage
###Source
* is_nid
* ss_type
* tm_title
* tm_field_location
* tm_field_reference
###Port
* is_nid
* ss_type
* tm_title
###Organisation
* is_nid
* ss_type
* tm_title
###Photos
* is_fid: Unique ID for a file
* tm_field_citation: Includes copyright information
* tm_field_file_image_title_text: Image title
* tm_field_location: Location of the image
* tm_field_source: Source of the image
* is_file_usage_count: Number of times this file is used in passengersinhistory.sa.gov.au
* im_file_usage_list: List of content that uses this file
* ss_url: Url to the file
",2016-06-28T10:54:22.341773,,,Creative Commons Attribution
917,South Australian Museum Terrestrial Invertebrate Collection,South Australian Museum,South Australian Museum,2013-03-21T05:57:20.220041,2016-07-05T03:45:43.422359,"The collection has Australia-wide and Indo-Pacific representation, plus some world specimens for comparative purposes. The collection comprises 1200000 pinned specimens, 450000 specimens in spirit and 20000 slides. There are 8670 holotypes, of which 5000 are from the A.M. Lea beetle collection. There are 23000 other types. In summary, the entomological collections Class Insecta comprise 662 Australian families and 85,961 known Australian species.
The arachnological collections comprise spiders (50000 specimens in alcohol), mites (25000 slide mounts and 20000 specimens in alcohol), scorpions (5000 specimens in alcohol) and myriapods.
Images from this collection are available on the Atlas of Living Australia.
The SA Museum manages this dataset using the KE EMu collection management system. It is interpreted into the [Darwin Core](http://rs.tdwg.org/dwc/index.htm) metadata schema (DwC) and semi-regularly exported to the [Atlas of Living Australia](http://www.ala.org.au/) (ALA) and the [Online Zoological Collections of Australian Museums](http://www.ozcam.org.au/) (OZCAM).
Data sourced from Australian museums on both the ALA and OZCAM should be identical, but on ALA they are combined with observational data from citizen science initiatives and other sources. Both of those sites make it possible to combine, interrogate and analyse data through web services such as the [Spatial Analysis Portal](http://spatial.ala.org.au/). In the Spatial Portal ALA data can be combined with meteorological and other environmental data sourced from and made accessible by relevant government agencies.
The ALA also has a fully documented [API](http://api.ala.org.au/)
Data about endangered species are either withheld from online publication, or coordinates or other data are obscured on the ALA and OZCAM. In those circumstances more specific information is available directly from SA Museum collection managers if it is genuinely required for research purposes.
SA Museum data can be downloaded in full from the Atlas of Living Australia, or broken down into discipline specific parts (e.g. Herpetology, Mammalogy etc). On download the ALA will request an email address (not mandatory) and a reason for download (mandatory) – this is required to track usage of the ALA data to help data providers determine priorities for upload and improvement. ",data.sa.gov.au,https://data.sa.gov.au/data/dataset/a81870be-82eb-4868-81e9-0a8d11622675,1860-2014,,Type Specimen images from the South Australian Museum Terrestrial Invertebrates collection,https://biocache.ala.org.au/occurrence/search?q=data_resource_uid:dr742,,"The South Australian Museum has been photographing the Type specimens in the Terrestrial Invertebrates collection. These images, of over 3500 specimens, can be accessed on the Atlas of Living Australia.",2016-06-28T11:25:21.631431,,,Creative Commons Attribution