Get timeseries from Kiwis (BOM)

This expects a ts_id or ts_path (but not both) to identify the timeseries (>= 1) to pull. ts_path can be generated on the fly with wildcards, but isn't straightforward to parse- see the output of getTimeseriesList(). Though ts_id would need to be extracted from getTimeseriesList(), and so may not be any easier to get programatically. The equivalent state (Hydstra) function is get_ts_traces() (to a close approximation). If period is used, only one (or none) of start_time and end_time can be used. If neither is used, it gets the most recent period.

Usage

getTimeseriesValues(
  portal,
  ts_id = NULL,
  ts_path = NULL,
  start_time = NULL,
  end_time = NULL,
  period = NULL,
  returnfields = "default",
  meta_returnfields = "default",
  extra_list = list(NULL),
  return_timezone = "UTC"
)

Arguments

portal

URL to Kisters KiWIS database.

ts_id

timeseries id, typically found from getTimeseriesList()

ts_path

timeseries path, which can be constructed, including wildcards, e.g. ts_path = '*/A4260505/Water*/*DailyMean' Gets the daily means for all 'Water' variables at gauge A4260505, which might include Level, Discharge, Temperature, etc..

start_time

character or date or date time for the start in database default timezone. Default NULL.

end_time

character or date or date time for the end in database default timezone. Default NULL.

period

character, default NULL. The special case 'complete' returns the full set of data. Otherwise, beginning with 'P', followed by numbers and characers indicating timespan, e.g. 'P2W'. See documentation.

returnfields

return fields for the data itself. Default is c('Timestamp', 'Value', 'Quality Code'). Full list from Kisters docs)

meta_returnfields

return fields about the variable and site. seems to be able to access most of what getTimeseriesList() has in its returnfields. Full list from Kisters docs)

extra_list

a named list, see getStationList(), with a special note that here we can include a 'timezone' argument that determines the timezone the API returns in. This is dangerous, since the API ingests dates in its own default timezone and that is inferred from the return in the absence of the ability to extract it. Thus, including a timezone in extra_list may yield unexpected outcomes when requesting dates. A better option is to use return_timezone to adjust the return values. That said, it may be that some databases return gauge-local tzs, which won't be allowed to be concatenated. A solution would be to just work in UTC with timezone = 'UTC' in extralist to make all outputs on the same tz.

return_timezone

character in OlsonNames() or one of three special cases: 'db_default', 'char' or 'raw'. Default 'UTC'. If 'db_default', uses the API default. BOM defaults to +10. If 'char' or 'raw', returns the time column as-is from the API (A string in the format 'YYYY-MM-DDTHH:MM:SS+TZ')

Value

a tibble of the timeseries values. Times are POSIXct in UTC by default.

Details

Timezone note: BOM documentation says data is returned in local time. This is true on the web interface, but not the API. The API defaults to +10, but we can choose, so here we default to return_timezone = 'UTC' for consistency. Further, start_time and end_time must be in database-local time; setting a different return time, either here or directly to the API with a timezone argument in extra_list, does not affect the interpretation of these times. getTimeseriesList() returns the database_timezone to make this easier. fetch_kiwis_timeseries() handles some of this work automatically.

Data size note: If you request too much data, this will throw a 500 error. "Maximum number of timeseries values surpassed. Please narrow your request. Limit is: 250000" Try to reduce the amount of data being requested in a single call. The limit in reality seems much lower; I frequently encounter it when asking for abou 120,000 records