JSON API coding

Author

Galen Holt

The issue

I’m working on an API that uses JSON in the body, but getting it to come out right with square brackets, curly brackets, commas, etc where they’re supposed to be has been trial and error. I’m going to put what I’ve figured out here. I’m using examples from the {hydrogauge} package, but the main point is to show how to get different sorts of output.

Using the httr2 examples with req_dry_run to see what the request looks like and check the formats.

library(httr2)
req <- request("http://httpbin.org/post")

Simple key-value

To pass simple one to one key-value pairs, wrapped in {} use a list.

params <- list("function" = 'get_db_info',
               "version" = "3")

req %>%
  req_body_json(params) %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 40

{"function":"get_db_info","version":"3"}

Nested key-value

To pass nested key-value pairs, use nested lists

params <- list("function" = 'get_variable_list',
               "version" = "1",
               "params" = list("site_list" = '123abc',
                               "datasource" = "A"))
req %>%
  req_body_json(params) %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 95

{"function":"get_variable_list","version":"1","params":{"site_list":"123abc","datasource":"A"}}

Comma-separated strings

These cannot be created with c(), because that does something else (square brackets- see below).

params <- list("function" = 'get_datasources_by_site',
               "version" = "1",
               "params" = list("site_list" = '233217, 405328, 405331'))
req %>%
  req_body_json(params) %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 100

{"function":"get_datasources_by_site","version":"1","params":{"site_list":"233217, 405328, 405331"}}

Square brackets

To get square brackets, we need a vector. So, typically c() the bits together in the call (or previously).

params <- list("function" = 'get_sites_by_datasource',
               "version" = "1",
               "params" = list("datasources" = c('A', 'TELEM')))

req %>%
  req_body_json(params) %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 91

{"function":"get_sites_by_datasource","version":"1","params":{"datasources":["A","TELEM"]}}

Double-square brackets

To get patterns like [['a', 'b'],['c', 'd']], use a matrix (and I think maybe a df). Which makes sense if we think of that as a group of vectors. The pattern is [[row1], [row2], [row_n]].

topleft <- c('-35', '148')
bottomright <- c('-36', '149')

rectbox <- rbind(topleft, bottomright)
params <- list("function" = 'get_db_info',
               "version" = "3",
               "params" = list("table_name" = "site",
                               "return_type" = "array",
                               "geo_filter" = list('rectangle' = rectbox)))
req %>%
  req_body_json(params) %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 150

{"function":"get_db_info","version":"3","params":{"table_name":"site","return_type":"array","geo_filter":{"rectangle":[["-35","148"],["-36","149"]]}}}

Using a df

Gives some horrible combination of curly and square braces including column and row names.

rectdf <- data.frame(rectbox)
params <- list("function" = 'get_db_info',
               "version" = "3",
               "params" = list("table_name" = "site",
                               "return_type" = "array",
                               "geo_filter" = list('rectangle' = rectdf)))
req %>%
  req_body_json(params) %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 208

{"function":"get_db_info","version":"3","params":{"table_name":"site","return_type":"array","geo_filter":{"rectangle":[{"X1":"-35","X2":"148","_row":"topleft"},{"X1":"-36","X2":"149","_row":"bottomright"}]}}}

Tibbles aren’t really any different, but the names are a bit cleaner

rectdf <- tibble::as_tibble(rectbox)

Warning: The `x` argument of `as_tibble.matrix()` must have unique column names if
`.name_repair` is omitted as of tibble 2.0.0.
ℹ Using compatibility `.name_repair`.

params <- list("function" = 'get_db_info',
               "version" = "3",
               "params" = list("table_name" = "site",
                               "return_type" = "array",
                               "geo_filter" = list('rectangle' = rectdf)))
req %>%
  req_body_json(params) %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 170

{"function":"get_db_info","version":"3","params":{"table_name":"site","return_type":"array","geo_filter":{"rectangle":[{"V1":"-35","V2":"148"},{"V1":"-36","V2":"149"}]}}}

Orientation of dfs and matrices

There are arguments to toJSON that alter how matrices and dfs get parsed. Matrices are by default row-wise, but we can change to cols (e.g. [['col1'], ['col2']] with matrix = 'columnmajor'.

params <- list("function" = 'get_db_info',
               "version" = "3",
               "params" = list("table_name" = "site",
                               "return_type" = "array",
                               "geo_filter" = list('rectangle' = rectbox)))
req %>%
  req_body_json(params, matrix = 'columnmajor') %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 150

{"function":"get_db_info","version":"3","params":{"table_name":"site","return_type":"array","geo_filter":{"rectangle":[["-35","-36"],["148","149"]]}}}

Similarly, we can alter how dfs work, which might actually be fairly useful in the way it handles named columns especially. The default (above) is dataframe = 'rows' , which is kind of a mess (or at least not how my brain parses what a dataframe means). But dataframe = 'columns' ends up with named vectors. I don’t currently need that, but it sure makes more sense.

params <- list("function" = 'get_db_info',
               "version" = "3",
               "params" = list("table_name" = "site",
                               "return_type" = "array",
                               "geo_filter" = list('rectangle' = rectdf)))
req %>%
  req_body_json(params, dataframe = 'columns') %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 160

{"function":"get_db_info","version":"3","params":{"table_name":"site","return_type":"array","geo_filter":{"rectangle":{"V1":["-35","-36"],"V2":["148","149"]}}}}

Using dataframe = 'values' is again a confusing list.

params <- list("function" = 'get_db_info',
               "version" = "3",
               "params" = list("table_name" = "site",
                               "return_type" = "array",
                               "geo_filter" = list('rectangle' = rectdf)))
req %>%
  req_body_json(params, datafraem = 'values') %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 170

{"function":"get_db_info","version":"3","params":{"table_name":"site","return_type":"array","geo_filter":{"rectangle":[{"V1":"-35","V2":"148"},{"V1":"-36","V2":"149"}]}}}

Square brackets around curly

To get square brackets around multiple sets of curlies, e.g. `[{‘key’: ‘value’}, {‘key2’: ‘value2’}], use a list of lists.

params <- list("function" = 'get_db_info',
               "version" = "3",
               "params" = list("table_name" = "site",
                               "return_type" = "array",
                               
                               "complex_filter" = list(list('fieldname' = 'stntype', 
                                                       'value' = 'HYD'),
                                                   list('combine' = 'OR',
                                                        'fieldname' = 'stntype',
                                                        'value' = 'VIR'))))
req %>%
  req_body_json(params, datafraem = 'values') %>%
  req_dry_run()

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: httr2/1.0.3 r-curl/5.2.2 libcurl/8.3.0
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/json
Content-Length: 203

{"function":"get_db_info","version":"3","params":{"table_name":"site","return_type":"array","complex_filter":[{"fieldname":"stntype","value":"HYD"},{"combine":"OR","fieldname":"stntype","value":"VIR"}]}}