Type:	Package
Title:	Approximate POMDP Planning Software
Version:	0.6.16
Description:	A toolkit for Partially Observed Markov Decision Processes (POMDP). Provides bindings to C++ libraries implementing the algorithm SARSOP (Successive Approximations of the Reachable Space under Optimal Policies) and described in Kurniawati et al (2008), <doi:10.15607/RSS.2008.IV.009>. This package also provides a high-level interface for generating, solving and simulating POMDP problems and their solutions.
License:	GPL-2
URL:	https://github.com/boettiger-lab/sarsop
BugReports:	https://github.com/boettiger-lab/sarsop/issues
RoxygenNote:	7.1.1
Imports:	xml2, parallel, processx, digest, Matrix
Suggests:	testthat, roxygen2, knitr, covr, spelling
LinkingTo:	BH
Encoding:	UTF-8
Language:	en-US
SystemRequirements:	mallinfo, hence Linux, MacOS or Windows
NeedsCompilation:	yes
Packaged:	2025-04-15 17:11:33 UTC; jovyan
Author:	Carl Boettiger [cre, aut, cph], Jeroen Ooms [aut], Milad Memarzadeh [aut], Hanna Kurniawati [ctb, cph], David Hsu [ctb, cph], Hanna Kurniawati [ctb, cph], Wee Sun Lee [ctb, cph], Yanzhu Du [ctb], Xan Huang [ctb], Trey Smith [ctb, cph], Tony Cassandra [ctb, cph], Lee Thomason [ctb, cph], Carl Kindman [ctb, cph], Le Trong Dao [ctb, cph], Amit Jain [ctb, cph], Rong Nan [ctb, cph], Ulrich Drepper [ctb], Free Software Foundation [cph], Tyge Lovset [ctb, cph], Yves Berquin [ctb, cph], Benjamin Grüdelbach [ctb], RSA Data Security, Inc. [cph]
Maintainer:	Carl Boettiger <cboettig@gmail.com>
Repository:	CRAN
Date/Publication:	2025-04-16 04:50:08 UTC

alphas_from_log

Description

Read alpha vectors from a log file.

Usage

alphas_from_log(meta, log_dir = ".")

Arguments

Value

a list with a matrix of alpha vectors for each entry in the provided metadata (as returned by sarsop).

Examples

 # takes > 5s

source(system.file("examples/fisheries-ex.R", package = "sarsop"))
log = tempfile()
alpha <- sarsop(transition, observation, reward, discount, precision = 10,
                log_dir = log)

test the APPL binaries

Description

Asserts that the C++ binaries for appl have been compiled successfully

Usage

assert_has_appl()

Value

Will return TRUE if binaries are installed and can be located and executed, and FALSE otherwise.

Examples

assert_has_appl()

compute_policy

Description

Derive the corresponding policy function from the alpha vectors

Usage

compute_policy(
  alpha,
  transition,
  observation,
  reward,
  state_prior = rep(1, dim(observation)[[1]])/dim(observation)[[1]],
  a_0 = 1
)

Arguments

Value

a data frame providing the optimal policy (choice of action) and corresponding value of the action for each possible belief state

Examples

m <- fisheries_matrices()
 ## Takes > 5s
if(assert_has_appl()){
alpha <- sarsop(m$transition, m$observation, m$reward, 0.95, precision = 10)
compute_policy(alpha, m$transition, m$observation, m$reward)
}

f from log

Description

Read transition function from log

Usage

f_from_log(meta)

Arguments

Details

note this function is unique to the fisheries example problem and assumes that sarsop call is run with logging specifying a column "model" that contains either the string "ricker" (corresponding to a Ricker-type growth function) or "allen" (corresponding to an Allen-type.)

Value

the growth function associated with the model indicated.

Examples

 # takes > 5s

source(system.file("examples/fisheries-ex.R", package = "sarsop"))
log = tempfile()
alpha <- sarsop(transition, observation, reward, discount, precision = 10,
                log_dir = log)

fisheries_matrices

Description

Initialize the transition, observation, and reward matrices given a transition function, reward function, and state space

Usage

fisheries_matrices(
  states = 0:20,
  actions = states,
  observed_states = states,
  reward_fn = function(x, a) pmin(x, a),
  f = ricker(1, 15),
  sigma_g = 0.1,
  sigma_m = 0.1,
  noise = c("rescaled-lognormal", "lognormal", "uniform", "normal")
)

Arguments

Details

assumes log-normally distributed observation errors and process errors

Value

list of transition matrix, observation matrix, and reward matrix

Examples

m <- fisheries_matrices()

hindcast_pomdp

Description

Compare historical actions to what pomdp recommendation would have been.

Usage

hindcast_pomdp(
  transition,
  observation,
  reward,
  discount,
  obs,
  action,
  state_prior = rep(1, dim(observation)[[1]])/dim(observation)[[1]],
  alpha = NULL,
  ...
)

Arguments

Value

a list, containing: a data frame with columns for time, obs, action, and optimal action, and an array containing the posterior belief distribution at each time t

Examples

m <- fisheries_matrices()
 ## Takes > 5s
if(assert_has_appl()){
alpha <- sarsop(m$transition, m$observation, m$reward, 0.95, precision = 10)
sim <- hindcast_pomdp(m$transition, m$observation, m$reward, 0.95,
                     obs = rnorm(21, 15, .1), action = rep(1, 21),
                     alpha = alpha)

}

meta from log

Description

load metadata from a log file

Usage

meta_from_log(
  parameters,
  log_dir = ".",
  metafile = paste0(log_dir, "/meta.csv")
)

Arguments

Value

a data.frame with the rows of the matching metadata.

Examples

 # takes > 5s

source(system.file("examples/fisheries-ex.R", package = "sarsop"))
log = tempfile()
alpha <- sarsop(transition, observation, reward, discount, precision = 10,
                log_dir = log)

model from log

Description

Read model details from log file

Usage

models_from_log(meta, reward_fn = function(x, h) pmin(x, h))

Arguments

Details

assumes transition can be determined by the f_from_log function, which is specific to the fisheries example

Value

a list with an element for each row in the requested meta data frame, which itself is a list of the three matrices: transition, observation, and reward, defining the pomdp problem.

Examples

 # takes > 5s

source(system.file("examples/fisheries-ex.R", package = "sarsop"))
log = tempfile()
alpha <- sarsop(transition, observation, reward, discount, precision = 10,
                log_dir = log)

APPL wrappers

Description

Wrappers for the APPL executables. The pomdpsol function solves a model file and returns the path to the output policy file.

Usage

pomdpsol(
  model,
  output = tempfile(),
  precision = 0.001,
  timeout = NULL,
  fast = FALSE,
  randomization = FALSE,
  memory = NULL,
  improvementConstant = NULL,
  timeInterval = NULL,
  stdout = tempfile(),
  stderr = tempfile(),
  spinner = TRUE
)

polgraph(
  model,
  policy,
  output = tempfile(),
  max_depth = 3,
  max_branches = 10,
  min_prob = 0.001,
  stdout = "",
  spinner = TRUE
)

pomdpsim(
  model,
  policy,
  output = tempfile(),
  steps = 100,
  simulations = 3,
  stdout = "",
  spinner = TRUE
)

pomdpeval(
  model,
  policy,
  output = tempfile(),
  steps = 100,
  simulations = 3,
  stdout = "",
  spinner = TRUE
)

pomdpconvert(model, stdout = "", spinner = TRUE)

Arguments

Examples

if(assert_has_appl()){
  model <- system.file("models", "example.pomdp", package = "sarsop")
  policy <- tempfile(fileext = ".policyx")
  pomdpsol(model, output = policy, timeout = 1)

# Other tools
  evaluation <- pomdpeval(model, policy, stdout = FALSE)
  graph <- polgraph(model, policy, stdout = FALSE)
  simulations <- pomdpsim(model, policy, stdout = FALSE)
}

read_policyx

Description

read a .policyx file created by SARSOP and return alpha vectors and associated actions.

Usage

read_policyx(file = "output.policyx")

Arguments

Value

a list, first element "vectors" is an n_states x n_vectors array of alpha vectors, second element is a numeric vector "action" of length n_vectors whose i'th element indicates the action corresponding to the i'th alpha vector (column) in the vectors array.

Examples

f <- system.file("extdata", "out.policy", package="sarsop", mustWork = TRUE)
policy <- read_policyx(f)

sarsop

Description

sarsop wraps the tasks of writing the pomdpx file defining the problem, running the pomdsol (SARSOP) algorithm in C++, and then reading the resulting policy file back into R. The returned alpha vectors and alpha_action information is then transformed into a more generic, user-friendly representation as a matrix whose columns correspond to actions and rows to states. This function can thus be used at the heart of most pomdp applications.

Usage

sarsop(
  transition,
  observation,
  reward,
  discount,
  state_prior = rep(1, dim(observation)[[1]])/dim(observation)[[1]],
  verbose = TRUE,
  log_dir = tempdir(),
  log_data = NULL,
  cache = TRUE,
  ...
)

Arguments

Value

a matrix of alpha vectors. Column index indicates action associated with the alpha vector, (1:n_actions), rows indicate system state, x. Actions for which no alpha vector was found are included as all -Inf, since such actions are not optimal regardless of belief, and thus have no corresponding alpha vectors in alpha_action list.

Examples

 ## Takes > 5s
## Use example code to generate matrices for pomdp problem:
source(system.file("examples/fisheries-ex.R", package = "sarsop"))
alpha <- sarsop(transition, observation, reward, discount, precision = 10)
compute_policy(alpha, transition, observation, reward)

simulate a POMDP

Description

Simulate a POMDP given the appropriate matrices.

Usage

sim_pomdp(
  transition,
  observation,
  reward,
  discount,
  state_prior = rep(1, dim(observation)[[1]])/dim(observation)[[1]],
  x0,
  a0 = 1,
  Tmax = 20,
  policy = NULL,
  alpha = NULL,
  reps = 1,
  ...
)

Arguments

Details

simulation assumes the following order of updating: For system in state[t] at time t, an observation of the system obs[t] is made, and then action[t] is based on that observation and the given policy, returning (discounted) reward[t].

Value

a data frame with columns for time, state, obs, action, and (discounted) value.

Examples

m <- fisheries_matrices()
discount <- 0.95
 ## Takes > 5s
if(assert_has_appl()){
alpha <- sarsop(m$transition, m$observation, m$reward, discount, precision = 10)
sim <- sim_pomdp(m$transition, m$observation, m$reward, discount,
                 x0 = 5, Tmax = 20, alpha = alpha)

}

write pomdpx files

Description

A POMDPX file specifies a POMDP problem in terms of the transition, observation, and reward matrices, the discount factor, and the initial belief.

Usage

write_pomdpx(
  P,
  O,
  R,
  gamma,
  b = rep(1/dim(O)[1], dim(O)[1]),
  file = "input.pomdpx",
  digits = 12,
  digits2 = 12,
  format = "f"
)

Arguments

Examples

m <- fisheries_matrices()
f <- tempfile()
write_pomdpx(m$transition, m$observation, m$reward, 0.95,
             file = f)