r-universe Lifecycle: experimental License: MIT

gpumetropolis

A generic Metropolis-Hastings sampler for Markov chain Monte Carlo. The user declares a model by writing its log-likelihood and log-prior as ordinary R formulas; the package compiles them to a portable kernel that runs on the CPU and the GPU from one source. The sampler advances many independent chains in one batched pass. It occupies a niche that no general R repository fills: a generic MCMC sampler with a vendor-agnostic GPU-portable kernel.

The model expression is compiled to a stack-machine bytecode that a single CubeCL kernel interprets, so any model in the supported operation set runs on the CPU and GPU with no runtime code generation. CubeCL compiles that one kernel for the CUDA, Vulkan and CPU backends the package exposes.

The package is under active development. See ## Project status below.

Installation

The package is distributed through R-universe. It is not on CRAN: the vendored Rust dependency tree, dominated by the CUDA and Vulkan stacks, exceeds the CRAN tarball-size limit, so R-universe is its home.

Installing requires a Rust toolchain (cargo, rustc >= 1.85); see https://rustup.rs.

install.packages("gpumetropolis",
                 repos = c("https://pcbrom.r-universe.dev",
                           "https://cloud.r-project.org"))

A source install detects the GPU toolchains present on the build host and enables the matching backends without the user passing any flag. The CUDA backend is added when nvcc is on PATH; the Vulkan backend, when vulkaninfo is on PATH. Hosts with no GPU toolchain build CPU-only.

The pre-built binaries on R-universe are CPU-only because the build runners have no GPU toolkits installed; to obtain a GPU-enabled binary, install from source on a host that has the toolkit:

install.packages("gpumetropolis", type = "source",
                 repos = c("https://pcbrom.r-universe.dev",
                           "https://cloud.r-project.org"))

The auto-detection can be overridden with GPUMETROPOLIS_BACKENDS (auto, cpu, cuda, vulkan, or a comma list) and the per-backend toggles GPUMETROPOLIS_CUDA and GPUMETROPOLIS_VULKAN (0 or 1).

Load package

library(gpumetropolis)

Quick start

A model is declared by writing its per-observation log-likelihood as a one-sided formula in the parameter and data names. The example below is the Gaussian mean with known standard deviation 2 and a flat prior; its posterior is available in closed form, which makes it a clean check that the sampler recovers known parameters.

set.seed(1)
y <- rnorm(20000, mean = 3.4, sd = 2)

model <- gpum_model(
  loglik = ~ -((y - mu)^2) / 8,   # sigma = 2, so 0.5 / sigma^2 = 1 / 8
  params = "mu",
  data = "y"
)

fit <- gpu_metropolis(model, data = list(y = y),
                      n_iter = 3000, n_chains = 8)
fit

# the same declaration runs on a specific backend; the default
# `backend = "auto"` picks CUDA when its feature is compiled in, then
# Vulkan, and falls back to the CPU.
fit_gpu <- gpu_metropolis(model, data = list(y = y),
                          n_iter = 3000, n_chains = 8, backend = "cuda")

# parallel tempering for multimodal targets, since v0.3.0
fit_pt <- gpu_metropolis(model, data = list(y = y),
                         n_iter = 3000, n_chains = 8, method = "pt")

Since v0.2.0 the warmup is adaptive by default and is discarded before fit$draws is returned; proposal_sd is only the warmup seed and the per-chain proposal scale adapts to its own geometry, so most users no longer need to tune it. Set adapt = FALSE to recover the trim-only warmup of 0.1.x.

The supported operations in a formula are +, -, *, /, ^, unary -, and exp, log, sqrt, lgamma. A symbol that is not a declared parameter or data column, or a function outside this set, is rejected at compile time with a clear error.

How the pieces fit

The package is organised in utility groups around one engine. A model is declared by one of the constructors, sampled by gpu_metropolis() on the chosen backend, and the resulting fit flows into the diagnostic, inference and visual groups. The dependence workflow is a pipeline of its own: each column gets an automatically selected marginal, whose fitted CDF transforms it to the unit square for the copula, completing Sklar's factorisation.

flowchart TB
    data["data: a column, a data.frame, or pseudo-observations"]

    subgraph DECL["Model declaration"]
        gm["gpum_model (formula to bytecode)"]
        glm["gpum_lm (exact conjugate Gaussian)"]
        gts["gpum_ts_model (Markov, time series)"]
    end

    subgraph MARG["Marginals and dependence (Sklar pipeline)"]
        gfc["gpum_fit_catalog (auto-select marginal)"]
        mcdf["marginal_cdf"]
        gcop["gpum_copula (auto-select family)"]
        kt["kendall_tau, tail_dependence"]
    end

    subgraph ENGINE["Sampling engine (one portable kernel)"]
        gmet["gpu_metropolis: rwm, pt, de, mala, exact"]
        back["backends: cpu, cuda, vulkan"]
    end

    fit["gpum_fit"]

    subgraph DIAG["Diagnostics"]
        gd["gpum_diagnose, rhat, ess"]
    end

    subgraph INFER["Bayesian inference and testing"]
        est["hdi, gpum_hypothesis, gpum_rope"]
        pred["gpum_waic, gpum_loo, gpum_lfo"]
        evid["gpum_evidence, gpum_bayes_factor"]
        crlb["gpum_crlb (Cramer-Rao bound)"]
    end

    subgraph VIS["Visual inspection"]
        pairs["gpum_pairs, gpum_surface, gpum_region"]
        ppc["gpum_ppc, gpum_density_compare"]
        plt["plot methods"]
    end

    data --> DECL
    data --> gfc
    gfc --> mcdf --> gcop
    gm --> gmet
    glm --> gmet
    gts --> gmet
    gcop --> gmet
    gmet --- back
    gmet --> fit
    gcop --> kt
    fit --> DIAG
    fit --> INFER
    fit --> VIS

    style DECL fill:#e3f0fb,stroke:#2b6cb0
    style MARG fill:#eef7ee,stroke:#2f855a
    style ENGINE fill:#eae6f7,stroke:#6b46c1
    style DIAG fill:#fdf2e6,stroke:#c05621
    style INFER fill:#fdf2e6,stroke:#c05621
    style VIS fill:#fdf2e6,stroke:#c05621

Features and the problem each solves

Every function is listed with the problem it addresses, grouped as in the diagram above.

Model declaration

Function What it does Problem it solves
gpum_model() declares any model by its per-observation log-density as a formula, compiled to portable bytecode run one model specification on the CPU, CUDA and Vulkan backends without writing a kernel
gpum_lm() fits the Gaussian linear model under a conjugate prior, sampling the closed-form posterior exactly draw independent samples (effective sample size equal to the draw count) where a Gibbs specialist would otherwise lead on conjugate regressions
gpum_ts_model() declares a Markov model of order p by conditional factorisation, with exogenous covariates fit dependent, non-i.i.d. data (time series) with the same engine

Sampling engine

Function What it does Problem it solves
gpu_metropolis(method = "rwm") adaptive random-walk Metropolis, warmup tunes a full-covariance proposal general-purpose sampling with no manual proposal tuning
gpu_metropolis(method = "pt") parallel tempering, one chain per temperature with swaps cross the barriers of a multimodal target a random walk cannot
gpu_metropolis(method = "de") Differential Evolution MCMC, proposal from the ensemble geometry correlated targets, with no explicit covariance to tune
gpu_metropolis(method = "mala") Metropolis-adjusted Langevin, gradient by reverse-mode AD of the model mix efficiently as the dimension grows, where a random walk degrades
backend = "cpu" / "cuda" / "vulkan" one portable kernel across the three backends vendor-agnostic GPU acceleration for many chains or a large data set, with a CPU fallback

Marginals and dependence (the Sklar pipeline)

Function What it does Problem it solves
gpum_fit_catalog() detects a column's support and modality, fits every eligible family, ranks by WAIC choose the marginal distribution of a column automatically instead of guessing
marginal_cdf() the fitted probability-integral transform of the selected marginal carry a column onto the unit square for the copula
gpum_copula() fits a bivariate copula and auto-selects the family model the dependence structure separately from the margins, and read where in the tails the coupling lives
kendall_tau(), tail_dependence() posterior of the family-free dependence summaries compare dependence across families and quantify extreme co-movement

Diagnostics

Function What it does Problem it solves
gpum_diagnose() one-call per-parameter table plus a multi-panel plot and a convergence verdict answer, in one call, whether the run converged
rhat(), ess() split R-hat and effective sample size the standard convergence and precision measures on their own

Bayesian inference and testing

Function What it does Problem it solves
hdi() highest density interval of a posterior report the credible interval directly from the draws
gpum_hypothesis() posterior probability that a parameter lies in a stated region answer a hypothesis as a probability, not a p-value
gpum_rope() probability inside a region of practical equivalence (Kruschke) decide practical, not just statistical, significance
gpum_waic(), gpum_loo() pointwise predictive comparison for independent data compare models by out-of-sample accuracy without a marginal likelihood
gpum_lfo() exact leave-future-out cross-validation compare models on dependent data, where leave-one-out is not valid
gpum_evidence(), gpum_bayes_factor() marginal likelihood by thermodynamic integration Bayes factors for model comparison, reusing the sampler
gpum_crlb() observed Fisher information and the Cramer-Rao bound by exact-gradient differences check whether the posterior attains the information bound, and meet the classical estimator

Visual inspection

Function What it does Problem it solves
gpum_pairs(), gpum_surface(), gpum_region() marginal and bivariate posterior views with credible regions see the joint posterior and its convergence regions
gpum_ppc(), gpum_density_compare() overlay the model-generated distribution on the observed data check the fit by observed against generated, on one graph
plot() methods a visual for every fitted object (copula, catalogue, hypothesis, ROPE, CRLB) inspect each result, paired with its text diagnostic

Convergence diagnostics

Since v0.2.0 the one-call entry point is gpum_diagnose(fit): it prints a per-parameter table (mean, standard deviation, the 2.5%, 50% and 97.5% quantiles, split R-hat, effective sample size and Monte Carlo standard error) with a convergence verdict from the canonical thresholds, and opens a multi-panel diagnostic plot per parameter; when the warmup was adaptive an extra row shows the per-chain acceptance over the warmup batches with the asymptotic optimum as a reference, and on a parallel-tempering fit a further row shows the swap acceptance per pair.

gpum_diagnose(fit)

The individual building blocks are also exported. Equivalence for MCMC is distributional, never bit-exact, because the algorithm is stochastic.

draws <- fit$draws[, , "mu"]   # iterations by chains
rhat(draws, warmup = 0)        # split potential scale reduction factor
ess(draws, warmup = 0)         # effective sample size, Geyer estimator

# distributional equivalence between the CPU and GPU runs
ks_equivalence(fit$draws[, , "mu"], fit_gpu$draws[, , "mu"])

ks_equivalence thins the pooled draws down to the effective sample size before the Kolmogorov-Smirnov test, because that test assumes independent draws while MCMC output is autocorrelated.

Since v0.4.0, gpum_crlb(fit, data) adds an optional Cramer-Rao reference: it forms the observed Fisher information by a finite-difference Hessian of the compiled log-likelihood and inverts it to the lower bound on the covariance of an unbiased estimator, reporting it beside the empirical posterior spread. The bound is a frequentist asymptotic, prior-free reference; under the regularity of the Bernstein-von Mises theorem the posterior covariance approaches it, so an agreement is a check that the sampler recovered the information-bound geometry. The function withholds the comparison, with a stated reason, when a model has no data term, when the largest R-hat signals a multimodal or unconverged posterior, or when the observed information is not positive definite.

Also since v0.4.0, a formal Bayesian decision and comparison layer reads off the fit. From the draws alone, gpum_hypothesis(fit, parameter, lower, upper) gives the posterior probability of an interval or one-sided hypothesis, and gpum_rope(fit, parameter, rope) applies the region-of-practical-equivalence rule against the highest density interval. For predictive model comparison without a marginal likelihood, gpum_waic(fit, data) and gpum_loo(fit, data) report WAIC and Pareto-smoothed importance-sampling leave-one-out, the latter through the loo package. For the weight of evidence, gpum_evidence(model, data) estimates the log marginal likelihood by thermodynamic integration from the prior to the posterior, and gpum_bayes_factor(model1, model0, data) forms the Bayes factor; both require a proper prior and report the prior-sensitivity caveat. The posterior predictive check is deferred to the release that adds data generation.

When the GPU helps, and when it does not

A GPU does not accelerate every MCMC. The sequential dependence inside a chain cannot be parallelised. The parallelism comes from two axes: many independent chains, and the data-parallel evaluation of the log-density. A GPU pays off when the log-density is expensive to evaluate, that is over a large data set, or when thousands of chains are run. For a small model with few chains the CPU-GPU transfer overhead dominates and the GPU is slower than the CPU. The package documentation states this regime plainly rather than promising unconditional speedups.

The table makes the regime concrete. Wall-clock seconds for the gamma model (which exercises the lgamma opcode), on one machine, an RTX 5090 through CUDA 13.0 and Vulkan against the CPU backend; the three backends return the same posterior to Monte-Carlo noise, so only the time differs.

chains observations iterations CPU CUDA Vulkan GPU speedup
8 500 4000 0.14 s 0.58 s 0.46 s 0.2x (GPU slower)
256 4000 4000 8.7 s 1.9 s 1.8 s 4.7x
2048 8000 6000 186 s 22 s 22 s 8.5x
512 100000 3000 267 s 7.6 s 7.4 s 35x
128 1000000 2000 459 s 20 s 20 s 23x

The pattern is the one the two axes predict. With few chains and little data the launch and transfer overhead is not repaid and the CPU wins. The GPU advantage grows with the total parallel work; it is largest when the data set is large, since the block-per-chain kernel splits the log-density sum across the threads of a block, exactly the cost that dominates for a big data set.

Massive data sets. The last row is already at a million observations: a fit that takes almost eight minutes on the CPU returns in twenty seconds on the RTX 5090, a twenty-three-fold saving, and CUDA and Vulkan track each other within a fraction of a second. The per-iteration cost on the GPU is dominated by the log-density sum, which the kernel parallelises across a block's threads, so the wall clock scales close to linearly in iterations and much more gently in observations until the threads saturate. A data set of tens of millions of observations with a few hundred chains therefore stays in the regime of minutes on this GPU, against an hour or more on the CPU. The figures are one machine's; the ordering is what transfers, and the exact numbers should be re-measured on the target hardware.

Project status

The development follows a phased plan.

The package is released through R-universe; see Installation above.

Roadmap

The direction beyond the current release is recorded, tiered, in ROADMAP.md. The in-scope Metropolis-Hastings family is now complete through Differential Evolution MCMC (v0.4.0), the plotting layer of v0.4.1 and the per-generation DE variant of v0.4.2 under de_sync = TRUE. The extreme-optimisation milestone (v0.5.0) is delivered: the adaptive warmup gained full-covariance proposals from a cross-chain pooled covariance, dimension-dependent acceptance targets and a mid-warmup restart, and the rwm path now wins the live head-to-heads of the case-study vignettes against Stan and the generic samplers, median of three runs on the versioned harness (amendment v1.1 of EXPERIMENT_PROTOCOL.md). The two regimes that verdict conceded are closed by 0.5.1 and 0.5.2: gpum_lm() samples conjugate Gaussian linear models exactly from the closed-form posterior (6 to 7 million effective draws per second, above the Gibbs specialists on their own ground), and method = "mala" adds Metropolis-adjusted Langevin with reverse-mode automatic differentiation of the model bytecode, JIT-compiled beside the density, winning threefold against Stan on a d = 21 logistic regression (amendment v1.2). The application trajectory follows with the bivariate copula workflow (v0.6.0), per-column marginal auto-selection (v0.7.0), a first-order Markov copula for serial dependence (v0.7.1), the vine copula for higher dimension together with the stationary D-vine for serial dependence of order p (v0.8.0) and synthesis, including synthetic time-series trajectories (v0.9.0). The distribution catalogue that drives the marginal auto-selection is specified in CATALOG_DESIGN.md. The long arc is a portable probabilistic computing runtime built around MCMC-driven copula synthesis; the current package is its foundation.

Benchmark

The package carries a pre-registered experiment that characterises, in a refutable way, the regime in which gpumetropolis beats, ties or loses to the established R MCMC packages (MCMCpack, mcmc, nimble, BayesianTools, Stan via cmdstanr). The design, the six hypotheses with their support and refutation conditions, and the decision rules are frozen in EXPERIMENT_PROTOCOL.md, committed before any result existed. The primary metric is effective sample size per second, computed uniformly with coda so the estimator is not a confounder.

The figure below is from the full M1 run (protocol amendments v0.5 and v0.6: model M1, the Gaussian mean; fifteen replications per cell over the registered grid of data sizes N and chain counts C; 1129 completed runs). The harder models M2 to M4 follow in their own subsection below.

ESS per second by backend

The result, stated plainly:

So the honest reading: gpumetropolis earns its place in the many-chains regime and on the portability of one kernel source across CPU, CUDA and Vulkan, not as a faster single-chain sampler. The per-cell numbers are in benchmark/full_m1_cell_summary.csv.

Beyond the Gaussian mean: models M2 to M4

The registered factorial continues with three harder targets: M2 a separated bimodal posterior, M3 a heavy-tailed Student-t location model, M4 an ill-conditioned three-dimensional Gaussian. The run completed 2568 replications across the three models; its design and the departures from the registered factorial are recorded in protocol amendment v0.7.

ESS per second by backend, models M2 to M4

The honest reading, model by model:

Parallel tempering and the M2 mode-crossing trap

The M2 paragraph above reports that the registered factorial gives every backend a high KS rejection rate on the bimodal target. The diagnostic in that report is the family-wise KS gate, which compares the pooled draws against the closed-form bimodal reference. The trap of that diagnostic is that pooling eight chains, each stuck in one of the two basins, produces a sample that already resembles the reference: half of the chains contribute the positive mode, half contribute the negative mode, and the pooled empirical CDF tracks the symmetric bimodal target closely. The pretty bimodal shape of the pooled draws is then the artefact of overlay, not the signature of correct mixing.

The honest diagnostic for M2 is the split R-hat, which compares the within-chain variance and the between-chain variance. Stuck chains have a within-chain variance set by the spread inside one mode and a between-chain variance set by the spread between the two modes, and the ratio is large: R-hat reports the mismatch as a value far above one.

The focused single-cell re-run of 2026-06-15 measures this directly. The cell is N = 400 observations, C = 8 chains, 4000 iterations per chain with 2000 discarded as warmup, twenty replications, comparing three adapters: gpumetropolis-cpu for the random-walk Metropolis baseline, gpumetropolis-cpu-pt for the method = "pt" path of v0.3.0, and nimble as the strongest M2 competitor of v0.7.

adapter R-hat ESS ESS/s wall-clock (s)
gpumetropolis-cpu (RWM) 62.28 3525 27782 0.13
gpumetropolis-cpu-pt 1.00 671 3007 0.23
nimble 61.97 3652 26973 0.15

Two figures make the verdict visible.

M2 pooled densities, RWM and PT and nimble

The three pooled densities look almost identical: two peaks at the reference modes (dashed red). The verdict is in the title of each panel. R-hat near 1.00 for parallel tempering says the cold chain visits both modes within a single run; R-hat near 62 for the random-walk panels says each chain stayed in its starting basin, and the bimodal shape of the pooled draws is the artefact of overlaying eight stuck chains.

M2 traces, RWM and PT and nimble

The traces tell the same story directly. The eight chains of the random-walk runs and of nimble form flat lines at +3 and -3, with no visible crossing across the 2000 post-warmup iterations. The cold chain of the parallel-tempering run, drawn in black in the centre panel and the lowest-temperature chain by construction, jumps repeatedly between basins, fed by the hot chains, drawn in colour, which mix freely on their tempered targets.

The cost of the correct answer is a 1.7x wall-clock factor over random walk for the host-side swap step and a lower nominal effective sample size, since each cold-chain draw now pays the autocorrelation of a true mode-crossing chain rather than the autocorrelation of a stuck one. The full numbers and the harness are in benchmark/m2_pt_summary.csv; the focused re-run is amendment v0.9 of EXPERIMENT_PROTOCOL.md; the corresponding worked-case vignette is vignettes/m2_parallel_tempering.Rmd, the first chapter of the package's living book of cases.

Where this package fits in the MCMC ecosystem

The results above are textbook successes of parallel tempering, not algorithmic novelty: PT (Geyer 1991; Earl-Deem 2005) is the classical remedy for multimodal targets and any correct implementation recovers R-hat near 1 on the M2 cell. The value gpumetropolis adds is product design and portability, not a new sampling algorithm. The honest positioning against the wider MCMC ecosystem follows.

Where the established tools dominate: for differentiable unimodal targets in d greater than five (GLMs, hierarchical models, regression) the gradient-based samplers of Stan, PyMC and NumPyro are dramatically more efficient than any random-walk Metropolis; ESS-per-second improvements of one to three orders of magnitude are routine. For fast approximate inference the variational paths through Stan (ADVI), PyMC (BBVI) or INLA (Rue et al. 2009) cost a small fraction of the MCMC wall-clock. For mixture models with unknown number of components reversible-jump MCMC (Richardson and Green 1997) or Dirichlet-process mixtures estimate the parameters and k jointly, which fixed-k tempering does not. For likelihoods that are intractable ABC and pseudo-marginal methods (Andrieu and Roberts 2009) are the right tool. The package does not compete on those axes and does not claim to.

Where gpumetropolis adds value: the package is the right pick when the workflow needs:

Need Why this package
One kernel source across CPU, CUDA and Vulkan the CubeCL kernel and the Rust glue produce a single declaration that runs on all three; Stan requires CmdStanGPU for partial GPU coverage on OpenCL, PyMC requires the JAX backend dominated by NVIDIA, no other package matches the vendor-agnostic guarantee
Hundreds to thousands of parallel chains the registered factorial completes the 4096-chain cell that the other backends do not, and the many-chains axis is the natural GPU parallelism axis exploited by the block-per-chain kernel
Parallel tempering in one line method = "pt" is a built-in path; Stan and PyMC users typically hand-code the swap step or reach for external wrappers
A formula-based DSL for the log-likelihood the same R formula a user would already write becomes the kernel; no Stan or BUGS to translate to

Honest verdict on the worked cases of the living book: the M2 PT fit and the Gumbel-mixture fit shown in the two vignettes are competitive, not state-of-the-art. A custom Stan script with hand- coded tempering reaches comparable accuracy in comparable wall-clock on both cases. The advantage of gpumetropolis is workflow (one formula plus method = "pt" against custom Stan code) and portability (the same kernel runs on whatever GPU is on the host), not algorithmic superiority.

The longer arc of the package, the application path of v0.6.0 through v0.9.0 (bivariate copula, marginal auto-selection, vine copula, synthesis), is where the design is intended to deliver something that no current single R package puts together end to end: dataset in, ranked Bayesian fits out, synthetic dataset out. The MCMC engine is the foundation, not the headline.

The complete picture: gpumetropolis is fast in the regime it claims, many chains and an expensive log-density, and M3 shows it can win even with a single chain. It is not a faster sampler than a specialised algorithm where that algorithm applies, as M4 makes explicit. The per-cell numbers are in benchmark/full_m234_cell_summary.csv.

The machine and software environment of the benchmark host is recorded in benchmark/ENVIRONMENT.md, regenerated by benchmark/capture_env.sh, so the run can be reproduced or audited.

Issues

Please report issues at https://github.com/pcbrom/gpumetropolis/issues.

Citation

citation("gpumetropolis")