future.p2p: Peer-to-Peer
Compute Clusters in R

- Share compute among friends

Henrik Bengtsson

University of California, San Francisco
R Foundation, R Consortium
@HenrikBengtsson

LatinR 2025 (2025W49)

Sharing compute among friends - it’s easy!

library(future)

plan(multisession)

ys <- xs |> furrr::future_map(slow_sum)

Sharing compute among friends - it’s easy!

library(future)

plan(future.p2p::cluster, cluster="alice/friends")

ys <- xs |> furrr::future_map(slow_sum)

Future … what?

When you write

y <- slow_sum(1:10)

R does

➔

1 minute

➔

➔ y ← 55

Futures gives more control on R evaluates code

R code

y <- slow_sum(1:10)

Same R code using futures

f <- future( slow_sum(1:10) )
y <- value(f)

What R does

future

➔

1 minute

➔

value

➔ y ← 55

R evaluates code sequentially

R code

y1 <- slow_sum( 1:10)          # 1 minute
y2 <- slow_sum(11:20)          # 1 minute
y3 <- slow_sum(21:30)          # 1 minute

What R does

➔

1 minute

➔ ➔ y1 ← 55

➔

1 minute

➔ ➔ y2 ← 155

➔

1 minute

➔ ➔ y3 ← 255

R with futures evaluates code parallelly

f1 <- future( slow_sum( 1:10) )   # 0 seconds
f2 <- future( slow_sum(11:20) )   # 0 seconds
f3 <- future( slow_sum(21:30) )   # 0 seconds

y1 <- value(f1)                   # 1 minute
y2 <- value(f2)                   # at the same time
y3 <- value(f3)                   # at the same time

future #1

future #2

future #3

➔

1 minute
total

➔

value #1

value #2

value #3

➔

y1 ← 55
y2 ← 155
y3 ← 255

Futures are the core building blocks
for parallel processing

The {future} package provides:

future()	resolved()	value()

Stay with your favorite coding style while parallelizing

ys <- lapply(xs, slow_sum)             # base R
ys <- xs |> map(slow_sum)              # {purrr}

are functions packaging things up in boxes and sending them to R;

➔ ➔
⋮
➔ ➔

Fast, futurized counterparts that send the boxes to parallel R workers;

ys <- future_lapply(xs, slow_sum)      # {future.apply}
ys <- xs |> future_map(slow_sum)       # {furrr}

…

➔

⋮

parallel

➔

…

Future boxes can be sent anywhere you want

You can parallelize “boxes” on your current computer, e.g. one of

plan(multisession)
plan(future.callr::callr)
plan(future.mirai::mirai_multisession)

You can distribute “boxes” to other computers, e.g. one of

plan(cluster, workers = c("n1", "desktop", "server.myuniv.org"))
plan(future.batchtools::batchtools_slurm)

… and now you can send them to your friends too!

Peer-to-Peer Computing
- share compute among friends

A P2P cluster has two components

Message Board

Used to announce futures and offers to do work
(centralized; lightweight - only metadata)

P2P file-transfer protocol

Used to send futures to workers and receive results
(peer-to-peer; full-size data transfers)

It’s easy to get started

1. Create a pico.sh account (1 minute)

2. Try with personal P2P cluster (3 minutes)

Instructions at https://future.p2p.futureverse.org/

Alice hosts a P2P cluster

Alice sets up P2P cluster and gives ‘bob’ and ‘carol’ access:

[alice]> future.p2p::host_cluster("alice/friends",
                                  users = c("bob", "carol"))

This is setting up a shared message board.

Q. What happens if Bob tries to use the P2P cluster?

[bob]> plan(future.p2p::cluster, cluster = "alice/friends")
[bob]> ys <- xs |> future_map(slow_sum)

A. It will get stuck - there are now workers

Contribute P2P workers at any time

[alice]> future.p2p::worker("alice/friends")
[alice]> future.p2p::worker("alice/friends")

[ bob ]> future.p2p::worker("alice/friends")

[carol]> future.p2p::worker("alice/friends")
[carol]> future.p2p::worker("alice/friends")
[carol]> future.p2p::worker("alice/friends")

=> A shared P2P cluster with 6 workers.

[bob]> plan(future.p2p::cluster, cluster = "alice/friends")
[bob]> ys <- xs |> future_map(slow_sum)

Pros and cons

High throughput, but with high latency

• High throughput:

  • any number of users and workers can join

  • a public P2P cluster could have thousands of P2P workers

• High latency:

  • round-trip takes time, because p2p file transfers take time

  • Example: 1+2 takes 1-10 seconds to send, evaluate, and return

  • This will improve!

⚠️ P2P computing requires mutual trust ⚠️

What if?

f <- future(system("erase-harddrive"))

Risks can be mitigated by “sandboxing” workers, e.g.

• in a virtual machine (more secure)
• in a Linux container (less secure)

=> I encourage you to work on these important topics

May the future be with you!

• It’s easy to get started - just try it
• Support: https://github.com/orgs/futureverse/discussions
• Tutorials: https://www.futureverse.org/tutorials.html
• Blog posts: https://www.futureverse.org/blog.html
• I love feedback, ideas, and bug reports 💜
• Lots of exiting improvements are coming your way!