Description
The future package defines a simple and uniform way of evaluating R expressions asynchronously. Future “backend” packages implement asynchronous processing over various shared-memory and distributed systems.
Redis is a fast networked key/value database that includes a stack-like data structure (Redis “lists”). This feature makes Redis useful as a lightweight task queue manager for elastic distributed computing.
The future.redis package defines a simple elastic distributed computing backend for future using the redux package to communicate with Redis. Elastic means that workers can be added or removed at any time, including during a running computation. Elasticity implies partial fault-tolerance (to handle workers that go away during a running computation). This style of distributed computing is well-suited to modern cloud environments, and especially cloud spot markets.
The future.redis package fullfills the Future API Backend Specification. This is validated using the future.tests (Test Suite for ‘Future API’ Backends) package.
Example: Hello world
To try out future.redis for the first time,
make sure Redis and future.redis are installed, and
if not already running, start the Redis server (see the Redis documentation). You can verify it’s running by calling
redux::redis_available().
Then, launch R, and try the following code:
library(future.redis)
## Start two future.redis parallel workers
startLocalWorkers(2)
## Create a future that calculates the sum of 1:100
f <- future(sum(1:100))
v <- value(f)
print(v)
## [1] 5050You should also get 5050. If it stalls when you call value(), make sure you did indeed start the parallel workers first.
For more information and examples, see ?future.redis::future.redis.
Requirements
The future.redis package requires that Redis is installed on the machine. Redis is available on all modern operating systems, including Linux, macOS, and MS Windows.
Installation
The future.redis package is only available from GitHub. To install it, and all of its dependencies, call:
remotes::install_github("bwlewis/future.redis")
