Part 6 Quick summary and comparison to other parallel frameworks
6.1 Feature comparisons
| future framework | foreach with doFuture | BiocParallel with doFuture | parallel | foreach* | BiocParallel* | |
|---|---|---|---|---|---|---|
| Exception handling | ✓ | ✓ (special) | ✓ (special) | ✓ (special) | ✓ (special) | ✓ (special) |
| Warnings | ✓ | ✓ | ✓ | no | no | no |
| Messages | ✓ | ✓ | ✓ | no | no | no |
| Standard output | ✓ | ✓ | ✓ | no | no | no |
| Parallel RNG | ✓ | ✓ (special) | ✓ (tedious) | ✓ (manual) | ✓ (special) | ✓ |
| Progress updates | ✓ (near-live) | ✓ | ✓ (near-live) | no | no | ✓ (per task) |
| Single-expression parallelization | ✓ (future() & value()) |
no | no | ✓ (mcparallel() & mcollect()) |
no | no |
(*) With any other parallel backend than a future one.
By “near-live” progress updates, we mean that progress is updated every time a worker is polled, which happens frequently.
6.2 Parallel-backend comparisons
| future framework | foreach with doFuture | BiocParallel with doFuture | parallel | foreach* | BiocParallel* | |
|---|---|---|---|---|---|---|
| sequential | ✓ | ✓ | ✓ | no | ✓ | ✓ |
| parallel (forked) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| parallel (PSOCK) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| parallel (> 125 workers) | ✓ (callr) | ✓ (callr) | ✓ (callr) | no | no | no |
| multiple machines | ✓ | ✓ | ✓ | ✓ (*) | ✓ (**) | ✓ (**) |
| HPC job scheduler | ✓ (batchtools) | ✓ | ✓ (batchtools) | no | no | ✓ (batchtools) |
(*) With parallelly::makeClusterPSOCK() it is easy to set up remote parallel workers over SSH. To do the same with parallel::makePSOCKcluster(), one needs to reconfigure the firewall.
(**) One can use registerDoParallel(workers) where workers <- parallelly::makeClusterPSOCK(...) to use remote workers with foreach. BiocParallel can use foreach via DoparParam().