# pbdR Packages * [Application](#application) * [Communication](#communication) * [Computation](#computation) * [Developers](#developers) * [I/O](#io) * [Profiling](#profiling) We maintain stable distributions of all of our packages on our [GitHub project page](https://github.com/RBigData). We also try, whenever possible, to maintain releases on the [CRAN](http://cran.r-project.org/). <!-- Additionally, you can find current binaries of our packages for the Windows operating system at our sister site, [HPSC](http://thirteen-01.stat.iastate.edu/snoweye/pbdr/?item=download). --> If you are not sure where to begin, start with pbdDEMO package and its [vignette](https://github.com/RBigData/pbdDEMO/blob/master/inst/doc/pbdDEMO-guide.pdf?raw=true). Whether you are an experienced parallel programmer or a complete novice, this will get you up to speed on utilizing the pbdR tools. ## <a id="application"></a>Application <small>Specifically purposed packages that utilize pbdR.</small> | Package | Description | Download | Documentation | |---------|------------- | :------: | :-----------: | | pbdML | Machine learning algorithms, using pbdDMAT. | [ghbutton](http://github.com/RBigData/pbdML) | [htmlbutton](http://rbigdata.github.io/documentation/pbdML/00Index.html) | cubfits | Codon Usage Bias Fits, using pbdMPI. Estimating mutation and selection coefficients on synonymous codon bias usage based on models of ribosome overhead cost (ROC). Multinomial logistic regression and Markov Chain Monte Carlo are used to estimate and predict protein production rates with/without the presence of expressions and measurement errors. | [ghbutton](http://github.com/snoweye/cubfits) <br> [cranbutton](http://cran.r-project.org/package=cubfits) | [htmlbutton](http://rbigdata.github.io/documentation/cubfits/00Index.html) | pmclust | Tools for parallel model-based clustering. These include k-means and Gaussian mixture modeling, and can be applied to ad hoc distributed matrices as well as pbdDMAT conformable ones. | [ghbutton](http://github.com/RBigData/pmclust) <br> [cranbutton](http://cran.r-project.org/package=pmclust) | [htmlbutton](http://rbigdata.github.io/documentation/pmclust/00Index.html) | pbdDEMO | A collection of pbdR package demonstrations and examples. Also included is a lengthy, textbook-style vignette to help quickly move R programmers from their laptops to distributed platforms. | [ghbutton](http://github.com/RBigData/pbdDEMO) <br> [cranbutton](http://cran.r-project.org/package=pbdDEMO) | [htmlbutton](http://rbigdata.github.io/documentation/pbdDEMO/00Index.html) ## <a id="communication"></a>Communication <small>Tools for handling multi-machine communication.</small> | Package | Description | Download | Documentation | |---------|------------- | :------: | :-----------: | | pbdRPC | Remote Procedure Call. A very light implementation yet secure for remote procedure calls with unified interface via ssh (OpenSSH) or plink/plink.exe (PuTTY). | [ghbutton](http://github.com/RBigData/pbdRPC) <br> [cranbutton](http://cran.r-project.org/package=pbdRPC) | [htmlbutton](http://rbigdata.github.io/documentation/pbdRPC/00Index.html) | remoter | A collection of utilities for performing R computations on a remote resource. With the package, one can use their local R session (and gui of preference, e.g. RStudio) to connect to a remote server instance, and without modifying their code, have it executed on the remote instance. The package is a version of pbdCS with the MPI innards stripped out. | [ghbutton](http://github.com/RBigData/remoter) <br> [cranbutton](http://cran.r-project.org/package=remoter) | [htmlbutton](http://rbigdata.github.io/documentation/remoter/00Index.html) | pbdCS | A client/server framework for pbdR. With this package, one can move from batch SPMD programming to interactivity at thousands of cores. | [ghbutton](http://github.com/RBigData/pbdCS) | pbdZMQ | A set of bindings for the well-known ZeroMQ communication library. pbdZMQ greatly eases the burdens of network programming for R users. Additionally, the package serves as a distribution of ZeroMQ, making installation trivial. It serves as the foundation of the remoter and pbdCS packages. | [ghbutton](http://github.com/RBigData/pbdZMQ) <br> [cranbutton](http://cran.r-project.org/package=pbdZMQ) | [htmlbutton](http://rbigdata.github.io/documentation/pbdZMQ/00Index.html) ## <a id="computation"></a>Computation <small>Packages frameworks for building other scalable tools.</small> | Package | Description | Download | Documentation | |---------|------------- | :------: | :-----------: | | pbdDMAT | Distributed matrix classes and methods. The package includes numerous methods for manipulating and reshaping distributed matrices, as well as linear algebra and statistics routines. Through extensive use of R's S4 methods, these functions have identical syntax to serial R. | [ghbutton](http://github.com/RBigData/pbdDMAT) <br> [cranbutton](http://cran.r-project.org/package=pbdDMAT) | [htmlbutton](http://rbigdata.github.io/documentation/pbdDMAT/00Index.html) | pbdMPI | A high-level interface to MPI. The package handles linking issues, as well as offers a very simple R interface for MPI programming. | [ghbutton](http://github.com/RBigData/pbdMPI) <br> [cranbutton](http://cran.r-project.org/package=pbdMPI) | [htmlbutton](http://rbigdata.github.io/documentation/pbdMPI/00Index.html) ## <a id="developers"></a>Developers <small>Tools for developers.</small> | Package | Description | Download | Documentation | |---------|------------- | :------: | :-----------: | | pbdTEST | A testing framework for pbdR. | [ghbutton](http://github.com/RBigData/pbdTEST) | | pbdBASE | Base utilities for distributed matrices. For advanced programmers only. | [ghbutton](http://github.com/RBigData/pbdBASE) <br> [cranbutton](http://cran.r-project.org/package=pbdBASE) | [htmlbutton](http://rbigdata.github.io/documentation/pbdBASE/00Index.html) | pbdSLAP | The Scalable Linear Algebra Package. A distribution of ScaLAPACK, this package greatly simplifies package build and linking issues for distribute matrix programming. | [ghbutton](http://github.com/RBigData/pbdSLAP) <br> [cranbutton](http://cran.r-project.org/package=pbdSLAP) | [htmlbutton](http://rbigdata.github.io/documentation/pbdSLAP/00Index.html) ## <a id="io"></a>I/O <small>Packages for large, scalable I/O.</small> | Package | Description | Download | Documentation | |---------|------------- | :------: | :-----------: | | pbdNCDF4 | An R package interface to NetCDF version 4. The package adds collective parallel read and write capability to the R package ncdf4 version 1.8. | [ghbutton](http://github.com/RBigData/pbdNCDF4) <br> [cranbutton](http://cran.r-project.org/package=pbdNCDF4) | [htmlbutton](http://rbigdata.github.io/documentation/pbdNCDF4/00Index.html) ## <a id="profiling"></a>Profiling <small>Performance analysis packages.</small> | Package | Description | Download | Documentation | |---------|------------- | :------: | :-----------: | | hpcvis | Profiler visualizations. The package offers numerous plotters for data generated by pbdPROF and pbdPAPI. The package is kept separate so as not to include the graphics dependency chain on pbdPAPI and pbdPROF, as these dependencies may not be easily installed on supercomputers. | [ghbutton](http://github.com/RBigData/hpcvis) | | pbdPROF | MPI profiling tools. The package offers utilities for linking with MPI-using R packages (pbdMPI and Rmpi), as well as tools for parsing profiler output files. fpmpi and mpiP are supported. | [ghbutton](http://github.com/RBigData/pbdPROF) <br> [cranbutton](http://cran.r-project.org/package=pbdPROF) | [htmlbutton](http://rbigdata.github.io/documentation/pbdPROF/00Index.html) | pbdPAPI | Bindings for PAPI. The package offers utilities for measuring and collecting performance counter data, such as number of floating point operations or cache misses. Linux (or FreeBSD with a custom kernel) only. | [ghbutton](http://github.com/RBigData/pbdPAPI) | # <a id="installation">Installation Instructions</a> There are two sets of instructions: one for those who wish to use Docker, and one for those who do not. Under most circumstances, using the Docker builds is much easier. This is a good fit for anyone wishing to use pbdR: * On a desktop/laptop (caveat: OS version has to be reasonably new) * In the cloud. * On a cluster running Singularity. For all other cases, installing the "native" versions will be necessary. | Docker | Native | |--------|--------| | [pdfbutton](https://github.com/RBigData/installation-instructions/raw/master/build/docker.pdf) [htmlbutton](https://rawgit.com/wrathematics/installation-instructions/master/build/docker.html) | [pdfbutton](https://github.com/RBigData/installation-instructions/raw/master/build/native.pdf) [htmlbutton](https://rawgit.com/wrathematics/installation-instructions/master/build/native.html) |