How to make a great R reproducible example
When discussing performance with colleagues, teaching, sending a bug report or searching for guidance on mailing lists and here on Stack Overflow, a reproducible example is often asked and always helpful.
What are your tips for creating an excellent example? How do you paste data structures from r in a text format? What other information should you include?
Are there other tricks in addition to using
structure()? When should you include
require() statements? Which reserved words should one avoid, in addition to
How does one make a great r reproducible example?
A minimal reproducible example consists of the following items:
- a minimal dataset, necessary to demonstrate the problem
- the minimal runnable code necessary to reproduce the error, which can be run on the given dataset
- the necessary information on the used packages, R version, and system it is run on.
- in the case of random processes, a seed (set by
set.seed()) for reproducibility1
For examples of good minimal reproducible examples, see the help files of the function you are using. In general, all the code given there fulfills the requirements of a minimal reproducible example: data is provided, minimal code is provided, and everything is runnable. Also look at questions on Stack Overflow with lots of upvotes.
Producing a minimal dataset
For most cases, this can be easily done by just providing a vector/data frame with some values. Or you can use one of the built-in datasets, which are provided with most packages.
A comprehensive list of built-in datasets can be seen with
library(help = "datasets"). There is a short description to every dataset and more information can be obtained for example with
?mtcars where 'mtcars' is one of the datasets in the list. Other packages might contain additional datasets.
Making a vector is easy. Sometimes it is necessary to add some randomness to it, and there are a whole number of functions to make that.
sample() can randomize a vector, or give a random vector with only a few values.
letters is a useful vector containing the alphabet. This can be used for making factors.
A few examples :
- random values :
x <- rnorm(10)for normal distribution,
x <- runif(10)for uniform distribution, ...
- a permutation of some values :
x <- sample(1:10)for vector 1:10 in random order.
- a random factor :
x <- sample(letters[1:4], 20, replace = TRUE)
For matrices, one can use
matrix(), eg :
matrix(1:10, ncol = 2)
Making data frames can be done using
data.frame(). One should pay attention to name the entries in the data frame, and to not make it overly complicated.
An example :
set.seed(1) Data <- data.frame( X = sample(1:10), Y = sample(c("yes", "no"), 10, replace = TRUE) )
For some questions, specific formats can be needed. For these, one can use any of the provided
as.someType functions :
as.xts, ... These in combination with the vector and/or data frame tricks.
Copy your data
If you have some data that would be too difficult to construct using these tips, then you can always make a subset of your original data, using
subset() or the indices. Then use
dput() to give us something that can be put in R immediately :
> dput(iris[1:4, ]) # first four rows of the iris data set structure(list(Sepal.Length = c(5.1, 4.9, 4.7, 4.6), Sepal.Width = c(3.5, 3, 3.2, 3.1), Petal.Length = c(1.4, 1.4, 1.3, 1.5), Petal.Width = c(0.2, 0.2, 0.2, 0.2), Species = structure(c(1L, 1L, 1L, 1L), .Label = c("setosa", "versicolor", "virginica"), class = "factor")), .Names = c("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width", "Species"), row.names = c(NA, 4L), class = "data.frame")
If your data frame has a factor with many levels, the
dput output can be unwieldy because it will still list all the possible factor levels even if they aren't present in the the subset of your data. To solve this issue, you can use the
droplevels() function. Notice below how species is a factor with only one level:
> dput(droplevels(iris[1:4, ])) structure(list(Sepal.Length = c(5.1, 4.9, 4.7, 4.6), Sepal.Width = c(3.5, 3, 3.2, 3.1), Petal.Length = c(1.4, 1.4, 1.3, 1.5), Petal.Width = c(0.2, 0.2, 0.2, 0.2), Species = structure(c(1L, 1L, 1L, 1L), .Label = "setosa", class = "factor")), .Names = c("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width", "Species"), row.names = c(NA, 4L), class = "data.frame")
dput, you may also want to include only relevant columns:
> dput(mtcars[1:3, c(2, 5, 6)]) # first three rows of columns 2, 5, and 6 structure(list(cyl = c(6, 6, 4), drat = c(3.9, 3.9, 3.85), wt = c(2.62, 2.875, 2.32)), row.names = c("Mazda RX4", "Mazda RX4 Wag", "Datsun 710" ), class = "data.frame")
One other caveat for
dput is that it will not work for keyed
data.table objects or for grouped
dplyr. In these cases you can convert back to a regular data frame before sharing,
Worst case scenario, you can give a text representation that can be read in using the
text parameter of
zz <- "Sepal.Length Sepal.Width Petal.Length Petal.Width Species 1 5.1 3.5 1.4 0.2 setosa 2 4.9 3.0 1.4 0.2 setosa 3 4.7 3.2 1.3 0.2 setosa 4 4.6 3.1 1.5 0.2 setosa 5 5.0 3.6 1.4 0.2 setosa 6 5.4 3.9 1.7 0.4 setosa" Data <- read.table(text=zz, header = TRUE)
Producing minimal code
This should be the easy part but often isn't. What you should not do, is:
- add all kind of data conversions. Make sure the provided data is already in the correct format (unless that is the problem of course)
- copy-paste a whole function/chunk of code that gives an error. First, try to locate which lines exactly result in the error. More often than not you'll find out what the problem is yourself.
What you should do, is:
- add which packages should be used if you use any (using
- if you open connections or create files, add some code to close them or delete the files (using
- if you change options, make sure the code contains a statement to revert them back to the original ones. (eg
op <- par(mfrow=c(1,2)) ...some code... par(op))
- test run your code in a new, empty R session to make sure the code is runnable. People should be able to just copy-paste your data and your code in the console and get exactly the same as you have.
Give extra information
In most cases, just the R version and the operating system will suffice. When conflicts arise with packages, giving the output of
sessionInfo() can really help. When talking about connections to other applications (be it through ODBC or anything else), one should also provide version numbers for those, and if possible also the necessary information on the setup.
If you are running R in R Studio using
rstudioapi::versionInfo() can be helpful to report your RStudio version.
If you have a problem with a specific package you may want to provide the version of the package by giving the output of
packageVersion("name of the package").
1 Note: The output of
set.seed() differs between R >3.6.0 and previous versions. Do specify which R version you used for the random process, and don't be surprised if you get slightly different results when following old questions. To get the same result in such cases, you can use the
Read more... Read less...
(Here's my advice from How to write a reproducible example . I've tried to make it short but sweet)
How to write a reproducible example.
You are most likely to get good help with your R problem if you provide a reproducible example. A reproducible example allows someone else to recreate your problem by just copying and pasting R code.
There are four things you need to include to make your example reproducible: required packages, data, code, and a description of your R environment.
Packages should be loaded at the top of the script, so it's easy to see which ones the example needs.
The easiest way to include data in an email or Stack Overflow question is to use
dput()to generate the R code to recreate it. For example, to recreate the
mtcarsdataset in R, I'd perform the following steps:
- Copy the output
- In my reproducible script, type
mtcars <-then paste.
Spend a little bit of time ensuring that your code is easy for others to read:
make sure you've used spaces and your variable names are concise, but informative
use comments to indicate where your problem lies
do your best to remove everything that is not related to the problem.
The shorter your code is, the easier it is to understand.
Include the output of
sessionInfo()in a comment in your code. This summarises your R environment and makes it easy to check if you're using an out-of-date package.
You can check you have actually made a reproducible example by starting up a fresh R session and pasting your script in.
Before putting all of your code in an email, consider putting it on Gist github . It will give your code nice syntax highlighting, and you don't have to worry about anything getting mangled by the email system.
Personally, I prefer "one" liners. Something along the lines:
my.df <- data.frame(col1 = sample(c(1,2), 10, replace = TRUE), col2 = as.factor(sample(10)), col3 = letters[1:10], col4 = sample(c(TRUE, FALSE), 10, replace = TRUE)) my.list <- list(list1 = my.df, list2 = my.df, list3 = letters)
The data structure should mimic the idea of writer's problem and not the exact verbatim structure. I really appreciate it when variables don't overwrite my own variables or god forbid, functions (like
Alternatively, one could cut a few corners and point to a pre-existing data set, something like:
library(vegan) data(varespec) ord <- metaMDS(varespec)
Don't forget to mention any special packages you might be using.
If you're trying to demonstrate something on larger objects, you can try
my.df2 <- data.frame(a = sample(10e6), b = sample(letters, 10e6, replace = TRUE))
If you're working with spatial data via the
raster package, you can generate some random data. A lot of examples can be found in the package vignette, but here's a small nugget.
library(raster) r1 <- r2 <- r3 <- raster(nrow=10, ncol=10) values(r1) <- runif(ncell(r1)) values(r2) <- runif(ncell(r2)) values(r3) <- runif(ncell(r3)) s <- stack(r1, r2, r3)
If you're in need of some spatial object as implemented in
sp, you can get some datasets via external files (like ESRI shapefile) in "spatial" packages (see the Spatial view in Task Views).
library(rgdal) ogrDrivers() dsn <- system.file("vectors", package = "rgdal") ogrListLayers(dsn) ogrInfo(dsn=dsn, layer="cities") cities <- readOGR(dsn=dsn, layer="cities")
Inspired by this very post, I now use a handy function
reproduce(<mydata>) when I need to post to StackOverflow.
myData is the name of your object to reproduce, run the following in R:
install.packages("devtools") library(devtools) source_url("https://raw.github.com/rsaporta/pubR/gitbranch/reproduce.R") reproduce(myData)
This function is an intelligent wrapper to
dput and does the following:
- automatically samples a large data set (based on size and class. Sample size can be adjusted)
- creates a
- allows you to specify which columns to export
- appends to the front of it
objName <- ...so that it can be easily copy+pasted, but...
- If working on a mac, the output is automagically copied to the clipboard, so that you can simply run it and then paste to your question.
The source is available here:
# sample data DF <- data.frame(id=rep(LETTERS, each=4)[1:100], replicate(100, sample(1001, 100)), Class=sample(c("Yes", "No"), 100, TRUE))
DF is about 100 x 102. I want to sample 10 rows, and a few specific columns
reproduce(DF, cols=c("id", "X1", "X73", "Class")) # I could also specify the column number.
Gives the following output:
This is what the sample looks like: id X1 X73 Class 1 A 266 960 Yes 2 A 373 315 No Notice the selection split 3 A 573 208 No (which can be turned off) 4 A 907 850 Yes 5 B 202 46 Yes 6 B 895 969 Yes <~~~ 70 % of selection is from the top rows 7 B 940 928 No 98 Y 371 171 Yes 99 Y 733 364 Yes <~~~ 30 % of selection is from the bottom rows. 100 Y 546 641 No ==X==============================================================X== Copy+Paste this part. (If on a Mac, it is already copied!) ==X==============================================================X== DF <- structure(list(id = structure(c(1L, 1L, 1L, 1L, 2L, 2L, 2L, 25L, 25L, 25L), .Label = c("A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y"), class = "factor"), X1 = c(266L, 373L, 573L, 907L, 202L, 895L, 940L, 371L, 733L, 546L), X73 = c(960L, 315L, 208L, 850L, 46L, 969L, 928L, 171L, 364L, 641L), Class = structure(c(2L, 1L, 1L, 2L, 2L, 2L, 1L, 2L, 2L, 1L), .Label = c("No", "Yes"), class = "factor")), .Names = c("id", "X1", "X73", "Class"), class = "data.frame", row.names = c(1L, 2L, 3L, 4L, 5L, 6L, 7L, 98L, 99L, 100L)) ==X==============================================================X==
Notice also that the entirety of the output is in a nice single, long line, not a tall paragraph of chopped up lines. This makes it easier to read on SO questions posts and also easier to copy+paste.
Update Oct 2013:
You can now specify how many lines of text output will take up (ie, what you will paste into StackOverflow). Use the
lines.out=n argument for this. Example:
reproduce(DF, cols=c(1:3, 17, 23), lines.out=7) yields:
==X==============================================================X== Copy+Paste this part. (If on a Mac, it is already copied!) ==X==============================================================X== DF <- structure(list(id = structure(c(1L, 1L, 1L, 1L, 2L, 2L, 2L, 25L,25L, 25L), .Label = c("A", "B", "C", "D", "E", "F", "G", "H","I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U","V", "W", "X", "Y"), class = "factor"), X1 = c(809L, 81L, 862L,747L, 224L, 721L, 310L, 53L, 853L, 642L), X2 = c(926L, 409L,825L, 702L, 803L, 63L, 319L, 941L, 598L, 830L), X16 = c(447L,164L, 8L, 775L, 471L, 196L, 30L, 420L, 47L, 327L), X22 = c(335L,164L, 503L, 407L, 662L, 139L, 111L, 721L, 340L, 178L)), .Names = c("id","X1", "X2", "X16", "X22"), class = "data.frame", row.names = c(1L,2L, 3L, 4L, 5L, 6L, 7L, 98L, 99L, 100L)) ==X==============================================================X==
Here is a good guide.
The most important point is: Just make sure that you make a small piece of code that we can run to see what the problem is. A useful function for this is
dput(), but if you have very large data, you might want to make a small sample dataset or only use the first 10 lines or so.
Also make sure that you identified where the problem is yourself. The example should not be an entire R script with "On line 200 there is an error". If you use the debugging tools in R (I love
browser()) and Google you should be able to really identify where the problem is and reproduce a trivial example in which the same thing goes wrong.
The R-help mailing list has a posting guide which covers both asking and answering questions, including an example of generating data:
Examples: Sometimes it helps to provide a small example that someone can actually run. For example:
If I have a matrix x as follows:
> x <- matrix(1:8, nrow=4, ncol=2, dimnames=list(c("A","B","C","D"), c("x","y")) > x x y A 1 5 B 2 6 C 3 7 D 4 8 >
how can I turn it into a dataframe with 8 rows, and three columns named 'row', 'col', and 'value', which have the dimension names as the values of 'row' and 'col', like this:
> x.df row col value 1 A x 1
(To which the answer might be:
> x.df <- reshape(data.frame(row=rownames(x), x), direction="long", varying=list(colnames(x)), times=colnames(x), v.names="value", timevar="col", idvar="row")
The word small is especially important. You should be aiming for a minimal reproducible example, which means that the data and the code should be as simple as possible to explain the problem.
EDIT: Pretty code is easier to read than ugly code. Use a style guide.