Showing posts with label for loop. Show all posts
Showing posts with label for loop. Show all posts
Sunday, September 20, 2015
Summarize Data by Several Variables
Here's an example how to conveniently summarize data with the cast function (package reshape). By the way you see how this could be done "in-conveniently" by hand. You also see how a for-loop works and how a matrix is constructed and filled. In addition this serves as an illustrative example how flexible "indexing" in R works, as seen in the below loop! (download data) (this example is adapted from https://stat.ethz.ch/pipermail/r-sig-ecology/2011-May/002174.html)
Read more »
# Set path were you downloaded data to:To cite reshape in publications, please use:
load("C:\\...\\Ecoregions_Taxa.RData")
# see what's in there:
ls()
# investigate data3:
str(data3)
# i want to know how many taxa are within each ecoregion -
# more presicly i want to know how many orders, families, genera are there within each region:
require(reshape)
dfm<- melt(data3, id = "ECO_NAME")
dfc<- cast(dfm, ECO_NAME~variable, function(x) length(unique(x)))
# the same by hand -
# make new variable of ECO_NAME*variable combinations:
dfm$variable2 <- as.factor(paste(dfm$variable, dfm$ECO_NAME, sep = " - "))
# make vector to collect results for all unique ECO_NAME*variable combinations:
Ns <- data.frame(count = rep(NA, length(unique(dfm$variable2))),
row.names = unique(dfm$variable2))
# loop through all unique ECO_NAME*variable combinations
# and record length of unique values:
for (i in levels(dfm$variable2)){
subset = dfm$value[dfm$variable2 == i]
Ns[i, "count"] <- length(unique(subset))
}
# put counts in matrix/table, Ns$count is in order of taxonomy
# levels (= "variable"), so I have to fill by cols (byrow = F),
# as the matrix/table colums are chosen to be the taxonomy levels:
result <- matrix(Ns$count,
nrow = length(levels(dfm$ECO_NAME)),
ncol = length(levels(dfm$variable)),
dimnames = list(levels(dfm$ECO_NAME), levels(dfm$variable)), byrow = F)
print(result)
H. Wickham. Reshaping data with the reshape package. Journal of
Statistical Software, 21(12), 2007.
Friday, September 18, 2015
Test Difference Between Diversity-Indices of Two Samples with Abundance Data
I adapted a permutation test from the PAST Software (Hammer & Harper, http://folk.uio.no/ohammer/past/diversity.html) that tests difference between diversity-indices of two samples with abundance data in R... See the example below:
### two samples with abundance data:
require(vegan)
### two samples with abundance data:
### 1st sample of counts (numbers representing species 1-15)
dat1 <- c(1,2,0,3,0,1,4,20,0,2,21,3,15,23,30)
### 2nd sample of counts (numbers representing species 1-15)
dat2 <- c(0,1,0,1,0,0,1,10,0,0,29,1,22,30,25)
(div1=diversity(t(dat1),"shannon"))
(div2=diversity(t(dat2),"shannon"))
(rich1=sum(dat1>0))
(rich2=sum(dat2>0))
(tr.diff.div=abs(div1-div2)) ### observed difference
(tr.diff.rich=abs(rich1-rich2)) ### ...
K=2000
pop.diff.div <- pop.diff.rich <- rep(NA,K) ### dataframe for null population of differences
pop.diff.div[1]=tr.diff.div
pop.diff.rich[1]=tr.diff.rich
for(i in 2:K){ ### loop to generate null pop.diff. of differences
ind1<-sum(dat1) ### sum of individuals sample no.1
ind2<-sum(dat2) ### sum of ... no.2
pool<-c(rep(1:length(dat1),dat1), ### pooled sample with numbers representing species
rep(1:length(dat2),dat2)) ### replicated as often as no. of individuals
temp1=sample(pool,ind1,replace=T) ### resample no.1
temp2=sample(pool,ind2,replace=T) ### resample no.2
### calculate diversity:
div1.temp=diversity(t(tabulate(temp1)),"shannon")
div2.temp=diversity(t(tabulate(temp2)),"shannon")
rich1.temp=sum(tabulate(temp1)>0)
rich2.temp=sum(tabulate(temp2)>0)
pop.diff.div[i]=abs(div1.temp-div2.temp)
pop.diff.rich[i]=abs(rich1.temp-rich2.temp)
}
(p.div=sum(pop.diff.div>=abs(tr.diff.div))/K)
(p.rich=sum(pop.diff.rich>=abs(tr.diff.rich))/K)
### diagramms to show null-distributions with obs. differences
par(mfrow=c(2,1))
hist(pop.diff.div)
abline(v=tr.diff.div,lty=3,col=2,lwd=2)
### diagramms to show null-distributions with obs. differences
hist(pop.diff.rich)
abline(v=tr.diff.rich,lty=3,col=2,lwd=2)
Taking into account what Jari commented below (simplifying code & fixing row and column totals - still having in mind that the test may be anti-conservative!)
dat1 <- c(1,2,0,3,0,1,4,20,0,2,21,3,15,23,30)Now with null model and vegan::oecosimu
dat2 <- c(0,1,0,1,0,0,1,10,0,0,29,1,22,30,25)
(div1=diversity(t(dat1)))
(div2=diversity(t(dat2)))
dat <- rbind(dat1, dat2)
B = 2000
sim <- r2dtable(B - 1, rowSums(dat), colSums(dat))
div <- sapply(sim, diversity)
pop.diff.div <- abs(div[1, ] - div[2, ])
(p <- sum(pop.diff.div >= abs(div1 - div2)) / B)
hist(pop.diff.div); abline(v = abs(div1 - div2))
### two samples with abundance data:
### 1st sample of counts (numbers representing species 1-15)
dat1 <- c(1,2,0,3,0,1,4,20,0,2,21,3,15,23,30)
### 2nd sample of counts (numbers representing species 1-15)
dat2 <- c(0,1,0,1,0,0,1,10,0,0,29,1,22,30,25)
dat <- rbind(dat1, dat2)
divdiff <- function(x) abs(diversity(x[1, ]) - diversity(x[2, ]))
oecosimu(dat, divdiff, "r2dtable", nsimul = 1999)
Multivariate Repeated Measurements with adonis():
Please check the updated code in the comment by Wallace Beiroz, 26 January 2015 at 13:37!
To cite package ‘vegan’ in publications use:
Jari Oksanen, F. Guillaume Blanchet, Roeland Kindt, Pierre Legendre,
Peter R. Minchin, R. B. O'Hara, Gavin L. Simpson, Peter Solymos, M.
Henry H. Stevens and Helene Wagner (2011). vegan: Community Ecology
Package. R package version 2.0-2.
http://CRAN.R-project.org/package=vegan
Read more »
Lately I had to figure out how to do a repeated measures (or mixed effects) analysis on multivariate (species) data. Here I share code for a computation in R with the adonis function of the vegan package. Credit goes to Gavin Simpson providing most of the important pieces of the below code in R-Help.
The design:
We have multivariate species data, sampled at different sites (n = 6) at 3 points in time (N = 24).
The hypothesis:
H0: Species composition is the same across time. The permutation will produce the H0-population by a restricted randomization of time points.
H1: Species composition differs between time points
The analysis:
With repeated measurements one has to allow for the temporal structure and for the dependency within repeated measures. In a pedantic manner one would account for temporal structure by permuting all time-points in the same way also keeping the ordering of time-points. With 3 time-points this would give you only 3 permutations!
But we may relax this very restrictive assumption and say it doesn't matter that all samples at t1 were taken at one time and the ones of t2, etc. at another. Still there would be the repeated measures and the fact that the samples were taken in a temporal sequence. So how to deal with this?
(1) Having in mind that repeated measures on one sample are not independent, we will not permute across these samples.
(2) Now to the temporal correlation within repeated samples: The ordering of time-points is another type of dependency: the ordering is 1, 2, 3 - we will change this ordering only to 3, 1, 2 or 2, 3, 1, This permutation is referred to as toroidal shift which is keeping the "neighborhood" of values intact.
Eventually, we have 3 permutations per site, with 6 sites - this yields 3^6 = 729 permutations overall. Thus, a minimal p-value of 1/729 = 0.0014 can be obtained. I'd say that's good enough!
### species matrix with 20 species abundances (mean = 50, sd = 10)
### one time variable, with 3 timepoints, which should be tested
### and a factor denoting sites that were repeatedly sampled (site)
## Load packages
require(vegan)
### Data:
sp <- matrix(rnorm(3 * 6 * 20, 50, 10), nrow = 3 * 6, ncol = 20,
dimnames = list(1:18, paste("Sp", 1:20, sep = "")))
time <- as.ordered(rep(1:3, 6))
site <- gl(6, 3)
cbind(site, time, sp)
### add time effect at timepoint 3,
### this will effect will be tested by adonis():
sp_1 <- sp
sp_1[time==3,] <- sp[time==3,] + rnorm(20, 10, 1)
cbind(site, time, sp_1)
### choose which species set to test:
test_sp <- sp_1
### computing the true R2-value
### (btw, using dist() defaults to euclidean distance):
print(fit <- adonis(dist(test_sp) ~ time, permutations=1))
### number of perms
B <- 1999
### setting up frame which will be populated by
### random r2 values:
pop <- rep(NA, B + 1)
### the first entry will be the true r2:
pop[1] <- fit$aov.tab[1, 5]
### set up a "permControl" object:
### we turn off mirroring as time should only flow in one direction
ctrl <- permControl(strata = site, within = Within(type = "series", mirror = FALSE))
### Number of observations:
nobs <- nrow(test_sp)
### check permutation (...rows represent the sample id):
### ..they are ok!
### within in each repeated sample (= sites) timepoints are shuffled,
### with keeping the sequence intact (e.g., for site 1: 1,2,3 - 2,3,1 - 3,2,1)
shuffle(nobs, control = ctrl)
### loop:
### in adonis(...) you need to put permutations = 1, otherwise
### adonis will not run
set.seed(123)
for(i in 2:(B+1)){
idx <- shuffle(nobs, control = ctrl)
fit.rand <- adonis(dist(test_sp) ~ time[idx], permutations = 1)
pop[i] <- fit.rand$aov.tab[1, 5]
}
### get the p-value:
print(pval <- sum(pop >= pop[1]) / (B + 1))
### [1] 0.0035
### the sign. p-value supports the H1 (->there is a time effect).
### ..and the fact that samples are not iid is allowed by
### the customized perms - so this p-value is trustworthy as opposed
### to tests not acknowledging dependency of data points..
### test sp set without an effect:
### replace test_sp with sp set without effect:
test_sp <- sp
### now re-run the script and see the result:
### it is insign. - as expected:
### setting up frame which will be populated by
### random r2 values:
pop <- rep(NA, B + 1)
### computing the true R2-value:
print(fit <- adonis(dist(test_sp) ~ time, permutations = 1))
### the first entry will be the true r2:
pop[1] <- fit$aov.tab[1, 5]
### run the loop:
set.seed(123)
for(i in 2:(B+1)){
idx <- shuffle(nobs, control = ctrl)
fit.rand <- adonis(dist(test_sp) ~ time[idx], permutations = 1)
pop[i] <- fit.rand$aov.tab[1, 5]
}
print(pval <- sum(pop >= pop[1]) / (B + 1))
### [1] 0.701
## make a histogram to see random R2-values and the true one:
hist(pop, xlab = "Population R2")
abline(v = pop[1], col = 2, lty = 3)
text(0.08, 300, paste("true R2,\np = ", pval, sep = ""))
To cite package ‘vegan’ in publications use:
Jari Oksanen, F. Guillaume Blanchet, Roeland Kindt, Pierre Legendre,
Peter R. Minchin, R. B. O'Hara, Gavin L. Simpson, Peter Solymos, M.
Henry H. Stevens and Helene Wagner (2011). vegan: Community Ecology
Package. R package version 2.0-2.
http://CRAN.R-project.org/package=vegan
Subscribe to:
Posts (Atom)
