Completed on 21 Jan 2015 by Scott Hunicke-Smith. Sourced from Publons: https://publons.com/review/87548.
Login to endorse this review.
The article deviates from the PeerJ suggested format of "Intro, Methods, Discussion, Conclusion" and instead uses "Intro, Results, Methods" with "Experiments" enumerated within Results. I find this somewhat disorganized, particularly in the semi-arbitrary distinction between "Experiments" described in results and "Methods", particularly since the article is fundamentally a comparison of several methods. I think the standard Intro/Methods/Discussion format would communicate the information more clearly.
The introduction and abstract should more clearly define the scope as pertaining to linear response of RNA-seq measurement. Many RNA-seq experiments rely on other important properties of the data such as representation of transcript direction and uniformity of coverage (i.e. Levin, 2010) which are not explored here and are not even possible with some of the low input methods described. Library diversity (another relevant metric) is not explored in the dataset at hand but is used as justification (line 72) for the study.
Figure 1A is very suspicious; the region around 10 ng D. virilis is difficult to interpret as a distribution, and it is odd that 10ng and 20ng are so broad while the region around 16ng appears unusually tight. I suspect this is a graphing artifact and so may be misleading.
The axes labels in all distributions are too small to be legible.
There are other minor edits noted in the attached PDF.
Overall, the experimental design and analysis reasonably documents the method comparison with respect to linearity of gene expression measurement. The introduction should more clearly state this scope particularly since many other RNA-seq comparison papers use many other metrics.
The simulation data is an excellent component of this paper.
The method used to establish the "practical lower input limit" of the TruSeq kit is based on two data points in a single replicate being at or below the lower limit of detection of a common but not sensitive assay (the Nanodrop). Neither the method ("failures") nor the measurement technique (Nanodrop) are sufficient to establish a lower limit. This should either be re-assessed using a Qubit assay, qPCR, or BioAnalyzer, or restricted in conclusion to "anecdotal".
Other clarifications are noted in the attached PDF.
With a few exceptions, the findings are robust, statistically sound, and controlled.
The justification of 70 ng as a conservative lower limit is not well justified. Since it is only 30% lower than the manufacturers' stated lower limit and, as stated in the text, this is still several orders of magnitude higher than practical for single-cell experiments, the authors might re-consider simply using the manufacturer's lower limit and present the anecdotal data as simply that.
An exploration of read depth is described in the introduction (lines 63/64) but is only touched on briefly in the simulation section. This analysis does not constitute "the effect of read depth on the quality of the data."
While the error may have been minor, the "pooling error" needs more explanation to understand whether this was an error in quantitation or simply in volume measurement. This is important to preserve the integrity of the rest of the experimental data.
There are several instances the authors make personal appeals (e.g. the phrase, "we believe") (lines 8, 91, 153, 218, 223). These should either be supported by data or citation, deleted, or rephrased as assertion/assumption.