What curatedMetagenomicData provides

curatedMetagenomicData provides processed data from whole-metagenome shotgun metagenomics, with manually-curated metadata, as integrated and documented Bioconductor TreeSummarizedExperiment objects or TSV flat text exports. It provides 6 types of data for each dataset:

1. Species-level taxonomic profiles, expressed as relative abundance from kingdom to strain level (relative_abundance)
2. Presence of unique, clade-specific markers (marker_presence)
3. Abundance of unique, clade-specific markers (marker_abundance)
4. Abundance of gene families (gene_families)
5. Metabolic pathway coverage (pathway_coverage)
6. Metabolic pathway abundance (pathway_abundance)

Types 1-3 are generated by MetaPhlAn3; 4-6 are generated by HUMAnN3 using the UniRef90 database.

Currently there are:

• 22588 samples from 93 datasets; see the list of available studies
• 141 fields of specimen metadata from original papers, supplementary files, and websites, with manual curation and automated syntax-checking to standardize annotations

Additional documentation and resources

See:

1. Available Studies
2. Our Pipeline
3. Changes in cMD 3
4. Reference for cMD Functions
5. The command-line tool
6. Example analyses in R and Python, Docker image, free Cloud instance for teaching/learning

Installation

To install curatedMetagenomicData from Bioconductor, use BiocManager as follows.

if (!require("BiocManager", quietly = TRUE))
    install.packages("BiocManager")

BiocManager::install("curatedMetagenomicData")

Most users should simply install curatedMetagenomicData from Bioconductor. Developers may install the GitHub repository with remotes::install_github and the latest Bioconductor development version.

R Packages

To demonstrate the functionality of curatedMetagenomicData, the dplyr and DT packages are needed.

library(dplyr)
library(DT)

Sample Metadata

The curatedMetagenomicData package contains a data.frame, sampleMetadata, of manually curated sample metadata to help users understand the nature of studies and samples available prior to returning resources. Beyond this, it serves two purposes: 1) to define study_name, which is used with the curatedMetagenomicData() function to query and return resources, and 2) to define sample_id, which is used with the returnSamples() function to return samples across studies.

To demonstrate, the first ten rows and columns (without any NA values) of sampleMetadata for the AsnicarF_2017 study are shown in the table below.

sampleMetadata |>
    filter(study_name == "AsnicarF_2017") |>
    select(where(~ !any(is.na(.x)))) |>
    slice(1:10) |>
    select(1:10) |>
    datatable(options = list(dom = "t"), extensions = "Responsive")

Data Access

There are three main ways to access data resources in curatedMetagenomicData.

1. The curatedMetagenomicData() function to search for and return resources.
2. The returnSamples() function to return samples across studies.
3. Through the curatedMetagenomicDataTerminal command-line interface.

curatedMetagenomicData("AsnicarF_20.+")
## 2021-03-31.AsnicarF_2017.gene_families
## 2021-03-31.AsnicarF_2017.marker_abundance
## 2021-03-31.AsnicarF_2017.marker_presence
## 2021-03-31.AsnicarF_2017.pathway_abundance
## 2021-03-31.AsnicarF_2017.pathway_coverage
## 2021-03-31.AsnicarF_2017.relative_abundance
## 2021-10-14.AsnicarF_2017.gene_families
## 2021-10-14.AsnicarF_2017.marker_abundance
## 2021-10-14.AsnicarF_2017.marker_presence
## 2021-10-14.AsnicarF_2017.pathway_abundance
## 2021-10-14.AsnicarF_2017.pathway_coverage
## 2021-10-14.AsnicarF_2017.relative_abundance
## 2021-03-31.AsnicarF_2021.gene_families
## 2021-03-31.AsnicarF_2021.marker_abundance
## 2021-03-31.AsnicarF_2021.marker_presence
## 2021-03-31.AsnicarF_2021.pathway_abundance
## 2021-03-31.AsnicarF_2021.pathway_coverage
## 2021-03-31.AsnicarF_2021.relative_abundance

curatedMetagenomicData("AsnicarF_2017.relative_abundance", dryrun = FALSE, rownames = "short")
## $`2021-10-14.AsnicarF_2017.relative_abundance`
## class: TreeSummarizedExperiment 
## dim: 298 24 
## metadata(0):
## assays(1): relative_abundance
## rownames(298): species:Escherichia coli species:Bifidobacterium bifidum
##   ... species:Streptococcus gordonii species:Abiotrophia sp. HMSC24B09
## rowData names(7): superkingdom phylum ... genus species
## colnames(24): MV_FEI1_t1Q14 MV_FEI2_t1Q14 ... MV_MIM5_t2M14
##   MV_MIM5_t3F15
## colData names(22): study_name subject_id ... pregnant lactating
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (298 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL

curatedMetagenomicData("AsnicarF_20.+.relative_abundance", dryrun = FALSE, counts = TRUE, rownames = "short")
## $`2021-10-14.AsnicarF_2017.relative_abundance`
## class: TreeSummarizedExperiment 
## dim: 298 24 
## metadata(0):
## assays(1): relative_abundance
## rownames(298): species:Escherichia coli species:Bifidobacterium bifidum
##   ... species:Streptococcus gordonii species:Abiotrophia sp. HMSC24B09
## rowData names(7): superkingdom phylum ... genus species
## colnames(24): MV_FEI1_t1Q14 MV_FEI2_t1Q14 ... MV_MIM5_t2M14
##   MV_MIM5_t3F15
## colData names(22): study_name subject_id ... pregnant lactating
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (298 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL
## 
## $`2021-03-31.AsnicarF_2021.relative_abundance`
## class: TreeSummarizedExperiment 
## dim: 639 1098 
## metadata(0):
## assays(1): relative_abundance
## rownames(639): species:Phocaeicola vulgatus species:Bacteroides
##   stercoris ... species:Pyramidobacter sp. C12-8 species:Brevibacterium
##   aurantiacum
## rowData names(7): superkingdom phylum ... genus species
## colnames(1098): SAMEA7041133 SAMEA7041134 ... SAMEA7045952 SAMEA7045953
## colData names(24): study_name subject_id ... BMI family
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (639 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL

curatedMetagenomicData("AsnicarF_20.+.marker_abundance", dryrun = FALSE) |>
    mergeData()
## class: SummarizedExperiment 
## dim: 76639 1122 
## metadata(0):
## assays(1): marker_abundance
## rownames(76639): 39491__A0A395UYA6__EUBREC_0408
##   39491__C4Z9E9__T1815_12341 ... 78448__A0A087CC86__BPULL_0419
##   356829__A0A087E8C8__BITS_5024
## rowData names(0):
## colnames(1122): MV_FEI1_t1Q14 MV_FEI2_t1Q14 ... SAMEA7045952
##   SAMEA7045953
## colData names(26): study_name subject_id ... BMI family

curatedMetagenomicData("AsnicarF_20.+.pathway_abundance", dryrun = FALSE) |>
    mergeData()
## class: SummarizedExperiment 
## dim: 16913 1122 
## metadata(0):
## assays(1): pathway_abundance
## rownames(16913): UNMAPPED UNINTEGRATED ... PWY-6277: superpathway of
##   5-aminoimidazole ribonucleotide
##   biosynthesis|g__Massiliomicrobiota.s__Massiliomicrobiota_timonensis
##   PWY-6151: S-adenosyl-L-methionine cycle
##   I|g__Massiliomicrobiota.s__Massiliomicrobiota_timonensis
## rowData names(0):
## colnames(1122): MV_FEI1_t1Q14 MV_FEI2_t1Q14 ... SAMEA7045952
##   SAMEA7045953
## colData names(26): study_name subject_id ... BMI family

curatedMetagenomicData("AsnicarF_20.+.relative_abundance", dryrun = FALSE, rownames = "short") |>
    mergeData()
## class: TreeSummarizedExperiment 
## dim: 679 1122 
## metadata(0):
## assays(1): relative_abundance
## rownames(679): species:Escherichia coli species:Bifidobacterium bifidum
##   ... species:Pyramidobacter sp. C12-8 species:Brevibacterium
##   aurantiacum
## rowData names(7): superkingdom phylum ... genus species
## colnames(1122): MV_FEI1_t1Q14 MV_FEI2_t1Q14 ... SAMEA7045952
##   SAMEA7045953
## colData names(26): study_name subject_id ... BMI family
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (679 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL

sampleMetadata |>
    filter(age >= 18) |>
    filter(!is.na(alcohol)) |>
    filter(body_site == "stool") |>
    select(where(~ !all(is.na(.x)))) |>
    returnSamples("relative_abundance", rownames = "short")
## class: TreeSummarizedExperiment 
## dim: 832 702 
## metadata(0):
## assays(1): relative_abundance
## rownames(832): species:Prevotella copri species:Prevotella sp. CAG:520
##   ... species:Corynebacterium aurimucosum species:Corynebacterium
##   coyleae
## rowData names(7): superkingdom phylum ... genus species
## colnames(702): JAS_1 JAS_10 ... YSZC12003_37879 YSZC12003_37880
## colData names(48): study_name subject_id ...
##   age_twins_started_to_live_apart zigosity
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (832 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL

Example Analysis

To demonstrate the utility of curatedMetagenomicData, an example analysis is presented below. However, readers should know analysis is generally beyond the scope of curatedMetagenomicData and the analysis presented here is for demonstration alone. It is best to consider the output of curatedMetagenomicData as the input of analysis more than anything else.

library(OmicsMLRepoR)
library(mia)
library(scater)
library(vegan)
library(stringr)
library(lefser)

cmd <- OmicsMLRepoR::getMetadata("cMD")

smoke <- cmd |> 
    filter(disease == "Healthy") |> 
    filter(age_group == "Adult") |>
    filter(!is.na(smoker)) |>
    filter(body_site == "feces") |>
    select(where(~ !all(is.na(.x))))  # remove metadata columns which are all `NA` values

table(smoke$smoker, useNA = "ifany")

## 
##                                Non-smoker (finding) 
##                                                 332 
##            Non-smoker (finding);Ex-smoker (finding) 
##                                                  94 
## Non-smoker (finding);Never smoked tobacco (finding) 
##                                                 460 
##                                    Smoker (finding) 
##                                                 172

smoke <- smoke %>%
  mutate(
    smoker_bin = as.factor(
      case_when(smoker == "Smoker (finding)" ~ "Smoker",
                smoker != "Non-smoker (finding);Never smoked tobacco (finding)" ~ "Never Smoker",
      )))

table(smoke$smoker_bin, useNA = "ifany")

## 
## Never Smoker       Smoker         <NA> 
##          426          172          460

smoke_tse <- smoke %>% returnSamples("relative_abundance", rownames = "short")

## Removing samples with NA values for smoker_bin
smoke_tse <- smoke_tse[,!is.na(smoke_tse$smoker_bin)]

smoke_tse_genus <- agglomerateByRank(smoke_tse, rank = "genus")

Alpha Diversity

Alpha diversity is a measure of the within sample diversity of features (relative abundance proportions here) and seeks to quantify the evenness (i.e. are the amounts of different microbes the same) and richness (i.e.  are they are large variety of microbial taxa present). The Shannon index (H’) is a commonly used measure of alpha diversity, it’s estimated here using the addAlpha() function from the mia package.

## Adding Shannon diversity values to colData
smoke_shannon <- smoke_tse_genus |>
  addAlpha(assay.type = "relative_abundance", index = "shannon_diversity")

## Violin plots
title <- "Alpha Diversity by Smoking Status"
smoke_shannon |> 
    plotColData(x = "smoker_bin", y = "shannon_diversity", colour_by = "smoker_bin", shape_by = "smoker_bin") +
    labs(x = "Smoking Status", y = "Alpha Diversity (H')") + 
    guides(colour = guide_legend(title = "Smoking Status"), shape = guide_legend(title = title)) +
    theme(legend.position = "none")

Alpha Diversity - Shannon Index (H’)

A p-value < 0.01 and a W value > 0 indicate that the Never Smoker group has higher alpha diversity compared to the Smoker group. This may serve as basis for further investigation as to whether smoking can lead to gut microbiome dysbiosis.

## Test if alpha diversity between smokers and non-smokers is significantly different
wilcox.test(shannon_diversity ~ smoker_bin, data = colData(smoke_shannon))

## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  shannon_diversity by smoker_bin
## W = 26768, p-value = 2.475e-07
## alternative hypothesis: true location shift is not equal to 0

Beta Diversity

Beta diversity is a measure of the between sample diversity of features (relative abundance proportions here) and seeks to quantify the magnitude of differences (or similarity) between every given pair of samples. Below it is assessed by Bray–Curtis Principal Coordinates Analysis (PCoA) and Uniform Manifold Approximation and Projection (UMAP).

Bray–Curtis PCoA

To calculate pairwise Bray–Curtis distance for every sample in our study we will use the runMDS() function from the scater package along with the vegdist() function from the vegan package.

To quickly plot the results of beta diversity analysis, the plotReducedDim() function from the scater package is used along with ggplot2 syntax.

smoke_tse %>% 
  agglomerateByRanks() |>
    runMDS(FUN = vegdist, method = "bray", exprs_values = "relative_abundance", altexp = "genus", name = "BrayCurtis") |>
    plotReducedDim("BrayCurtis", colour_by = "smoker_bin", shape_by = "smoker_bin") +
    labs(x = "PCo 1", y = "PCo 2") +
    guides(colour = guide_legend(title = "Smoking Status"), shape = guide_legend(title = "Smoking Status")) +
    theme(legend.position = c(0.80, 0.25))

Beta Diversity – Bray–Curtis PCoA

UMAP

To calculate the UMAP coordinates of every sample in our study we will use therunUMAP() function from the scater package package, as it handles the task in a single line.

To quickly plot the results of beta diversity analysis, the plotReducedDim() function from the scater package is used along with ggplot2 syntax again.

smoke_tse %>%
  agglomerateByRanks() |>
    runUMAP(exprs_values = "relative_abundance", altexp = "genus", name = "UMAP") |>
    plotReducedDim("UMAP", colour_by = "smoker_bin", shape_by = "smoker_bin") +
    labs(x = "UMAP 1", y = "UMAP 2") +
    guides(colour = guide_legend(title = "Smoking Status"), shape = guide_legend(title = "Smoking Status")) +
    theme(legend.position = c(0.80, 0.55))

lefser(
    relativeAb(smoke_tse_genus),
    kruskal.threshold = 0.05,
    wilcox.threshold = 0.05,
    lda.threshold = 2,
    classCol = "smoker_bin",
    subclassCol = NULL,
    assay = 1L,
    trim.names = FALSE,
    checkAbundances = TRUE
) %>%
    lefserPlot()

## The outcome variable is specified as 'smoker_bin' and the reference category is 'Never Smoker'.
##  See `?factor` or `?relevel` to change the reference category.

Type Conversion

Finally, the curatedMetagenomicData package previously had functions for conversion to phyloseq class objects, and they have been removed. It is likely that some users will still want to do analysis using phyloseq, and we would like to help them do so – it is just easier if we don’t have to maintain the conversion function ourselves. As such, the mia package has a function, makePhyloseqFromTreeSummarizedExperiment, that will readily do the conversion – users needing this functionality are advised to use it.

convertToPhyloseq(alcoholStudy, assay.type = "relative_abundance")

Session Info

## R version 4.6.1 (2026-06-24)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 26.04.1 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.32.so;  LAPACK version 3.12.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Etc/UTC
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats4    stats     graphics  grDevices utils     datasets  methods  
## [8] base     
## 
## other attached packages:
##  [1] lefser_1.22.0                   stringr_1.6.0                  
##  [3] vegan_2.7-6                     permute_0.9-10                 
##  [5] scater_1.40.2                   ggplot2_4.0.3                  
##  [7] scuttle_1.22.0                  mia_1.20.0                     
##  [9] MultiAssayExperiment_1.38.0     OmicsMLRepoR_1.6.1             
## [11] DT_0.34.0                       dplyr_1.2.1                    
## [13] curatedMetagenomicData_3.20.0   TreeSummarizedExperiment_2.20.0
## [15] Biostrings_2.80.2               XVector_0.52.0                 
## [17] SingleCellExperiment_1.34.0     SummarizedExperiment_1.42.0    
## [19] Biobase_2.72.0                  GenomicRanges_1.64.0           
## [21] Seqinfo_1.2.0                   IRanges_2.46.0                 
## [23] S4Vectors_0.50.3                BiocGenerics_0.58.1            
## [25] generics_0.1.4                  MatrixGenerics_1.24.0          
## [27] matrixStats_1.5.0               BiocStyle_2.40.0               
## 
## loaded via a namespace (and not attached):
##   [1] splines_4.6.1               ggplotify_0.1.3            
##   [3] filelock_1.0.3              tibble_3.3.1               
##   [5] DirichletMultinomial_1.54.1 lifecycle_1.0.5            
##   [7] httr2_1.3.0                 vroom_1.7.1                
##   [9] lattice_0.23-1              MASS_7.3-66                
##  [11] crosstalk_1.2.2             magrittr_2.0.5             
##  [13] sass_0.4.10                 rmarkdown_2.32             
##  [15] jquerylib_0.1.4             yaml_2.3.12                
##  [17] otel_0.2.0                  DBI_1.3.0                  
##  [19] buildtools_1.0.0            RColorBrewer_1.1-3         
##  [21] multcomp_1.4-32             abind_1.4-8                
##  [23] purrr_1.2.2                 yulab.utils_0.2.5          
##  [25] TH.data_1.1-5               rappdirs_0.3.4             
##  [27] sandwich_3.1-3              gdtools_0.5.1              
##  [29] data.tree_1.2.0             ggrepel_0.9.8              
##  [31] irlba_2.4.1                 rols_3.8.3                 
##  [33] tidytree_0.4.8              maketools_1.3.2            
##  [35] testthat_3.3.2              RSpectra_0.16-2            
##  [37] DelayedMatrixStats_1.34.0   codetools_0.2-20           
##  [39] coin_1.4-6                  DelayedArray_0.38.2        
##  [41] tidyselect_1.2.1            aplot_0.3.2                
##  [43] farver_2.1.2                ScaledMatrix_1.20.0        
##  [45] viridis_0.6.5               BiocFileCache_3.2.0        
##  [47] jsonlite_2.0.0              BiocNeighbors_2.6.0        
##  [49] decontam_1.32.0             survival_3.8-12            
##  [51] systemfonts_1.3.2           tools_4.6.1                
##  [53] treeio_1.36.1               Rcpp_1.1.2                 
##  [55] glue_1.8.1                  gridExtra_2.3.1            
##  [57] SparseArray_1.12.3          xfun_0.61                  
##  [59] mgcv_1.9-4                  withr_3.0.3                
##  [61] BiocManager_1.30.27         fastmap_1.2.0              
##  [63] bluster_1.22.0              digest_0.6.39              
##  [65] rsvd_1.0.5                  R6_2.6.1                   
##  [67] gridGraphics_0.5-1          RSQLite_3.53.3             
##  [69] ecodive_2.2.6               DiagrammeR_1.0.12          
##  [71] tidyr_1.3.2                 fontLiberation_0.1.0       
##  [73] DECIPHER_3.8.1              FNN_1.1.4.1                
##  [75] httr_1.4.9                  htmlwidgets_1.6.4          
##  [77] S4Arrays_1.12.1             uwot_0.2.5                 
##  [79] pkgconfig_2.0.3             gtable_0.3.6               
##  [81] modeltools_0.2-25           blob_1.3.0                 
##  [83] S7_0.2.2                    sys_3.4.3                  
##  [85] brio_1.1.5                  htmltools_0.5.9            
##  [87] fontBitstreamVera_0.1.1     scales_1.4.0               
##  [89] png_0.1-9                   ggfun_0.2.2                
##  [91] knitr_1.52                  tzdb_0.5.0                 
##  [93] reshape2_1.4.5              visNetwork_2.1.4           
##  [95] nlme_3.1-171                curl_8.0.0                 
##  [97] cachem_1.1.0                zoo_1.9-1                  
##  [99] BiocVersion_3.23.1          parallel_4.6.1             
## [101] vipor_0.4.7                 libcoin_1.0-13             
## [103] AnnotationDbi_1.74.0        pillar_1.11.1              
## [105] grid_4.6.1                  vctrs_0.7.3                
## [107] BiocSingular_1.28.1         dbplyr_2.6.0               
## [109] beachmat_2.28.0             cluster_2.1.8.3            
## [111] beeswarm_0.4.0              evaluate_1.0.5             
## [113] readr_2.2.0                 mvtnorm_1.4-2              
## [115] cli_3.6.6                   compiler_4.6.1             
## [117] rlang_1.3.0                 crayon_1.5.3               
## [119] labeling_0.4.3              plyr_1.8.9                 
## [121] fs_2.1.0                    ggbeeswarm_0.7.3           
## [123] ggiraph_0.9.6               stringi_1.8.9              
## [125] viridisLite_0.4.3           BiocParallel_1.46.0        
## [127] lazyeval_0.2.3              fontquiver_0.2.1           
## [129] Matrix_1.7-6                ExperimentHub_3.2.2        
## [131] hms_1.1.4                   patchwork_1.3.2            
## [133] sparseMatrixStats_1.24.0    bit64_4.8.6                
## [135] KEGGREST_1.52.2             AnnotationHub_4.2.2        
## [137] igraph_2.3.4                memoise_2.0.1              
## [139] bslib_0.12.0                ggtree_4.2.0               
## [141] bit_4.6.0                   ape_5.8-1