QIIME 2 for Experienced Microbiome Researchers¶

Pro-tips for power users¶

That said, here are a few tips we’ve learned that should substantially improve your experience in transitioning your workflows to QIIME 2:

Pro-tip #1: QIIME 2 artifacts are just zip files. If at any point you want to look at what actual files are in the .qza artifact, you can use qiime tools export to extract the data file directly (which is basically just a wrapper for unzip). Alternatively, you can also unzip your artifact directly (unzip -k file.qza) and look through the files in the data/ folder.

Pro-tip #2: the QIIME 2 command line interface tools are slow because they have to unzip and re-zip the data contained in the artifacts each time you call them. If you need to process your data more interactively, you might want to use the Python API - it is much faster since objects can be simply stored in memory. You can learn more about the different QIIME 2 interfaces.

Importing data into QIIME 2¶

Relevant plugin: qiime tools import

If you’re using QIIME 2 to process your data, the first thing you need to do is get that data into a format that QIIME 2 can understand. Various importing methods currently available in QIIME 2 are highlighted in the QIIME 2 importing tutorial.

This step has the potential to be the most confusing part of the QIIME 2 pipeline as there are dozens of import and format types to choose from. To see a full list of available import/format types use: qiime tools import --show-importable-formats and qiime tools import --show-importable-types

If you’re importing FASTQ data that you’ve generated, you’ll likely need to generate a manifest file, which is just a text file that maps each FASTQ file to its sample ID and direction (if applicable).

If you have sequencing data with one of two very specific formats (EMP or Casava), you can directly import the folder containing your sequencing files with the --type EMPSingleEndSequences or --type 'SampleData[PairedEndSequencesWithQuality]' flags (or their respective paired-end types). Otherwise if you don’t have one of these two very specific formats, you’ll need to make the manifest file to give import instructions on what and how to import your files.

If you want to import FASTA files or a feature table directly, you can also do that by using a different --type flag for qiime tools import. The importing tutorial goes through all these options in detail.

Demultiplexing sequences¶

Relevant plugins

• q2-demux

• cutadapt

If you have reads from multiple samples in the same file, you’ll need to demultiplex your sequences.

If your barcodes have already been removed from the reads and are in a separate file, you can use q2-demux to demultiplex these.

If your barcodes are still in your sequences, you can use functions from the cutadapt plugin. The cutadapt demux-single method looks for barcode sequences at the beginning of your reads (5’ end) with a certain error tolerance, removes them, and returns sequence data separated by each sample. The QIIME 2 forum has a tutorial on various functions available in cutadapt, including demultiplexing. You can learn more about how cutadapt works under the hood by reading their documentation.

Note: Currently q2-demux and q2-cutadapt do not support demultiplexing dual-barcoded paired-end sequences, but only can demultiplex with barcodes in the forward reads. So for the time being, this type of demultiplexing needs to be done outside of QIIME 2 using other tools, for example bcl2fastq.

Merging reads¶

Relevant plugin: q2-vsearch

Whether or not you need to merge reads depends on how you plan to cluster or denoise your sequences into amplicon sequence variants (ASVs) or operational taxonomic units (OTUs). If you plan to use deblur or OTU clustering methods next, join your sequences now. If you plan to use dada2 to denoise your sequences, do not merge — dada2 performs read merging automatically after denoising each sequence.

If you need to merge your reads, you can use the QIIME 2 q2-vsearch plugin with the merge-pairs method.

Removing non-biological sequences¶

Relevant plugins

• q2-cutadapt

• dada2

If your data contains any non-biological sequences (e.g. primers, sequencing adapters, PCR spacers, etc), you should remove these.

The q2-cutadapt plugin has comprehensive methods for removing non-biological sequences from paired-end or single-end data.

If you’re going to use DADA2 to denoise your sequences, you can remove biological sequences at the same time as you call the denoising function. All of DADA2’s denoise fuctions have some sort of --p-trim parameter you can specify to remove base pairs from the 5’ end of your reads. (Deblur does not have this functionality yet.)

Grouping similar sequences¶

There are two main approaches for grouping similar sequences together: denoising and clustering. The overview tutorial provides more in-depth discussion of these approaches.

Regardless of how you group your sequences, the grouping methods will output:

1. A list of representative sequences for each of your OTUs and/or ASVs (QIIME 2 data format FeatureData[Sequence]), and

2. A feature table which indicates how many reads of each OTU/sequence variant were observed in each sample. (QIIME 2 data format FeatureTable[Frequency])

DADA2 and deblur will also produce a stats summary file with useful information regarding the filtering and denoising.

Denoising¶

Relevant plugins:

• dada2

• deblur

DADA2 and deblur are currently the two denoising methods available in QIIME 2. You can read more about the methods in the overview tutorial.

DADA2 and deblur both output exact sequence variants, which supposedly represent the true biological sequences present in your data. Their creators have different terminology for these sequences (DADA2 calls them “amplicon sequence variants” (ASVs) and deblur calls them “sub-OTUs”). We’ll be using the ASV terminology throughout this tutorial to refer to both outputs.

OTU Clustering¶

In this tutorial, we’ll cover QIIME 2 methods that perform de novo and closed reference clustering. The QIIME OTU clustering tutorial also covers these in more detail.

To cluster your sequences, you need to make sure that:

• paired-end reads are merged

• non-biological sequences are removed

• reads are all trimmed to the same length

• low-quality reads are discarded

We discussed merging paired-end reads and removing non-biological sequences above (Sections Merge reads and Remove non-biological sequences).

Once your data are ready, you’ll also need to dereplicate your reads before clustering.

Assigning taxonomy¶

Relevant plugin: feature-classifier

Assigning taxonomy to ASV or OTU representative sequences is covered in the taxonomy classification tutorial. All taxonomy assignment methods are in the feature-classifier plugin.

There are two main approaches for assigning taxonomy, each with multiple methods available.

The first involves aligning reads to reference databases directly:

• classify-consensus-blast: BLAST+ local alignment

• classify-consensus-vsearch: VSEARCH global alignment

Both use the consensus approach of taxonomy assignment, which you can learn more about in the overview and tweak with the maxaccepts, perc-identity, and min-consensus parameters.

The second approach uses a machine learning classifier to assign likely taxonomies to reads, and can be used through the classify-sklearn method.

This method needs a pre-trained model to classify the sequences: you can either download one of the pre-trained taxonomy classifiers from the data resources page, or train one yourself (following the steps outlined in the feature classifier tutorial). (You can also learn a lot more about the specific models implemented in the plugin’s associated paper.)

Export the data¶

Relevant plugin: qiime tools export

If you’re a veteran microbiome scientist and don’t want to use QIIME 2 for your analyses, you can extract your feature table and sequences from the artifact using the export tool. While export only outputs the data, the extract tool allows you to also extract other metadata such as the citations, provenance etc.

Note that this places generically named files (e.g. feature-table.txt) into the output directory, so you may want to immediately rename the files to something more information (or somehow ensure that they stay in their original directory)!

You can also use the handy qiime2R package to import QIIME 2 artifacts directly into R.