Downloadable files

The ScPCA Portal download packages include gene expression data, a QC report, and associated metadata for each processed sample. Gene expression data is available as either SingleCellExperiment objects (.rds files) or AnnData objects (.h5ad files). These files are delivered as a zip file. When you uncompress the zip file, the root directory name of your download will include the date you accessed the data on the ScPCA Portal. We recommend you record this date in case there are future updates to the Portal that change the underlying data or if you need to cite the data in the future (see How to Cite for more information). Please see our CHANGELOG for a summary of changes that impact downloads from the Portal.

For all data downloads, sample folders (indicated by the SCPCS prefix) contain the files for all libraries (SCPCL prefix) derived from that biological sample. Most samples only have one library that has been sequenced. For multiplexed sample libraries, the sample folder name will be an underscore-separated list of all samples found in the library files that the folder contains. Note that multiplexed sample libraries are only available as SingleCellExperiment objects, and are not currently available as AnnData objects.

See the FAQ section about samples and libraries for more information.

The files shown below will be included with each library (example shown for a library with ID SCPCL000000):

  • An unfiltered counts file: SCPCL000000_unfiltered.rds or SCPCL00000_unfiltered_rna.h5ad,

  • A filtered counts file: SCPCL000000_filtered.rds or SCPCL00000_filtered_rna.h5ad,

  • A processed counts file: SCPCL000000_processed.rds or SCPCL00000_processed_rna.h5ad,

  • A quality control report: SCPCL000000_qc.html,

  • A supplemental cell type report: SCPCL000000_celltype-report.html

Every download also includes a single single_cell_metadata.tsv file containing metadata for all libraries included in the download.

Metadata-only downloads are also available, either by downloading the metadata for all samples in a single project or by downloading the metadata for all samples on the Portal. Please see the section on metadata for a full description of the contents of the metadata files.

If downloading a project containing bulk RNA-seq data, two tab-separated value files, bulk_quant.tsv and bulk_metadata.tsv, will be included in the project download. The bulk_quant.tsv file contains a gene by sample matrix (each row a gene, each column a sample) containing raw gene expression counts quantified by Salmon. The bulk_metadata.tsv file contains associated metadata for all samples with bulk RNA-seq data.

See also processing bulk RNA samples.

The folder structure within the zip file is determined by whether individual samples or all samples associated with a project are selected for download. Note that if a sample selected for download contains a spatial transcriptomics library, the files included will be different than pictured below. See the description of the Spatial transcriptomics output section below.

SingleCellExperiment downloads

Download folder structure for project downloads:

project download folder

Download folder structure for individual sample downloads:

sample download folder

AnnData downloads

Download folder structure for project downloads:

project download folder

Download folder structure for individual sample downloads:

sample download folder

Download folder structure for individual sample downloads with CITE-seq (ADT) data:

sample download folder

If downloading a sample that contains a CITE-seq library as an AnnData object (.h5ad file), the quantified CITE-seq expression data is included as a separate file with the suffix _adt.h5ad.

Gene expression data

Single-cell or single-nuclei gene expression data is provided as either SingleCellExperiment objects (.rds files) or AnnData objects (.h5ad files). Three files will be provided for each library included in the download - an unfiltered counts file, a filtered counts file, and a processed counts file.

The unfiltered counts file, SCPCL000000_unfiltered.rds or SCPCL000000_unfiltered_rna.h5ad, contains the counts matrix, where the rows correspond to genes or features and the columns correspond to cell barcodes. Here, all potential cell barcodes that are identified after running alevin-fry are included in the counts matrix. The object also includes summary statistics for each cell barcode and gene, as well as metadata about that particular library, such as the reference index and software versions used for mapping and quantification.

The filtered counts file, SCPCL000000_filtered.rds or SCPCL000000_filtered_rna.h5ad contains a counts matrix with the same structure as above. The cells in this file are those that remain after filtering using emptyDrops. As a result, this file only contains cell barcodes that are likely to correspond to true cells.

The processed counts file, SCPCL000000_processed.rds or SCPCL000000_processed_rna.h5ad, contains both the raw and normalized counts matrices. The filtered counts file is further filtered to remove low quality cells, such as those with a low number of genes detected or high mitochondrial content. This file contains the raw and normalized counts data for cell barcodes that have passed both levels of filtering. In addition to the counts matrices, the SingleCellExperiment or AnnData object stored in the file includes the results of dimensionality reduction using both principal component analysis (PCA) and UMAP.

See Single-cell gene expression file contents for more information about the contents of the SingleCellExperiment and AnnData objects and the included statistics and metadata. See also Using the provided RDS files in R and Using the provided H5AD files in Python.

QC report

The included QC report, SCPCL000000_qc.html, serves as a general overview of each library, including processing information, summary statistics and general visualizations of cell metrics.

Cell type report

The cell type report, SCPCL000000_celltype-report.html, includes an overview of cell type annotations present in the processed objects. This report contains details on methodologies used for cell type annotation, information about reference sources, comparisons among cell type annotation methods, and diagnostic plots. For more information on how cell types were annotated, see the section on Cell type annotation.

If the downloaded library was from a cell line sample, no cell type annotation will have been performed. Therefore, there will be no cell type report in the download for these libraries.

Metadata

Included with each download is a single_cell_metadata.tsv file containing relevant metadata for each sample included in the download. Each row corresponds to a unique sample/library combination and contains the following columns:

column_id

contents

scpca_sample_id

Sample ID in the form SCPCS000000

scpca_library_id

Library ID in the form SCPCL000000

diagnosis

Tumor type

subdiagnosis

Subcategory of diagnosis or mutation status (if applicable)

disease_timing

At what stage of disease the sample was obtained, either diagnosis or recurrence

age

Age provided by submitter

age_timing

Whether age is the age at diagnosis (diagnosis), age at collection (collection), or unknown. This will be diagnosis for all samples collected at diagnosis, indicated by the disease_timing column

sex

Sex of patient that the sample was obtained from

tissue_location

Where in the body the tumor sample was located

participant_id

Unique id corresponding to the donor from which the sample was obtained

submitter_id

Original sample identifier from submitter

submitter

Submitter name/id

organism

The organism the sample was obtained from (e.g., Homo_sapiens)

development_stage_ontology_term_id

HsapDv ontology term indicating the age at which the sample was collected. unknown indicates age is unavailable.

sex_ontology_term_id

PATO term referring to the sex of the sample. unknown indicates sex is unavailable.

organism_ontology_id

NCBI taxonomy term for organism, e.g. NCBITaxon:9606.

self_reported_ethnicity_ontology_term_id

For Homo sapiens samples, a Hancestro term. multiethnic indicates more than one ethnicity is reported. unknown indicates unavailable ethnicity and NA is used for all other organisms.

disease_ontology_term_id

MONDO term indicating disease type. PATO:0000461 is used for normal or healthy tissue.

tissue_ontology_term_id

UBERON term indicating tissue of origin. NA indicates tissue is unavailable.

seq_unit

cell for single-cell samples, nucleus for single-nuclei samples, bulk for bulk RNA-seq samples, and spot for spatial transcriptomics samples.

technology

10x kit used to process library

total_reads

Total number of reads processed by salmon

mapped_reads

Number of reads successfully mapped

sample_cell_count_estimate

Total number of cells found in the filtered object for all libraries from a given sample

unfiltered_cells

Total number of cells detected by alevin-fry

filtered_cell_count

Number of cells after filtering with emptyDrops

filtered_cells

Number of cells after filtering with emptyDrops. Only present for multiplexed libraries

processed_cells

Number of cells after removing low quality cells

has_cellhash

Boolean indicating if the library has associated cell hashing data

includes_anndata

Boolean indicating if AnnData is available to download for the library

is_cell_line

Boolean indicating whether or not the sample was obtained from a cell line

is_multiplexed

Boolean indicating if the library contains multiplexed samples

is_xenograft

Boolean indicating whether or not the sample was obtained from a patient-derived xenograft

scpca_project_id

Project ID in the form SCPCP000000

pi_name

Name of primary investigator

project_title

Title of project

genome_assembly

Ensembl version of genome used for mapping with salmon, alevin-fry, or spaceranger

mapping_index

Name of index used for mapping with salmon, alevin-fry, or spaceranger

spaceranger_version

Version of 10x Genomics’ Space Ranger used for mapping, only present for spatial transcriptomics samples

alevin_fry_version

Version of alevin-fry used for gene expression quantification

salmon_version

Version of salmon used for mapping

transcript_type

Types of counts matrices included in SingleCellExperiment or AnnData objects. total indicates both spliced and unspliced reads are included in the final count, spliced indicates only spliced reads are included in the final count

droplet_filtering_method

The method used for cell filtering. One of emptyDrops, emptyDropsCellRanger, or UMI cutoff

cell_filtering_method

Method used by the Data Lab to filter low quality cells prior to normalization. Either miQC or Minimum_gene_cutoff

prob_compromised_cutoff

The minimum cutoff for the probability of a cell being compromised, as calculated by miQC

min_gene_cutoff

The minimum cutoff for the number of unique genes detected per cell

normalization_method

The method used for normalization of raw RNA counts. Either deconvolution, described in Lun, Bach, and Marioni (2016), or log-normalization

demux_method

Methods used to calculate demultiplexed sample numbers. Only present for multiplexed libraries

demux_samples

Samples included in multiplexed library. Only present for multiplexed libraries

date_processed

Date sample was processed through AlexsLemonade/scpca-nf

Additional metadata may also be included, specific to the disease type and experimental design of the project. Examples of this include treatment or outcome. Metadata pertaining to processing will be available in this table and inside of the SingleCellExperiment and AnnData objects. See the SingleCellExperiment experiment metadata section for more information on metadata columns that can be found in the SingleCellExperiment object. See the AnnData experiment metadata section for more information on metadata columns that can be found in the AnnData object.

For projects with bulk RNA-seq data, the bulk_metadata.tsv file will be included for project downloads. This file will contain fields equivalent to those found in the single_cell_metadata.tsv related to processing the sample, but will not contain patient or disease specific metadata (e.g. age, sex, diagnosis, subdiagnosis, tissue_location, or disease_timing).

Metadata-only downloads

Metadata for all samples on the Portal is available to download separately from gene expression data downloads. Each project page has an option to download metadata for all of its samples as a single zip file containing the metadata.tsv file and a README.md file. Project-specific metadata will contain all columns listed in the above table and any additional project-specific columns, such as treatment or outcome.

Additionally, a single TSV file containing the metadata for all samples from all projects on the Portal is available for download. The Portal-wide metadata will contain all columns listed in the above table.

Multiplexed sample libraries

For libraries where multiple biological samples were combined via cellhashing or similar technology (see the FAQ section about multiplexed samples), the organization of the downloaded files and metadata is slightly different. Note that multiplexed sample libraries are only available as SingleCellExperiment objects, and are not currently available as AnnData objects.

For project downloads, the counts and QC files will be organized by the set of samples that comprise each library, rather than in individual sample folders. These sample set folders are named with an underscore-separated list of the sample ids for the libraries within, e.g., SCPCS999990_SCPCS999991_SCPCS999992. Bulk RNA-seq data, if present, will follow the same format as bulk RNA-seq for single-sample libraries.

multiplexed project download folder

Because we do not perform demultiplexing to separate cells from multiplexed libraries into sample-specific count matrices, sample downloads from a project with multiplexed data will include all libraries that contain the sample of interest, but these libraries will still contain cells from other samples.

For more on the specific contents of multiplexed library SingleCellExperiment objects, see the Additional SingleCellExperiment components for multiplexed libraries section.

The metadata file for multiplexed libraries (single_cell_metadata.tsv) will have the same format as for individual samples, but each row will represent a particular sample/library pair, meaning that there may be multiple rows for each scpca_library_id, one for each scpca_sample_id within that library. In addition, an estimate of the total number of cells for each sample after demultiplexing will be found in the sample_cell_estimate (as opposed to the sample_cell_count_estimate column used for non-multiplexed samples).

Merged object downloads

When downloading a full ScPCA project, you can choose to download data from all samples as individual files, or you can download a single file containing all samples merged into a single object.

Merged object downloads contain all single-cell or single-nuclei gene expression data for a given ScPCA project within a single object, provided as either a SingleCellExperiment object (.rds file) or an AnnData object (.h5ad file).

The object file, SCPCP000000_merged.rds or SCPCP000000_merged_rna.h5ad, contains both a raw and normalized counts matrix, each with combined counts for all samples in an ScPCA project. In addition to the counts matrices, the SingleCellExperiment or AnnData object stored in the file includes the results of library-weighted dimensionality reduction using both principal component analysis (PCA) and UMAP. See the section on merged object processing for more information about how merged objects were created.

If downloading a project that contains at least one CITE-seq library, the quantified CITE-seq expression data will also be merged. In SingleCellExperiment objects (rds files), the CITE-seq expression data is provided as an alternative experiment in the same object as the gene expression data. However, for AnnData objects, (.h5ad files), the quantified CITE-seq expression is instead provided as a separate file called SCPCP000000_merged_adt.h5ad.

Every download also includes a single single_cell_metadata.tsv file containing metadata for all libraries included in the merged object. For a full description of this file’s contents, refer to the metadata section above.

If downloading a project containing bulk RNA-seq data, two tab-separated value files, bulk_quant.tsv and bulk_metadata.tsv, will be included in the merged object download. The bulk_quant.tsv file contains a gene by sample matrix (each row a gene, each column a sample) containing raw gene expression counts quantified by Salmon. The bulk_metadata.tsv file contains associated metadata for all samples with bulk RNA-seq data. This file will contain fields equivalent to those found in the single_cell_metadata.tsv related to processing the sample, but will not contain patient or disease specific metadata (e.g. age, sex, diagnosis, subdiagnosis, tissue_location, or disease_timing).

Every download includes a summary report, SCPCL000000_merged-summary-report.html, which provides a brief summary of the samples and libraries included in the merged object. This includes a summary of the types of libraries (e.g., single-cell, single-nuclei, with CITE-seq) and sample diagnoses included in the object, as well as UMAP visualizations highlighting each library.

Every download also includes the individual QC report and, if applicable, cell type annotation reports for each library included in the merged object.

Download folder structure for SingleCellExperiment merged downloads:

project download folder

Download folder structure for AnnData merged downloads:

project download folder

Download folder structure for AnnData merged downloads with CITE-seq (ADT) data:

project download folder

Spatial transcriptomics libraries

If a sample includes a library processed using spatial transcriptomics, the spatial transcriptomics output files will be available as a separate download from the single-cell/single-nuclei gene expression data.

For all spatial transcriptomics libraries, a SCPCL000000_spatial folder will be nested inside the corresponding sample folder in the download. Inside that folder will be the following folders and files:

A full description of all files included in the download for spatial transcriptomics libraries can also be found in the spaceranger count documentation.

Every download also includes a single spatial_metadata.tsv file containing metadata for all libraries included in the download.

sample download with spatial