1 of 24

2025-01-15 NHLBI BioData Catalyst Ecosystem Release Notes

Introduction

The 2025-01-15 release marks the 20th release for the NHLBI BioData Catalyst® (BDC) ecosystem. This release includes several new features (e.g., storage cost savings and PFB handoff of cohort data) along with documentation and tutorials (e.g., video guides to BDC Data Studio environments) to help new users get started on the system. This release also includes enhanced support for working with cohort data. Please find more detail on the new features and user support materials in the sections below.

The 2025-01-15 data releases include the addition of studies on pulmonary fibrosis, COPD, asthma, and congenital heart defects, along with new imaging from atherosclerosis and echocardiogram studies. Updates also include research on cardiovascular health, genetic epidemiology, COVID-19, blood pressure, veteran health, and lifestyle interventions. Please refer to the Data Releases section below for more information as well as the Data page on the BDC website.

Significant new features

Save on storage costs in Terra with bucket lifecycle rules: This feature on BDC Powered by Terra (BDC-Terra) gives users better controls to delete unnecessary workspace bucket files and manage cloud storage costs.

PFB Handoff of Cohort Data from PIC-SURE to Terra: After exploring data and adding filters to build a cohort of interest in BDC Powered by PIC-SURE (BDC-PIC-SURE), investigators can now seamlessly move the participant-level data to BDC-Terra for analysis. This feature allows investigators to bring the data into a new or previously existing BDC-Terra workspace using the Portable Format for Bioinformatics, or PFB, format. This format includes two tables: the participant-level data and the associated data dictionary. and at .

Links to Original Files from Selected Cohort Data: The selected participant-level data from BDC-PIC-SURE is now connected back to the original data file. The data is connected using DRS URIs, a GA4GH standard used to allow access to data in a single, standard way. This allows investigators to refer back to the original source of the BDC-PIC-SURE data. This feature is currently available in the data dictionary table with the PFB formatted BDC-PIC-SURE data. Note: This is currently available for some studies, but the DRS URIs of other studies are being added regularly.

Connect Cohort Data to Genomic Information via Sample Identifiers: Investigators can automatically include sample identifiers when preparing selected cohort data for analysis in BDC-PIC-SURE. The sample identifiers allow researchers to connect the phenotypic information to the associated genomic data or other sample types.

Explore Data with Social Determinants of Health (SDOH) Gravity Domains: Several variables from BDC data have been mapped to SDOH domains from the Gravity Project, a collaborative public-private initiative with the goal of developing consensus-driven data standards to support the collection, use, and exchange of data to address SDOH. These mappings can be used to explore the data in BDC-PIC-SURE.

New user support materials and documentation

Video guides to BDC Data Studio Environments: Three new onboarding videos were created to introduce and orient users to the three kinds of Data Studio environments available on BDC: JupyterLab, RStudio, and SAS Studio. These videos are available on the as platform-generated videos.

Data Releases

The table below highlights which studies were included in the 2025-01-15 data release.

The latest release features NHLBI TOPMed projects, including the San Antonio Family Heart Study (SAFHS), Women's Health Initiative (WHI), and the Trans-Omics for Precision Medicine (TOPMed) Whole Genome Sequencing Project, with updates from the Cardiovascular Health Study (CHS). New additions include the study of African Americans, Asthma, Genes, and Environment (SAGE), the Pulmonary Fibrosis Whole Genome Sequencing project, and the Genetic Epidemiology of COPD (COPDGene). Furthermore, the release highlights studies on the Molecular Genetics of Heterotaxy and Related Congenital Heart Defects, and the Collaborative Cohort of Cohorts for COVID-19 Research (C4R) with data from SPIROMICS and Jackson Heart Study (JHS). Featured are several BioLINCC studies, such as the Systolic Blood Pressure Intervention Trial (SPRINT), Heart Failure Network studies, and the Resuscitation Outcomes Consortium (ROC). This release introduces the Multi-Ethnic Study of Atherosclerosis (MESA) Echocardiogram Image Repository and includes data from the Veterans Administration (VA) Million Veteran Program (MVP) as well as the Healthy Lifestyle Program (HeLP).

The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned upcoming Data Releases

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

For detailed platform release notes please consult the following resources:

BDC Powered by Gen3 release notes

2023-10-04 NHLBI BioData Catalyst Ecosystem Release Notes

Introduction

The 2023-10-04 release marks the 15th release for the NHLBI BioData Catalyst® (BDC) ecosystem. This release includes several new features (e.g., the ability to view cohort variables prior to access, and the ability to export selected data into an analysis workspace). Please find more detail on the new features in the section below.

The 2023-10-04 data releases include the addition of TOPMed studies spanning early-onset COPD, heart studies from various geographies, diabetes heart studies, and more. CRAMs and unharmonized clinical files were updated for six TOPMed studies already in BDC. BioLINCC Multi-Ethnic Study of Atherosclerosis studies were also added. Please refer to the Data Releases section below for more information as well as the on the BDC website.

2023-04-04 BioData Catalyst Ecosystem Release Notes

Introduction

The 2023-04-04 release marks the thirteenth release for the NHLBI BioData Catalyst® (BDC) ecosystem. This release includes several new features, e.g., a new gallery for Public Projects and new project-based download restrictions on BDC Powered by Seven Bridges (BDC-Seven Bridges). It also includes documentation and tutorials to help new users get started on the system, e.g., how to start using the BDC Powered by PIC-SURE (BDC-PIC-SURE) API. Please find more details on the new features and user support materials in the sections below.

Please refer to the Data Releases section below for information on upcoming data releases. A list of currently available data can be viewed on the

2025-01-15 NHLBI BioData Catalyst Ecosystem Release Notes

Introduction

Significant new features

New user support materials and documentation

Data Releases

The table below highlights which studies were included in the 2025-01-15 data release.

The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned upcoming Data Releases

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

For detailed platform release notes please consult the following resources:

BDC Powered by Gen3 release notes

Minimac 4 4.1.2: a tool for imputing genotypes.
GATK 4.4.0.0
- GATK IndexFeatureFile for indexing of provided feature files.
- GATK MergeVcfs for combining multiple variant files.
- GATK VariantEval BETA for evaluating variant calls.
- GATK FilterMutectCalls filter somatic SNVs and indels called by Mutect2.
HTSeq-count 2.0.2: HTSeq-count is a Python tool for counting how many reads map to each feature.
GraphicsMagick 1.3.38
- GraphicsMagick compare compares two images using statistics and/or visual differencing. The tool compares two images and reports difference statistics according to specified metrics, and/or outputs an image with a visual representation of the differences.
- GraphicsMagick composite composites (combines) images to create a new image.
MHC-I Binding Prediction tool (MHC I 3.1.2 toolkit) - which is used for prediction of peptides that bind to MHC I molecules.
MHC-II Binding Prediction tool (MHC II 3.1.6 toolkit) - which is used for prediction of peptides that bind to MHC II molecules.
MHCflurry Predict tool (MHCflurry 2.0.4 toolkit) - which is used for peptide/MHC I binding affinity prediction.
MHCflurry Scan tool (MHCflurry 2.0.4 toolkit) - which is designed to scan protein sequences and predict MHC-I ligands.
AXEL-F: Antigen eXpression based Epitope Likelihood-Function tool (AXEL-F 1.0.0 toolkit) - which is used for MHC-I epitope prediction.
NetChop tool (NetChop 3.0 toolkit) - which is a predictor of proteasomal processing based upon a neural network.
NetCTL tool (NetCTL 3.0 toolkit) - which is a T cell epitopes predictor.
NetCTLpan tool (NetCTLpan 3.0 toolkit) - which is a T cell epitopes predictor.
Class I Immunogenicity tool (Class I Immunogenicity 3.0 toolkit) - which predicts the immunogenicity of a peptide MHC (pMHC) complex.
TCRMatch tool (TCRMatch 1.0.2 toolkit) - which predicts T-Cell receptor specificity based on sequence similarity to characterized receptors.
BCell tool (BCell 3.1 toolkit) - which predicts linear B cell epitopes based on the antigen characteristics.
ElliPro tool (ElliPro 1.0 toolkit) - which predicts antibody epitopes based upon solvent-accessibility and flexibility.
Population Coverage tool (Population Coverage 3.0 toolkit) - which calculates the fraction of individuals predicted to respond to a given set of epitopes.
Epitope Cluster Analysis tool (Epitope Cluster Analysis 1.0 toolkit) - which groups epitopes into clusters based on sequence identity.
Picard 3.0.0 toolkit:
- Picard CollectMultipleMetrics collects BAM statistics by running multiple Picard modules at once.
- Picard ValidateSamFile validates an alignments file against the SAM specification.
MetaCyto workflow (1.16.0 in CWL 1.2): based on R package MetaCyto that performs meta-analysis of both flow cytometry and mass cytometry (CyTOF) data. It is able to jointly analyze cytometry data from different studies with diverse sets of markers.

2021-04-02 BioData Catalyst Ecosystem Release Notes

Introduction

The 2021-04-02 release marks the fifth release for the NHLBI BioData Catalyst ecosystem. This release includes several new features (e.g., CWL tools for QC pipelines) along with documentation and tutorials to help new users get started on the system. This release also includes enhanced support for searching across documentation. Please find more detail on the new features and user support materials in the sections below.

The 2021-04-02 data release includes updates of CRAMs and unharmonized clinical files for 6 TOPMed studies previously hosted on BioData Catalyst. For each study and consent group, VCF files are available on a per chromosome basis and in an un-tarred format.

Please refer to the Data Release section below for more information as well as the on the BioData Catalyst website.

Significant new features

Documentation Search: BioData Catalyst users can now use to search across various types of documentation over the entire ecosystem. Favorite results can be saved in a folder and revisited later.

CWL tools for QC pipelines: Users can now find the following CWL tools for quality control of GWAS data in the:

- This UW-GAC tool calculates heterozygosity by sample.
- This UW-GAC workflow checks expected relationships specified in a pedigree file against empirical kinship values from KING or PC-Relate.

Import files from Kids First Data Resource Center: Users can now access datasets from the Kids First Data Resource portal directly from BioData Catalyst Powered by Seven Bridges using DRS links. Users must have dbGaP approvals for the Kids First datasets in order to access the dataset on BioData Catalyst. In addition, users can import DRS links from open access datasets available via DRS servers.

PIC-SURE Data Access Dashboard: Users on PIC-SURE can now see a list of studies with data available in PIC-SURE. The Data Access Dashboard will show the study name, identifier, and the number of variables/samples present. Additionally the user can see if they have access to the study or click to a link where they can learn more about the study and request access to studies they are not yet authorized to use.

Query annotations for all SNVs and dbSNP INDELS in the Annotation Explorer: Users on Seven Bridges can now use the Annotation Explorer to interactively aggregate and filter all SNVs (over 8 billion variants) and publicly available INDELs from dbSNP using ~700 annotations. Variant grouping files can be created from the results and exported to a workspace for use in rare variant association testing. This database is available to all authenticated users of BioData Catalyst. for more information about how to use the Annotation Explorer.

New user support materials and documentation

were published on Dockstore to help users developing containers and descriptor files for their bioinformatics pipelines.

The has been revamped.

Created and published all materials from the Fellows onboarding session including a recording of the session, all materials used, instructions, etc. Additional webinars will be posted in the coming weeks.

Launched a 3-part video tutorial series on workflows, which helps users, particularly Fellows that are new to the platform, gain more insight into how to best utilize workflows for their data analysis.

- How to run a pre-configured workflow
- How to configure and run a workflow from scratch
- How to run downstream analysis (on the data that resulted from your workflow)

Published a clearly walking researchers through how they can leverage free cloud credits from Google Cloud in Terra. Published a related to the aforementioned one on free cloud credits through GCP. This post covers additional funding sources for covering researchers’ cloud costs, highlighting Google EDU providing up to $10,000 in coupons for supported research projects and the NIH STRIDES initiative. Further, Terra added a covering how the call caching feature in Cromwell can help users save time and money.

Started a new blog post series focused on highlighting papers that may be of interest to the BDCatalyst community. covers a about workflow systems from C. Titus Brown’s lab at UC Davis.

Published a officially announcing that RStudio is available in Terra, and this includes a for getting up and running.

Uploaded a demonstrating the use of Terra for viral genomics by guiding the user through the COVID-19 workspace.

Published a introducing a new feature for task-level checkpointing in workflows. This makes it possible to save intermediate outputs for a task and resume work from that point if the task gets interrupted. Full documentation of this checkpoint feature can be found .

Uploaded a proposing a cross-domain, common data model built specifically to facilitate search and reuse.

Data Releases

The table below highlights which studies were included in the 2021-04-02 data release. CRAMs and unharmonized clinical files were updated for 6 TOPMed studies previously hosted on BioData Catalyst. For each study and consent group, VCF files are available on a per chromosome basis and in an un-tarred format. The data is now available for access across the entire ecosystem.

For detailed platform release notes please consult the following resources:

Gen3 release notes

2020-04-02 BioData Catalyst Ecosystem Release Notes

Introduction

The 2020-04-02 release marks the first significant release for the NHLBI BioData Catalyst ecosystem. This release offers an integrated system of platforms and services for researchers to search metadata of hosted datasets, find data files, and analyze data files in workspace environments which support a variety of different analysis modalities.

The hosted data for this release includes TOPMed multi-sample VCF data for ~55,000 sequenced participants within 32 TOPMed studies included in Freeze 5b as well as CRAM files for those participants. In addition, this release includes raw phenotype files for participants in TOPMed studies, providing clinical information such as BMI and lipids levels. In some cases, these data are in different dbGaP accessions than the genomic data. The hosted data is stored in both Amazon Web Services and Google Cloud and users have the option to run computation on either cloud provider. To access the hosted TOPMed data on BioData Catalyst, users must have dbGaP approval. Please refer to the on the BioData Catalyst website for more information.

For more in depth information please see the "List of significant new features" below.

List of significant new features

The following features in this release support primarily TOPMed researchers ranging in technical skills (both command-line and GUI) and with approval for the controlled TOPMed studies in dbGaP:

System login and data access: Researchers can log into the BioData Catalyst platforms using their eRA Commons ID. Approvals for TOPMed studies in dbGaP are recognized by the platforms.
Search TOPMed phenotypic data: Create cohorts on PIC-SURE by searching and selecting phenotypic variables of interest from dbGaP and then export cohorts to Seven Bridges or Terra for use in analysis workspaces. Users can also explore the TOPMed phenotype variables harmonized by the TOPMed Data Coordinating Center.
Find and access TOPMed genomics files, raw phenotype data files, and reference data files:

Data Releases

Information on the status of data releases is forthcoming.

For detailed platform release notes please consult the following resources:

Gen3 release notes

2024-10-21 NHLBI BioData Catalyst Ecosystem Release Notes

Introduction

The 2024-10-21 release marks the 19th release for the NHLBI BioData Catalyst® (BDC) ecosystem. This release includes several new features (e.g., supporting seqr genomics analysis, and exporting selected cohort data in PFB format). Please find more detail on the new features in the sections below.

The 2024-10-21 data releases include the addition of studies on asthma and sickle cell disease, plus new imaging from cardiovascular and atherosclerosis studies. Updates are highlighted for COPD, atrial fibrillation, and childhood asthma studies, and new additions include liver disease, myocardial genomics, and exRNA studies. The release also introduces the RECOVER-Pediatric project and the REDS-IV-P Epidemiology of COVID-19 study. Please refer to the Data Releases section below for more information as well as the Data page on the BDC website.

Significant new features

BDC Powered by Terra (BDC-Terra) now supports seqr genomics analysis: seqr provides rich gene and variant-level annotations and powerful filtration tools to perform variant searches within a family or across projects. To get started, check out the tutorials, including a video describing how to load your data in seqr.

Export selected cohort data in Portable Format for Biomedical Data (PFB): BDC Powered by PIC-SURE (BDC-PIC-SURE) now allows researchers to export selected participant-level data in PFB file format. When using the Select and Package Data tool in Authorized PIC-SURE, simply choose “Package Data as PFB” to export in this file format.

Data Releases

The table below highlights which studies were included in the 2024-10-21 data release.

The latest release features NHLBI TOPMed projects such as the Severe Asthma Research Program (SARP) and Pharmacogenomics of Hydroxyurea in Sickle Cell Disease (PharmHU). Additionally, it includes new imaging XML schemas from the Cardiovascular Health Study (CHS) and the Multi-Ethnic Study of Atherosclerosis. Updates are also highlighted in the Boston Early-Onset COPD Study, Cleveland Clinic Atrial Fibrillation Study, and the Childhood Asthma Management Program (CAMP). New additions include the Human Liver Cohort and studies on myocardial genomics and exRNA profiles. The release also introduces the RECOVER-Pediatric project and the REDS-IV-P Epidemiology of COVID-19 study.

The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned Upcoming Data Releases

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

For detailed platform release notes please consult the following resources:

BDC Powered by Gen3 release notes

2025-11-15 BDC Ecosystem Release Notes

This is the 23rd update to the BDC ecosystem.

Introduction

The 2025-11-15 release introduces several new features, including expanded data access, new interoperability tools, enhanced compute capabilities, and new bioinformatics workflows.

The 2025-11-15 data releases include the addition of 109 new datasets. See the Data Releases section for more information.

Significant New Features

The following new features this were released this quarter to improve the researcher experience.

BDC Powered by Seven Bridges (BDC-Seven Bridges)

SlicerJupyter, a new Data Studio environment for python-based interaction with 3D Slicer. The environment comes with several example notebooks explaining how to use the functionality for working with volumetric imaging data (for example, MRI, CT).
Expanded Data Access: We enabled full Sequence Read Archive (SRA) access via RAS Passport. This integration unlocks petabytes of controlled SRA data for researchers with valid Data Access Requests (DARs), improving data interoperability and access.
New Interoperability Tools: We published the PFB Unwrapper app, a new CWL tool that accepts .avro PFB files from Gen3, PIC-SURE, or AnVIL and creates a DRS manifest for streamlined import into BDC-Seven Bridges projects. This is a significant step in developing the PFB Importer v2 functionality to seamlessly execute the BDC Handoff Standard.

BDC Powered by Terra (BDC-Terra)

New workspaces now use a more optimized format for data tables to increase performance and scalability.
Locking a workspace now makes the underlying bucket read-only while the workspace is locked.
More information about new features can be found in the .

Data Releases

The table below highlights which studies were included in the data releases in the months of August, September, and October 2025.

Study Name

phs I.D. #

Upcoming Data Releases

The table below highlights studies that are planned for release in November and December.

Study Name

phs I.D. #

For detailed platform release notes please consult the following resources:

Gen3 release notes

2023-07-11 NHLBI BioData Catalyst Ecosystem Release Notes

Introduction

The 2023-07-11 release marks the fourteenth release for the NHLBI BioData Catalyst® (BDC) ecosystem. This release includes several new features, e.g., Faceted Search in BDC Powered by Seven Bridges (BDC-Seven Bridges), along with documentation to help new users get started on the ecosystem, e.g., updated WDL documentation in BDC Powered by Terra (BDC-Terra). This release also includes enhanced support for discovering what datasets are available via BDC Powered by Gen3. Please find more detail on the new features and user support materials in the sections below.

The 2023-07-11 data releases include the addition of various research projects related to COVID-19, lung development, platelet transfusion refractoriness, sickle cell anemia, asthma, pregnancy outcomes, and family health studies. Please refer to the Data Releases section below for information on upcoming data releases. A list of currently available data can be viewed on the of the BDC website.

Significant new features

Faceted Search in BDC-Seven Bridges: Version 1 of Faceted Search has been deployed for all users on BDC-Seven Bridges. This feature enables users to query or filter any BDC ingested data in a faceted way to find files and form groups of files by searching characteristics such as authorization status, study accession number, type of data, etc. With the release of v1 Faceted Search, users can now more easily find data that is relevant to their research. Faceted Search is currently available for 10 datasets and will be expanded to all hosted datasets in the following quarter. The Faceted Search feature can be found under the Data drop-down menu.

BDC-Gen3 Metadata Being Updated to bring data from dbGaP FHIR database: BDC-Gen3’s Discovery Page (and underlying BDC-Gen3 Source of Truth Metadata API) allows unauthenticated users to discover what datasets are available in BDC. Fast Health Interoperability Resources (FHIR) is an Health Level Seven International (HL7) specification for Healthcare Interoperability. Last quarter, BDC-Gen3 worked to consume the new metadata from the dbGaP FHIR Server (as part of the officially defined data ingestion process). This quarter, BDC-Gen3’s Data Ingestion Pipeline has been updated to load FHIR metadata every new data release. The loaded metadata is available to all clients/users through BDC-Gen3’s Metadata API, and loaded metadata is viewable in BDC-Gen3’s Discovery Page.

New and Improved Genomic Filtering on BDC Powered by PIC-SURE (BDC-PIC-SURE): The Genomic Filtering modal on BDC-PIC-SURE has been updated to more accurately represent the relatedness between the various filtering fields. This includes the revamped “Variant consequence calculated” field, which includes different levels of severity and their associated consequences. Additionally, the “Selected Genomic Filters” section now more explicitly summarizes the filter criteria being applied.

Edit Queries Built in BDC-PIC-SURE Using the API: Researchers that created a cohort on BDC-PIC-SURE’s user interface can now edit that query’s parameters using Python or R code via the BDC-PIC-SURE API. This provides more flexibility for researchers wanting to refine or change their cohort after export and eliminates the need to return to the user interface.

New user support materials and documentation

Updated WDL documentation in BDC-Terra: Based on user feedback, Terra documentation has been expanded and updated to include: A new with a section dedicated to resources created by the WDL community, a new wdl-docs website to host the documentation from the new wdl-docs GitHub repository, updates to all existing WDL syntax documentation to match the WDL 1.0 spec, 17 new articles, 11 cookbook-style documents to teach users about specific use cases and provide example workflows, and 6 best practices documents to help users understand some of the grayer areas of coding in WDL. The documents are now available on the new wdl-docs GitHub repository.

New Code in “0_Export_from_UI” BDC-PIC-SURE API Examples: The example code has been updated to include new coding examples on how to use the BDC-PIC-SURE API to edit query parameters of a cohort built in the BDC-PIC-SURE user interface. These examples are available in both Python and R in both Jupyter and RStudio.

Data Releases

The table below highlights which studies were included in the 2023-07-11 data release. The Q2 data release included various research projects related to COVID-19, lung development, platelet transfusion refractoriness, sickle cell anemia, asthma, pregnancy outcomes, and family health studies. These include two studies from the COVID-19 Therapeutic Interventions and Vaccines initiative (ACTIV4a and ACTIV4c). There is a study on lung development (LungMAP) and another tackling platelet transfusion refractoriness in patients with severe thrombocytopenia using Eculizumab (DIR-Eculizumab). Other studies revolve around the use of hydroxyurea in children with sickle cell anemia (BABYHUG), the genetic epidemiology of asthma in Costa Rica (CRA), nulliparous pregnancy outcomes (nuMoM2b), multicenter study of hydroxyurea (MSH), and the Cleveland Family Study (CFS). The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned Upcoming Data Releases

For detailed platform release notes please consult the following resources:

BDC-Gen3 release notes

2024-01-08 NHLBI BioData Catalyst Ecosystem Release Notes

Introduction

The 2024-01-08 release marks the 16th release for the NHLBI BioData Catalyst® (BDC) ecosystem. This release includes several new features (e.g., enabling Azure and searching data without logging in) along with documentation and tutorials (e.g., data dictionary field documentation) to help new users get started on the system. Please find more detail on the new features and user support materials in the sections below.

The 2024-01-08 data releases include the addition of research on multisystem inflammatory syndrome in children linked to COVID-19, bone marrow transplant and pulmonary hypertension in sickle cell disease, atherosclerosis, and psoriasis. Please refer to the Data Releases section below for more information as well as the Data page on the BDC website.

Significant new features

Azure available on BDC Powered by Seven Bridges (BDC-SB): Velsera expanded their existing multi-cloud offerings by enabling Microsoft Azure (southcentralus) on BDC-SB. Users can select that computing and storage environment when creating a project. This allows users to avoid any egress charges when computing on data stored in Azure. This is of particular interest to users who want to connect their own Azure cloud buckets to BDC-SB.

SAS upgrade in BDC-SB: SAS on BDC-SB has been upgraded from SAS Viya 3.5 to SAS Studio 9.4. SAS 9.4 has improved functionality over SAS 3.5 including more complete data management solutions and additional programming languages.

Open PIC-SURE without login: Open PIC-SURE is now publicly available on BDC Powered by PIC-SURE (BDC-PIC-SURE), meaning no eRA Commons credentials are required to access the site. Researchers can access this site to search terms of interest, apply filters at the variable-value level, retrieve obfuscated, aggregate counts, and view single variable distributions of their selected cohort. This new functionality allows researchers to discover and interact with data available on BDC without needing to log in, decreasing the barrier to data exploration. Check out Open PIC-SURE .

Data Hierarchies in BDC-PIC-SURE: Researchers are now able to view the data hierarchy associated with variables in BDC-PIC-SURE by clicking the “Data Tree” icon in the “Actions” column of the search results. This enables researchers to understand better how variables are related and obtain additional context for these variables. Note that this feature is currently in beta and will only be available for some studies. Feedback and input on this feature is welcome!

New user support materials and documentation

BDC-PIC-SURE Data Dictionary fields documentation: Documentation outlining the data dictionary fields returned from the PIC-SURE API was created. This provides a detailed account of what each field represents, including relationships between fields. This documentation can be found in the BDC-PIC-SURE GitBook .

Data Releases

The table below highlights which studies were included in the 2024-01-08 data release. The release features research on long-term outcomes of multisystem inflammatory syndrome in children linked to COVID-19 (COVID19-MUSIC_GRU), bone marrow transplant for severe sickle cell disease (BioLINCC-BMT_CTN_HMB), and ApoA-1, atherosclerosis, and psoriasis (DIR-ApoA-1_Atherosclerosis_in_Psoriasis_GRU). Additionally, updated metadata is provided for the ongoing study on sildenafil therapy in treating pulmonary hypertension in sickle cell disease (walk-PHaSST). This data includes clinical files and is now available for access. The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned Upcoming Data Releases

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

For detailed platform release notes please consult the following resources:

BDC-Gen3 release notes

2022-07-11 BioData Catalyst Ecosystem Release Notes

Introduction

The 2022-07-11 release marks the tenth release for the NHLBI BioData Catalyst ecosystem. This release includes several new features (e.g., importing files from AnVIL via DRS and creating multi-sample VCFs). This release also includes enhanced support for CWL tools on GitHub. Please find more detail on the new features in the sections below.

The 2022-07-11 data release includes the addition of COVID-19 dataset C3PO and TOPMed Freeze 9 batch 3 and 4. Please refer to the Data Release section below for more information as well as the Data page on the BioData Catalyst website.

Significant new features

Import files from AnVIL to BioData Catalyst Powered by Seven Bridges via DRS

Seven Bridges released an interoperability feature enabling . A TOPMed researcher working in BioData Catalyst who identifies a causal variant through association testing might want to next investigate how that variant affects gene expression. The AnVIL ecosystem hosts the Genotype-Tissue Expression (GTEx) datasets which can be used to understand which tissues are affected by novel variants. Seven Bridges’ latest release allows a TOPMed researcher to go to AnVIL and push data they have permissions for to BioData Catalyst Powered by Seven Bridges, thus allowing the researcher to run the variant association test on TOPMed data and identify how that variant changes tissue expression with GTEx data in one workspace.

Create multi-sample VCFs with the Variant Store

Researchers who have access to many TOPMed studies will want to mix and combine VCF files into a multi-sample VCF. Additionally, researchers might want to subset samples based on genomic regions. Using standard bioinformatics tools, this process involves many manual steps and can be time intensive and cost prohibitive. The on BioData Catalyst Powered by Seven Bridges uses a series of API calls to combine VCFs from studies of interest and subset the multi-sample VCF based on the selected genomic region. The latest release allows researchers to track the costs associated with generating multi-sample VCFs via the Variant Store as a dedicated line item in their billing group separate from analysis and storage costs.

Explore, tag, and annotate phenotypes in the Study Variable Explorer

The on BioData Catalyst Powered by Seven Bridges allows researchers to explore phenotypic variables from the TOPMed data dictionaries in an open access manner. Previously, researchers were limited to searching data dictionary information on dbGaP and making comparisons between different study variables was cumbersome with poor UX. Study Variable Explorer enables researchers to select phenotypic variables from across TOPMed studies and view detailed information and distributions of the variable data. By searching keywords, such as obesity, a researcher can compare like variables within and across hosted datasets including the number of subjects and descriptions of the variables. Additionally, users can create custom searchable tags and notes for each variable to track their variable selection and pre-harmonization process.

New CWL Tools and Workflows on BioData Catalyst Powered by Seven Bridges

An updated version of the SRA Download and Set Metadata workflow (SRA Toolkit 3.0.0) that downloads metadata associated with SRA accession via SRA Run Info CGI, (on-demand instance) FASTQ files and sets corresponding metadata.
fastENLOC (v1.0, CWL1.2), a tool that enables integrative genetic association analysis of molecular QTL data and GWAS data. It performs integration of the results from molecular quantitative trait loci (QTL) mapping into genome-wide genetic association analysis of complex traits, with the primary objective of quantitatively assessing the enrichment of the molecular QTLs in complex trait-associated genetic variants and the colocalizations of the two types of association signals.

Dockstore GitHub app support expanded to CWL tools

Researchers can now register your tool to automatically sync with GitHub. Using GitHub Apps, Dockstore can react to changes on GitHub as they are made, keeping Dockstore synced with GitHub automatically. Additional details are available .

Data Releases

The table below highlights which studies were included in the Q2 2022 data releases. The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned Upcoming Data Releases

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

For detailed platform release notes please consult the following resources:

Gen3 release notes PIC-SURE release notes

2023-01-09 BioData Catalyst Ecosystem Release Notes

Introduction

The 2023-01-09 release marks the twelfth release for the NHLBI BioData Catalyst® (BDC) ecosystem. This release includes several new features (e.g., Azure volumes now available on both main analysis platforms) along with documentation and tutorials (e.g., information on how variable tags are generated) to help new users get started on the system. This release also includes enhanced support for moving data seamlessly across platforms. Please find more detail on the new features and user support materials in the sections below.

The 2023-01-09 data releases include the addition of the Pediatric Cardiac Genomics Consortium (PCGC). Please refer to the Data Releases section below for more information as well as the Data page on the BDC website.

Significant new features

Azure volumes are now available on BDC Powered by Seven Bridges: Users can now link a Microsoft Azure bucket to their Seven Bridges workspaces. After logging in, go to Data > Volumes and select “Microsoft Azure” to be led through a bucket-linking wizard.

DRS Manifest Export: In order to further improve interoperability and allow users to move their data in a seamless way across platforms, the DRS export option on the Seven Bridges’ platforms is now available. With the new functionality, users can generate links to platform files (DRS URIs) and metadata into a manifest file, which can then be used for importing the files and metadata on other platforms.

OmicsCircos R Shiny app now available on BDC-Seven Bridges: OmicCircos app is a R Shiny application created around the OmicCircos R package for more effective generation of high-quality circular plots for visualizing genomic data. Common use cases include mutation patterns, copy number variations (CNVs), expression patterns, and methylation patterns. Such variations can be displayed as scatterplot, line, or text-label figures.

Introduction to SAS Public Project on BDC-Seven Bridges: Seven Bridges released a Public Project to train users on how to use SAS. The public project contains three notebooks that walk a user through: 1) loading and cleaning data in SAS using ICD9 codes, 2) pulling the CDC’s Social Vulnerability Index data via API and running a regression, and 3) loading hosted 1000 Genomes data into SAS and visualizing mutation information. A user can copy the public project to their own workspace and modify the tutorial notebooks to suit their needs.

New CWL Tools/Workflows on BDC-Seven Bridges:

BEDTools 2.30.0 toolkit:
- BEDTools Coverage - returns the depth and breadth of coverage of features from B on the intervals in A
- BEDTools Genomecov - computes histograms of feature coverage for a given genome

Azure is now available on BDC Powered by Terra: Users can now log into Terra with a Microsoft Azure Cloud account. This is an invite-only version of Terra on the Azure platform. The public offering of Terra on Azure is expected in early 2023.

A new spend report is now available for BDC-Terra billing projects: The report identifies which workspaces are costing the most, to provide more transparency around cloud costs incurred in Terra. To access the spend report, go to your billing project (main menu > billing > billing project) and click on the "Spend report" tab.

New streamlined user journey from BDC Powered by PIC-SURE to analysis platforms: PIC-SURE has added “Export to Seven Bridges” and “Export to Terra” buttons to streamline data export into a BioData Catalyst analysis workspace. After exploring and filtering variables in PIC-SURE Authorized Access, users can package their data with the Select and Package Data Tool. Once the data is packaged, users can select their preferred BDC analysis platform with the new Export buttons. This provides all information needed and points the user directly to the public PIC-SURE project on either Seven Bridges or Terra.

Take a Tour of BDC-PIC-SURE: PIC-SURE has updated the guided tour of the interface to interactively display search results based on the user’s authorization. This guided tour walks through the different parts of the platform, including how to use tags, where search results are displayed, and how to interpret the Results Panel.

Known issues and workarounds

BABYHUG Data Field Issue: The study BABYHUG, phs002415, contained a data file that included SAS-derived new line characters in data fields. As provided by the data submitter this caused shifts in the data rows, leading to fields being incorrectly mapped to the wrong variable. A new corrected version of the file has been requested from the data submitter.

New user support materials and documentation

BDC-PIC-SURE Tag Generation: PIC-SURE has updated help text in the user interface and documentation to address the frequently asked question, “How are variable tags generated?” Users can find this help text in the “Filter by Variable Tags” box on the PIC-SURE platform and in the .

Updated BDC-PIC-SURE documentation on the Export buttons: The and were updated to include information about the new Export buttons. These updates were also released in the .

BDC GitBook on BDC-PIC-SURE: Users can now access the BDC GitBook documentation directly from the PIC-SURE platform under the “Help” tab.

Data Releases

The table below highlights which studies were included in the 2023-01-09 data release.

The PCGC substudy contains whole exome sequences, targeted sequences, and SNP array data. It is a multi-center, observational cohort study of individuals with congenital heart defects. The study aims to investigate the relationship between genetic factors and phenotypic and clinical outcomes in patients with CHD. Summary level phenotypes for the study participants can be viewed on the top-level study page. Individual level data and molecular data for the study are available by requesting Authorized Access. The study has collected phenotypic data and source DNA from 10,000 probands, parents, and families of interest. The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned Upcoming Data Releases

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

For detailed platform release notes please consult the following resources:

Gen3 release notes PIC-SURE release notes

2022-04-04 BioData Catalyst Ecosystem Release Notes

Introduction

The 2022-04-04 release marks the ninth release for the NHLBI BioData Catalyst ecosystem. This release includes several new features (e.g., machine learning tools for chest CT imaging) along with documentation and tutorials (e.g., a new guide to sharing content) to help new users get started on the system. This release also includes enhanced support for synchronizing tools and workflows between Dockstore and GitHub. Please find more detail on the new features and user support materials in the sections below.

The 2022-04-04 data release includes the addition of COVID-19 datasets ACTIV4a and ACTIV4b. Please refer to the Data Release section below for more information as well as the Data page on the BioData Catalyst website.

Significant new features

Machine learning tools for chest CT imaging: Seven Bridges and Harvard Medical School have collaborated to release a Public Project of machine learning tools titled: Automated Chest Imaging Platform (CIP) CT Phenotyping and Machine Learning Discovery in COPD. The Public Project includes a detailed guide for other researchers to use the tools and notebooks on COPD datasets or modify the tools for their own lung CT data.

Storage optimized instances on Seven Bridges: Users can now access i3 and i3en AWS instances for Interactive Analysis (R Studio, JupyterLabs, SAS Studio) on Seven Bridges. These storage optimized instances provide access to between 5 TB and 60 TB of storage for interactive environments which enables researchers to harmonize larger datasets.

New CWL tools and workflows on Seven Bridges:

short variant discovery 4.2.0.0
toolkit
0.2.4

New user support materials and documentation

Share content through Public Projects: Seven Bridges has published in the knowledge center offering an alternative way to share new workflows, notebooks, and open access data with the BDCatalyst community. Public Projects provide a space for researchers to publish their analyses with open access sample data, detailed walkthroughs, and contact information for feedback and improvements. Both researchers developing new tools and researchers using preconfigured pipelines benefit from published Public Projects.

Dockstore synchronization with GitHub: Dockstore has simplified its tool and workflow registration process to automatically synchronize with GitHub. Dockstore released several for how you can set up your GitHub repo with another file (.dockstore.yml) needed to kick off this process. Check out this for an introduction, and visit the updated Getting Started tutorials for registering and on Dockstore to learn more.

Data Releases

The table below highlights which studies were included in the Q1 2022 data releases. COVID-19 datasets ACTIV4a and ACTIV4b were released to production. Most of the work for ingestion of COVID19-C3PO dataset has been done and will be released in early April. TOPMed Freeze 9 datasets were ingested as the data became available. Twenty datasets were ingested and will be released as part of the fourth batch in early April as well. The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned upcoming Data Releases

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

For detailed platform release notes please consult the following resources:

PIC-SURE release notes

2022-10-03 BioData Catalyst Ecosystem Release Notes

Introduction

The 2022-10-03 release marks the eleventh release for the NHLBI BioData Catalyst ecosystem. This release includes several new features (e.g., PIC-SURE's new search interface) along with updated documentation. This release also includes updated versions of the Study Variable Explorer and the Annotation Explorer. Please find more detail on the new features and user support materials in the sections below.

The 2022-10-03 data releases include the addition of TOPMed Boston-Brazil SCD and PCGC datasets. Please refer to the Data Releases section below for more information as well as the Data page on the BioData Catalyst website.

Significant new features

Now export with Study Variable Explorer on BioData Catalyst Powered by Seven Bridges: The on BioData Catalyst Powered by Seven Bridges allows researchers to explore phenotypic variables from the TOPMed data dictionaries in an open access manner. Seven Bridges released Study Variable Explorer version 2 which expands on version 1 by adding tag search, notes, and data export. The latest update enables researchers to track their variable selection process through notes tied to study and variable information which can be shared with collaborators through .json export. This gives analysts tractable information for reproducing decision-making during the harmonization process.

New Interactive Web Apps Gallery: Under the “Public Gallery” dropdown on BioData Catalyst Powered by Seven Bridges, a new display for “Interactive Web Apps” provides access to the LocusZoom and Model Explorer R Shiny applications.

Annotation Explorer Version 2: The Annotation Explorer enables users to interactively explore, query, and study characteristics of an inventory of annotations for the variants across the genome. This application can be used pre-association testing to interactively explore variant aggregation, filtering strategies, and generate input files for multiple-variant association testing, or post-association testing to explore annotations associated with a set of significant variants or variants of interest. Seven Bridges previously released the Annotation Explorer R Shiny application through a Public Project. Now, Annotation Explorer is integrated with BioData Catalyst Powered by Seven Bridges through the “Data” dropdown. The new integration enables querying genome wide annotations and variants (including the TOPMed Freeze5 and Freeze8 datasets) in a more user-friendly interface without running an R Studio notebook. This release is integrated into the billing system so a user can select their compute needs based on price and monitor Annotation Explorer-specific costs through their billing group.

New CWL Tools and Workflows on BioData Catalyst Powered by Seven Bridges:

GATK VariantEval BETA 4.2.5.0 tool which is used for evaluating variant calls.
GATK FilterMutectCalls 4.2.5.0 tool which is used to filter somatic SNVs and indels called by Mutect2.
Picard CreateSequenceDictionary 2.25.7 tool for creating a DICT index file for a sequence.

Updated Interactive Analysis interface on Terra: Under the new design, the “Notebooks” tab is transformed into the more general “Analyses” tab, from where you can access the multiple applications available for Interactive Analysis in Terra. Accordingly, the list of Notebook files (.ipynb) becomes the list of “Your Analyses”, which now supports including R Markdown files (.Rmd). Just like Notebook files, any R Markdown files created in or added to the Analyses tab will be automatically stored in the workspace bucket and synced between the bucket and your persistent disk.

PIC-SURE's new search interface: PIC-SURE has released an improved dynamic data exploration experience, allowing users to easily search and query at the variable value and genomic variant level. The streamlined search experience enables users to search variables and view associated information, such as decoded variable level information, details about the dataset, and study information - all without opening any data files. Updates to the interface include filtering search results by variable and study tags, a new genomic filtering model, adding variables to export without filtering, a simpler select and package data process, and visualizing single variable distributions.

Dedicated PIC-SURE images within Seven Bridges analysis workspaces: The Seven Bridges and PIC-SURE teams have collaborated to provide users with dedicated workspace images that contain all the pre-installed packages necessary to run the PIC-SURE example notebooks. PIC-SURE API users in Seven Bridges will not have to worry about changes to package dependencies and/or versions, and R users in particular will notice a significantly faster start-up time during environment set-up. The PIC-SURE images are available in both the JupyterLab and RStudio Seven Bridges environments. Users can find this feature by specifying the Environment setup of any Data Cruncher analysis.

Cure Sickle Cell Metadata Catalog integration: PIC-SURE has updated the Data Access Table to integrate information about sickle cell disease (SCD) studies from the (MDC). The “Additional Information” column includes a link to that SCD study’s page on the MDC. The Data Access Table also includes other new information, such as study design and study focus.

New user support materials and documentation

New BioData Catalyst Powered by PIC-SURE search interface: The documentation associated with PIC-SURE has been updated to reflect the recent release of the new search interface. This includes the and the tutorial videos on the .

Updated documentation on new Terra Interface: The documentation associated with Terra has been updated to reflect the recent release of the new analysis interface. This includes the Terra and the tutorial videos on the .

Data Releases

The table below highlights which studies were included in the Q3 2022 data releases. The data is now available for access across the entire ecosystem.

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

Planned Upcoming Data Releases

Study Name

phs I.D. #

Acronym

New to BioData Catalyst

New study version

For detailed platform release notes please consult the following resources:

Gen3 release notes PIC-SURE release notes

2021-01-15 BioData Catalyst Ecosystem Release Notes

Introduction

The 2021-01-15 release marks the fourth release for the NHLBI BioData Catalyst ecosystem. This release includes several new features (e.g., CWL workflows to create dataset specific files needed for GWAS) along with documentation and tutorials to help new users get started on the system. This release also includes enhanced support for CWL tools for post-GWAS analysis and a CWL tool for Bcftools Merge and Filter. Please find more detail on the new features and user support materials in the sections below.

The 2021-01-15 data release includes the addition of both TOPMed studies and the ORCHID Study, conducted by the (PETAL) Clinical Trials Network of NHLBI. Multi-sample VCFs, CRAMs and unharmonized clinical files were added for 27 TOPMed studies new to BioData Catalyst. Additionally, 7 TOPMed studies previously hosted on BioData Catalyst were updated to the latest study versions. These updates include new CRAMs, unharmonized clinical files and multi-sample VCFs for Freeze 8. For each study and consent group, VCF files are available on a per chromosome basis and in an un-tarred format. The associated clinical files were added for the ORCHID study.

Please refer to the Data Release section below for more information as well as the on the BioData Catalyst website.

Significant new features

CWL workflows to create dataset specific files needed for GWAS: Users can now find the following CWL workflows for creating dataset specific files needed for GWAS in the :

- Filter variants based on linkage disequilibrium measures
and - Estimate kinship coefficients
- Perform principal components analysis

CWL tools for post-GWAS analysis: Users can now find the following CWL tools for post-GWAS analysis in the :

- Generate screenshots of specific regions of aligned files provided as inputs
- Standalone tool for generating static locus zoom plots. Users can make annotated Manhattan plots on specific regions from association files generated with the GENESIS association workflows.

CWL tool for Bcftools Merge and Filter: Users can now find a CWL tool for in the Seven Bridges Public Apps Gallery. This tool merges multiple VCF/BCF files from non-overlapping sample sets to create one multi-sample file and filter out any monomorphic variants. This tool is useful when working with input files that contain monomorphic variants like the TOPMed datasets.

New user support materials and documentation

Genetic Association Testing Using the GENESIS Workflows tutorial: Seven Bridges updated this tutorial to show how to perform an association test using the GENESIS workflows using TOPMed Freeze 8 multi-sample VCF data. Previous versions of this tutorial used TOPMed Freeze 5 data. Version 1.1 of this tutorial can be downloaded as a PDF from the .

ORCHID Clinical Trial Statistical Analysis Reproduction: NHLBI BioData Catalyst has made data available to authorized investigators for the study titled: PETAL Network: Outcomes Related to COVID-19 Treated With Hydroxychloroquine Among Inpatients With Symptomatic Disease (ORCHID) Trial, phs002299.v1.p1. This is based on the multi-center, double blinded, randomized clinical trial conducted to assess the efficacy of hydroxychloroquine in the treatment of COVID-19. Results were published in JAMA on November 9th, 2020 (). This notebook enables anybody with authorized credentials to reproduce the ORCHID clinical trial results by showing how to 1) Access the data using the PIC-SURE API and 2) Reproduce the results of this study using the open-source R programming language. Available in or through .

Data release

The table below highlights which studies were included in the 2021-01-15 data release which includes both TOPMed studies and The Outcomes Related to COVID-19 treated with hydroxychloroquine among In-patients with symptomatic Disease study, or ORCHID Study, conducted by the (PETAL) Clinical Trials Network of NHLBI. Multi-sample VCFs, CRAMs and unharmonized clinical files were added for 27 TOPMed studies new to BioData Catalyst. Additionally, 7 TOPMed studies previously hosted on BioData Catalyst were updated to the latest study versions. These updates included new CRAMs, unharmonized clinical files and multi-sample VCFs for Freeze 8. For each study and consent group, VCF files are available on a per chromosome basis and in an un-tarred format. The associated clinical files were added for the ORCHID study. The data is now available for access across the entire ecosystem.

For detailed platform release notes please consult the following resources:

Gen3 release notes

2020-10-23 BioData Catalyst Ecosystem Release Notes

Introduction

The 2020-10-23 release marks the third release for the NHLBI BioData Catalyst ecosystem. This release includes several new features along with documentation and tutorials (e.g., bringing your own data and tools) to help new users get started on the system. This release also includes enhanced support for querying annotations for TOPMed Freeze 8 variants in the Annotation Explorer, and querying combined phenotypic and genomic data in PIC-SURE. Please find more detail on the new features and user support materials in the sections below.

The 2020-10-23 data release includes the addition of both Parent and TOPMed studies. A total of 8 new Parent studies and their respective unharmonized clinical files were added. Multi-sample VCFs, CRAMs and unharmonized clinical files were added for 2 TOPMed studies new to BioData Catalyst. Additionally, 6 studies were updated to the latest version. These updates included new CRAMs, unharmonized clinical files and multi-sample VCFs for Freeze 8. For each study and consent group, VCF files are available on a per chromosome basis and in an un-tarred format. The data is now available for access across the entire ecosystem. Please refer to the Data Release section below for more information as well as the on the BioData Catalyst website.

Significant new features

Form cohorts on Gen3 Exploration page and export to Seven Bridges workspace: Users can now export PFB (Portable Format for Bioinformatics) files from Gen3 (e.g., synthetic cohort files from multiple groups) to Seven Bridges.
CWL workflows for EPACTS and Plink association tests: Users can now find CWL workflows in the Seven Bridges Public Apps Gallery for the association test methods EPACTS and Plink. More information can be found in .
Query annotations for TOPMed Freeze 8 variants in the Annotation Explorer:

New user support materials and documentation

: This guide introduces users to the two Docker-based workflow languages used to run batch analyses in the ecosystem: the Workflow Description Language (WDL) in Terra and the Common Workflow Language (CWL) in Seven Bridges. The guide links to resources that lead users from the early steps of learning to wrap their current pipelines for use in the cloud to how to publish their work in our open access catalog Dockstore to share with the community. This guide was originally conceived in discussion with fellows during the BDCatalyst September Face-to-Face. Fellows developed content and provided feedback and are listed as contributors within the publication.
Benchmarking guide for GENESIS association test workflows: This guide provides users with comprehensive benchmarking information for the CWL versions of the GENESIS association workflows. This guide shows the computation costs and execution times for a variety of association tests using 2.5K samples, 10K samples, 36K samples, and 50K samples run on both AWS and Google Cloud. The benchmarking guide can be found on the page “” of the Seven Bridges documentation.

Data Release

The table below highlights the new data release on BioData Catalyst which includes both Parent and TOPMed studies. A total of 8 new Parent studies and their respective unharmonized clinical files were added to the ecosystem. Multi-sample VCFs, CRAMs and unharmonized clinical files were added for 2 TOPMed studies new to BioData Catalyst. Additionally, 6 TOPMed studies previously hosted on BioData Catalyst were updated to the latest study versions. These updates included new CRAMs, unharmonized clinical files and multi-sample VCFs for Freeze 8 (previously hosted Freeze 5b only). For each study and consent group, VCF files are available on a per chromosome basis and in an un-tarred format. The data is now available for access across the entire ecosystem.

For detailed platform release notes please consult the following resources:

Gen3 release notes

2021-10-04 BioData Catalyst Ecosystem Release Notes

Introduction

The 2021-10-04 release marks the seventh release for the NHLBI BioData Catalyst ecosystem. This release includes several new features (e.g., project cost reporting on Terra and archiving files on AWS) along with documentation and tutorials (e.g., estimating and managing cloud costs) to help new users get started on the system. This release also includes enhanced support for semantic search and R Shiny apps. Please find more detail on the new features and user support materials in the sections below.

The 2021-10-04 data release includes the addition of the final BioLINCC training dataset plus another BioLINCC study, BabyHug. The TOPMed Combined Exchange Area buckets were updated with more datasets from multiple new freezes. The last dataset ingested was PCGC’s CMG. Please refer to the Data Release section below for more information as well as the on the BioData Catalyst website.

Significant new features

Updated Semantic Search UI: Dug, the BioData Catalyst's Semantic Search, has an updated user interface. The new interface makes it easy to see more results on one page. A zoom feature lets users expand individual results to explore in greater detail. Provenance in knowledge graphs and links to published literature are presented where available.

Archive files on AWS: Users on BioData Catalyst Powered by Seven Bridges can now select files to move from AWS S3 storage to AWS Glacier (archival storage). Moving files to archival storage can result in an ~80% cost reduction. It’s recommended that users move files to archival storage if the files will not be used for three or more months.

Project Per Work Space Cost Reporting on Terra: Users on BioData Catalyst Powered by Terra will now have more transparency and access to cost information with the . This update associates each Terra workspace with its own Google Project, created by Terra on behalf of users when workspaces are created. Switching to this “project-per-workspace” model enables added functionality for displaying a breakdown of costs per workspace in the Terra user interface, and allows Terra users to to be notified of cloud spending. This change will only apply to new workspaces created, with plans to migrate existing workspaces over to this model in the future.

Try out R Shiny apps in Terra: Since the rollout of last quarter, Terra’s Interactive Analysis team has expanded the capabilities of the cloud environments framework that supports running RStudio, Jupyter Notebook and in Terra. Most recently, Terra users now have the ability to . Check out an example of an developed by the Manning Lab to visualize whole-genome association data.

Save data from an IA environment: With the new R Shiny apps in Terra, users can . Saving data from an interactive cloud environment (such as an instance of RStudio or a Jupyter notebook) is a useful trick in some situations. Users worried about losing work done in an interactive environment because they need to delete or modify the persistent disk can use "gsutil" to copy it to the workspace bucket.

Speed up machine learning work with GPUs on Terra: Terra’s Interactive Analysis team has released an upgrade that enables . Terra already offered the , and are now responding to user requests to run GPU-enabled computations interactively with GPU support for Jupyter Notebooks.

Speed up workflows and save costs using N2 instances sporting Intel’s 2nd Generation Xeon CPUs on Terra: Terra users will now have the option to use new-generation N2 instances, which have demonstrated faster performance and reduced cost. Read more about these updates and how to request N2 instances for workflows .

New user support materials and documentation

Cross-study harmonization example notebook: will demonstrate how to query and work with the BioData Catalyst studies, particularly cross-study harmonization using the PIC-SURE API.

Estimate and Manage Cloud Costs on Seven Bridges: describes how to estimate costs associated with using Seven Bridges. The tutorial includes an overview of both cloud storage costs and cloud computation costs and the primary drivers of those costs. The tutorial also provides guidance on how to approach estimating cloud storage and computation costs so that researchers can budget for cloud costs in their grants, request cloud credits, and plan their work on BioData Catalyst.

Public project for TOPMed Freeze8 variant calling pipelines: Users on Seven Bridges can now access a public project that walks through how to use the CWL tools and workflows that were used to perform variant calling of TOPMed Freeze8. The public project provides explanations of the purpose of all of the tools and workflows and how they are used together, along with examples of completed analyses. All of the CWL tools and workflows in the project are available in the Public Apps Gallery.

Need an easy way to explain Terra to your colleagues or collaborators? Try this

Estimate Workflow Costs on Terra: Terra users can also follow . This is the original document describing the steps summarized in this blog post.

Understanding and controlling cloud costs on Terra: includes a detailed breakdown of the types of costs that you may incur when working on Google Cloud, plus some advice on how to reduce costs.

Understanding costs and billing on Terra: includes an overview of how billing works, including how billing accounts, projects and workspaces relate to each other, and the difference between workspace permissions and billing permissions.

Controlling cloud costs on Terra – sample use cases: includes a selection of typical analysis use cases, for which the costs are broken down in several scenarios in order to illustrate the effect of cost control strategies.

New tools and workflows released to :

Three additional WDL workflows have been released in the , including KING, PC-Relate, and PC-AIR.
WDL was released to the Utilities collection. This workflow provides the full power of to subset, subsample, and filter VCF files.
New with CWL workflows can predict gene expression (or whatever biology the models predict) in a cohort with available genotypes and run associations to a trait measured in the cohort.

, including Terra:

New to Galaxy? The Galaxy Training Network is continuing to add training material in their on Dockstore.
Additionally, users can explore some of the Galaxy community’s best practices workflows in their on Dockstore.

Ready to publish and share the tool or workflow you developed with the research community? Dockstore users can link their accounts to their ORCID and Zenodo accounts, , and now can .

New video tutorials demonstrate exporting data from PIC-SURE to and sing BioLINCC/Sickle Cell related data.

Data Releases

The table below highlights which studies were included in the 2021-10-04 data release. The final BioLINCC training dataset was uploaded, plus another BioLINCC study, BabyHug. The ORCHID dataset was re-ingested after the data owners found they had provided incorrect versions of the files at the time of initial ingestion. The TOPMed Combined Exchange Area buckets were updated with more datasets from multiple new freezes. The last dataset ingested was PCGC’s CMG. The data is now available for access across the entire ecosystem.

Planned upcoming Data Releases

For detailed platform release notes please consult the following resources:

Gen3 release notes

2021-07-09 BioData Catalyst Ecosystem Release Notes

Introduction

The 2021-07-09 release marks the sixth release for the NHLBI BioData Catalyst ecosystem. This release includes several new features (e.g., SAS on Seven Bridges and Galaxy’s integration with Terra) along with documentation and tutorials to help new users get started on the system (e.g., PIC-SURE Open Access). This release also includes enhanced support for maintaining and versioning CWL on external tool repositories. Please find more details on the new features and user support materials in the sections below.

The 2021-07-09 data release includes the addition of CRAMs and unharmonized clinical files for the parent project CARDIA and 3 other TOPMed programs. The TOPMed program REDS-III received a version update. The unharmonized clinical files were uploaded for the 5 BioLINCC projects and the 2 open tutorial projects. Please refer to the Data Release section below for more information as well as the page on the BioData Catalyst website.

Significant new features

Authentication through the NIH Researcher Authentication Service: The BioData Catalyst ecosystem updated the authentication mechanism to use the NIH Researcher Authentication Service. Researchers will now be redirected to the NIH RAS page to enter their eRA Commons credentials when logging into one of the platforms within the ecosystem.

New Jupyter notebook published in the BioData Catalyst Collection Featured Workspace on Terra.

Galaxy has integrated with Terra: Galaxy is now available through the other “faces” of Terra, including the . You can launch your very own Galaxy server without having to do any configuration yourself, right from the Terra web interface. This marks a transition from alpha to beta development status of Galaxy on Terra, meaning that the software is more mature and considered reliable enough for regular work, with the caveat that minor changes may occur over time as we smooth out any remaining rough edges and improve user experience in the application. Learn more about how to use Galaxy in Terra . Features of Galaxy and its use within Terra are also featured in our blog post . You can also import Dockstore workflows into Galaxy when it's launched in Terra. Speaking of workflows, Cromwell 64 is now live on Terra.

: The RStudio image now has , a tool for single-cell transcriptomics, as well as , a package for verifying the integrity of an object in Google Cloud Storage.

Interactive Analysis: Jupyter Notebook images have been updated with Bioconductor 3.13.0. See the Bioconductor release notes .

SAS: Users on Seven Bridges can now launch SAS for interactive analysis from the Data Cruncher feature. All project files are available within SAS. Users can select from three SAS offerings built on top of SAS Studio: 1) SAS Business Intelligence enables users to utilize SAS code to manage data, create, modify and compare descriptive and predictive models. Capabilities include clustering, decision trees, linear and logistic regression. 2) SAS Analytics adds the power of SAS Viya’s Data Mining and Machine Learning algorithms such as neural networks, gradient boosting, and random forest. 3) SAS Data Science provides access to text analysis, time series models, advanced forecasting and model governance.

LocusZoom Interactive Application: Users on Seven Bridges can now launch an R Shiny application that enables users to select, visualize and interactively explore single variant association test results data, with no prior R programming knowledge. Researchers can explore existing analyses available in the database, generate LocusZoom plots for example data, or provide their own association .RData files. The app also provides the JSONizer tool, which enables researchers to subset their association test results (.RData) files and to convert them into the appropriate JSON files required by LocusZoom. Users can launch the application from the in the top navigation bar.

Example notebook for data import with DRS: These example notebooks on Seven Bridges provide users with the code and steps for importing data from CAVATICA (Kids First data) as well as importing GTEx data from the NHGRI AnVIL system. The import utilizes the DRS functionality to access files that are stored on other NIH cloud systems. Users can find notebooks from the in the top navigation bar.

CWL v1.2 available: BioData Catalyst Powered by Seven Bridges now supports Common Workflow Language (CWL) version 1.2. The new version of CWL brings a major new functionality - , as well as several minor features and improvements. For the detailed change log please see the and the .

New CWL tools and workflows on BioData Catalyst Powered by Seven Bridges: Users can find all these tools and more in the :

- This is a tool for whole genome regression analysis.
- This UW-GAC tool is a standalone app for creating Manhattan and QQ plots from the GENESIS association test results with additional filtering and stratification options available.
- This is a scalable SNV and INDEL annotation pipeline, performing a spectrum of annotations in a single tool. It integrates annotations from dozens of databases and annotation tools.

PIC-SURE Data Access Dashboard Updates: PIC-SURE’s Data Access Dashboard has been updated to include the number of studies and participants the user has access to based on their authorization.

New PIC-SURE Open Access: is now available in BioData Catalyst! PIC-SURE Open Access is available to users who have an eRA Commons account, including those who are not authorized to access any studies. The Open Access feature allows users to explore de-stigmatized, phenotypic data available in PIC-SURE prior to requesting access to data. For more information check out the and .

New PIC-SURE are available as public projects in Seven Bridges and Terra as follows:

Example showing the users how to access lipid measurements across harmonized variables and multiple visits using the PIC-SURE API in R, RStudio, and Python.
All previous notebooks examples are now available in RStudio on Seven Bridges.

New on Dockstore under the BioData Catalyst organization: This collection includes two WDL workflows to help users prepare their data for association testing: one for converting VCF files to GDS and one for linkage disequilibrium pruning. Stay tuned as more workflows are released.

New on Dockstore under the BioData Catalyst organization: The WDL workflows in this collection enable scalable, efficient, and flexible genome-wide gene-environment interaction analysis. GEM conducts single-variant analysis for common variants (currently in unrelated individuals only) and MAGEE conducts single-variant and variant set-based analysis for common or rare variants while allowing for relatedness. The collection also includes examples of cloud costs in the README.

Known issues and workarounds

BioLINCC Phase 2 data dictionaries: These data dictionaries were submitted in PDF format which required additional intervention and delayed general release to the platform. These data dictionaries will be released as soon as is feasible for use across the platform.

New user support materials and documentation

Maintaining and Versioning CWL on External Tool Repositories: presents best practices for writing and maintaining CWL tools/workflows in an external tool repository, such as GitHub, so that users can better manage versions of their tools. Users should follow these best practices if they would like to publish and share their CWL tools and workflows in the since Dockstore has the ability to automatically pull changes from GitHub. These best practices will ensure that the CWL is fully portable and can run successfully not only on Seven Bridges Platforms, but also on other CWL executors such as cwltool and Toil.

Transferring Files Between Seven Bridges and Terra: guides users through the process of transferring files between the two workspace environments Seven Bridges and Terra.

Accessing Egress-Free GTEx Data From AnVIL: A new data interoperability page that includes linked instructions for how to access egress-free GTEx data from NHGRI’s AnVIL cloud ecosystem is .

PIC-SURE Documentation Updates: New provides new information on the Data Access Dashboard, PIC-SURE Open Access, and a new table for understanding study-specific subject identifiers.

PIC-SURE Video Tutorials: are now available for the following topics:

Introduction to PIC-SURE
Introduction to PIC-SURE Open Access: Harmonized
Introduction to PIC-SURE Open Access: One Criterion Search

Published a on the role of a secure cloud ecosystem for supporting infrastructure projects and creating connected communities, highlighting BioData Catalyst as one of several NIH-commissioned infrastructure development projects that involve not just putting data on the cloud but also building the additional layers of services that are necessary to deliver on the extraordinary promise of this new model for data sharing and analysis.

Data Releases

The table below highligts which studies were included in the 2021-07-09 data release. CRAMs and unharmonized clinical files were uploaded for the parent project CARDIA and 3 other TOPMed programs. The TOPMed program REDS-III received a version update. The unharmonized clinical files were uploaded for the 5 BioLINCC projects and the 2 open tutorial projects. The data is now available for access across the entire ecosystem.

Planned upcoming Data Releases

For detailed platform release notes please consult the following resources:

Gen3 release notes

GENESIS tutorial and public project: Seven Bridges has made a public project available that introduces users to the GENESIS R package and related R packages (SeqArray, SeqVarTools, and SNPRelate) used in mixed model association testing in sequence data. The examples in the project help users understand the code that is used in the GENESIS public apps (available on GitHub), prepare data for input to those apps, and interact with the results. The “GENESIS Tutorial” public project can be found in the list of Seven Bridges public projects on the top navigation bar of the platform.
Launch machine learning packages in Jupyterlab Notebooks: Users on Seven Bridges can now use a docker image with pre-installed libraries that support machine learning analyses when working in Jupyterlab Notebooks. This docker image can be found in the Data Cruncher feature: Select “Create new analysis” and then, under the Environment setup menu, select “SB Machine Learning - TensorFlow 2.0, Python 3.7.”
Support for larger GPU instances: A larger is now available for researchers working in Jupyterlab Notebooks and RStudio on Seven Bridges. The p3dn.24xlarge instance has 1800GB SSD, 96vCPUs, 768GB RAM, and 8 GPUs. These higher memory cards enable machine learning training on large 3D images and high-performance computing applications.
Data Cruncher Interactive Analyses: Seven Bridges now features a “” public project, found in the list of public projects on the top navigation bar of the platform, with example analyses to help users interpret results from secondary analysis. The project has eight separate analyses - three in RStudio and five in Jupyterlab Notebooks - including one on VCF visualization and one on structural variant analysis. Read the .
Launch Dockstore workflows in Seven Bridges: Users can now find CWL workflows in Dockstore and launch them in the Seven Bridges workspace environment.
Export large PFBs from Gen3 to Terra: Users can now export large PFB (Portable Format for Bioinformatics) files from Gen3 (e.g. synthetic cohort files from multiple groups) to Terra. New backend systems now automatically parse files more efficiently.
Automatic syncing with GitHub apps: Dockstore now with any changes you make to your linked GitHub repository.
Link ORCID iDs to published workflows: Users can now link their to their Dockstore accounts, and make iDs visible via their organizations, and in workflows and tools they have starred. Users searching Dockstore’s catalog will be able to associate workflows you contribute with scientific publications.
GA4GH TRS Support: Dockstore now implements the (TRS) v2 standard. The goal of the TRS API is to provide a standardized way to describe the availability of tools and workflows.
Transfer datasets to Jupyter Notebooks with Query Id: Users that query the PIC-SURE UI and apply filters to create datasets can submit their query ID to the PIC-SURE client library via an R or Python and do not need to re-build the query manually.
Data Tree Optimizations: The PIC-SURE data tree has been optimized to show users only the studies they have been authorized to see and rendered more efficiently to allow users to select studies faster.
Export data dictionary of clinical variables: are now available that provide directions on exporting the full data dictionary of all clinical variables to a CSV via PIC-SURE.

Overview of the ecosystem: This collaboratively developed overview document guides new users through the process of understanding what the BioData Catalyst is to getting started using the ecosystem.
Tips for reliable and efficient analysis set-up: This guide provides recommendations on how to set up your initial set of analyses, tips for running tools/workflows, and specifications for computational resources on Seven Bridges.
Genetic Association Testing Using GENESIS Workflows: guides users through the steps of running a single variant or multiple variant association test on Seven Bridges using the GENESIS R package pipelines.
Troubleshooting Tasks: presents some of the most common errors in task execution on Seven Bridges and shows you how to debug and resolve them.
GWAS tutorial and example cloud costs: Terra’s walks users through the steps of preparing data for input using Hail in Jupyter notebooks and running association tests as workflows with the GENESIS R package and provides example cloud costs derived from the tutorial.
Code Library: This release includes a Terra containing R and Python Jupyter Notebooks that cover how to use the Integrated Genomics Viewer with data from Gen3, workflows for merging VCF files, and expanded features for interacting with data using the Data Repository Service (DRS) such as bulk downloads.
Dockstore Fundamentals: A are available from the recent workshop Dockstore Fundamentals: Introduction to Docker and Descriptors for Reproducible Analysis.
API and User Interface Technical Documentation: provides users with information about the PIC-SURE API and user interface and examples of how to load data into the PIC-SURE High Performance Data Store.
Scalability and cost-effectiveness analysis of whole genome-wide association studies in the Cloud: from PIC-SURE provides a straightforward and innovative methodology to optimize cloud configuration in order to conduct genome-wide association studies and helps users understand the trade-off between speed and cost.

2025-01-15 NHLBI BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagSignificant new features

hashtagNew user support materials and documentation

hashtagData Releases

hashtagPlanned upcoming Data Releases

hashtagFor detailed platform release notes please consult the following resources:

2023-10-04 NHLBI BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

2023-04-04 BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

2025-01-15 NHLBI BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagSignificant new features

hashtagNew user support materials and documentation

hashtagData Releases

hashtagPlanned upcoming Data Releases

hashtagFor detailed platform release notes please consult the following resources:

2023-10-04 NHLBI BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

2023-04-04 BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagKnown issues and workarounds

hashtagData Releases

hashtagPlanned Upcoming Data Releases

hashtagFor detailed platform release notes please consult the following resources:

hashtagSignificant new features

hashtagNew user support materials and documentation

hashtagData Releases

hashtagPlanned Upcoming Data Releases

hashtagFor detailed platform release notes please consult the following resources:

2021-04-02 BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagSignificant new features

hashtagNew user support materials and documentation

hashtagData Releases

hashtag For detailed platform release notes please consult the following resources:

2020-04-02 BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagList of significant new features

hashtagFor detailed platform release notes please consult the following resources:

Release Notes

2025-04-15 BDC Release Notes

hashtagIntroduction

2025-07-15 BDC Release Notes

hashtag

hashtagSignificant new features

hashtagNew user support materials and documentation

hashtagData Releases

hashtag Planned upcoming Data Releases

2024-10-21 NHLBI BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagSignificant new features

hashtagData Releases

hashtagPlanned Upcoming Data Releases

hashtagFor detailed platform release notes please consult the following resources:

2025-11-15 BDC Ecosystem Release Notes

hashtagIntroduction

hashtagSignificant New Features

hashtagBDC Powered by Seven Bridges (BDC-Seven Bridges)

hashtagBDC Powered by Terra (BDC-Terra)

hashtagData Releases

hashtagUpcoming Data Releases

2023-07-11 NHLBI BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagSignificant new features

hashtagNew user support materials and documentation

hashtagData Releases

hashtagPlanned Upcoming Data Releases

hashtagFor detailed platform release notes please consult the following resources:

2024-01-08 NHLBI BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagSignificant new features

hashtagNew user support materials and documentation

hashtagData Releases

hashtagPlanned Upcoming Data Releases

hashtagFor detailed platform release notes please consult the following resources:

2022-07-11 BioData Catalyst Ecosystem Release Notes

hashtagIntroduction

hashtagSignificant new features

Introduction

Significant new features

New user support materials and documentation

Data Releases

Planned upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction

Introduction

Introduction

Significant new features

New user support materials and documentation

Data Releases

Planned upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction

Introduction

Known issues and workarounds

Data Releases

Planned Upcoming Data Releases

For detailed platform release notes please consult the following resources:

Significant new features

New user support materials and documentation

Data Releases

Planned Upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction

Significant new features

New user support materials and documentation

Data Releases

For detailed platform release notes please consult the following resources:

Introduction

List of significant new features

For detailed platform release notes please consult the following resources:

Introduction

Significant new features

New user support materials and documentation

Data Releases

Planned upcoming Data Releases

Introduction

Significant new features

Data Releases

Planned Upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction

Significant New Features

BDC Powered by Seven Bridges (BDC-Seven Bridges)

BDC Powered by Terra (BDC-Terra)

Data Releases

Upcoming Data Releases

Introduction

Significant new features

New user support materials and documentation

Data Releases

Planned Upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction

Significant new features

New user support materials and documentation

Data Releases

Planned Upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction

Significant new features

Data Releases

Planned Upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction

Significant new features

Known issues and workarounds

New user support materials and documentation

Data Releases

Planned Upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction

Significant new features

New user support materials and documentation

Data Releases

Planned upcoming Data Releases

For detailed platform release notes please consult the following resources:

Introduction