For citation of BDC:

National Heart, Lung, and Blood Institute, National Institutes of Health, U.S. Department of Health and Human Services (2020). The NHLBI BioData Catalyst. Zenodo. https://doi.org/10.5281/zenodo.3822858

To acknowledge BDC, use:

The authors wish to acknowledge the contributions of the consortium working on the development of the NHLBI BioData Catalyst® (BDC) ecosystem.

Our Culture: Though the primary goal of the BDC project is to build a data science ecosystem, at its core, this is a people-centric endeavor. BDC is also building a community of practice working collaboratively to solve technical and scientific challenges in biomedical science.

Principal Investigators (PIs):

• Stan Ahalt, PI RENCI (Coordination Center)

• Rebecca Boyles, Co-PI RTI (Coordination Center)

• Paul Avillach, PI HMS (Team Carbon)

• Kira Bradford, Co-PI RENCI (Team Helium)

• Steve Cox, Co-PI RENCI (Team Helium)

• Brandi Davis-Dusenbery, PI Seven Bridges (Team Xenon)

• Robert Grossman, PI UChicago (Team Calcium)

• Ashok Krishnamurthy, PI RENCI (Team Helium )

• Benedict Paten, PI UCSC (Team Calcium)

• Anthony Philippakis, PI Broad Institute (Team Calcium)

Note: BDC collaboration is organized around teams based on elements in the periodic table. There are additional modes of collaboration in BDC including Tiger Teams, Working Groups, Steering Committee, and Publications.

More about who we are and the partners empowering our ecosystem can be found at the BioData Catalyst About page.

• Agile Development

  Agile software development is an approach to software development under which requirements and solutions evolve through the collaborative effort of self-organizing and cross-functional teams and their customer(s)/end user(s).

• Alpha Users

  A small group of users who are more willing to tolerate working in a system that isn’t as fully developed, providing detailed feedback & maybe some back & forth discussions

• [Amazon] EFS

  [Amazon] Elastic File System a simple, scalable, elastic file system for Linux-based workloads for use with AWS Cloud services and on-premises resources.

• Ambassadors

  A small group of experts that represent the personas featured within the priority User Narratives. For their time and help, Ambassadors will receive early access to the BDC platform, free compute time, monetary fee for time, and relevant travel expenses will be covered.

• App

  1. In Seven Bridges, an app is a general term to refer to both tools and workflows.

  2. App may also refer to persistent software that is integrated into a platform.

• API

  Application Programmer Interfaces. API technologies serve as software-based intermediaries to exchange data.

• AWS

  Amazon Web Services. A provider of cloud services available on-demand.

• BagIt

  BagIt is a hierarchical file packaging format for storage and transfer of arbitrary digital content.

• BDC3

  BioData Catalyst Coordinating Center

• Beta Users

  A slightly larger group than the alpha users who are not as tolerant to a difficult/clunky environment but understand that the version they are using is not polished and they need to give feedback.

• Beta-User Training

  Once the platform is available to a broader audience, we will support freely-accessible online training for beta-users at any time.

• Carpentries Instructor Training Program

  Ambassadors attend this training program to become BDC trainers.

• CCM

  Change Control Management; the systematic approach to managing all changes made to a document or process. Ensures no unnecessary changes are made, all changes are documented, and a process exists for implementing approved change.

• CIO

  Chief Information Officer

• Cloud Computing

  Internet-based computing, wherein computing power, networking, storage or applications running on computers outside an organization are presented to that organization in a secure, services-oriented way.

• Components

  Software units that implement a specific function or functions and which can be reused.

• ConOps

  Concept of Operations

• Consortium

  A collection of teams and stakeholders working to deliver on the common goals of integrated and advanced cyberinfrastructure, leading-edge data management and analysis tools, FAIR data, and HLBS researcher engagement.

• Containers

  A standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another (for example, Docker).

• Command

  In workflow writing, the command specifies the literal command line run (akin to what you would run in the terminal).

• COPDGene

  Chronic Obstructive Pulmonary Disease (COPD) Gene

• Cost Monitoring (level)

  At the Epic Level The Coordinating Center will facilitate this process by developing reporting templates (see example in PM Plan, Financial Management) for distribution to the teams. The BDC teams will complete these templates and send them directly to NHLBI. Each team is responsible for tracking their finances based upon the award conditions and for providing status updates as requested to NHLBI.

• CRAM File Compressed columnar file format for storing biological sequences aligned to a reference sequence. Designed to be an efficient reference-based alternative to the Sequence Alignment Map (SAM) and Binary Alignment Map (BAM) file formats. It optionally uses a genomic reference to describe differences between the aligned sequence fragments and the reference sequence, reducing storage costs. Additionally each column in the SAM format is separated into its own blocks, improving compression ratio. CRAM files typically vary from 30 to 60% smaller than BAM, depending on the data held within them (from Wikipedia).

• CSOC Alpha

  Common Services Operations Center (CSOC): operates cloud, commons, compliance and security services that enable the operation of data commons; has ATO and hosts production system.

• CSOC Beta

  Development/testing; Real data in pilot (not production) that can be accessed by users

• Common Workflow Language (CWL)

  Simple scripting language for describing computational workflows for performing sequential operations on data. CWL is a way to describe command line tools and connect them together to create workflows. CWL is well suited for describing large-scale workflows in cluster, cloud and high performance computing environments where tasks are scheduled in parallel across many nodes.

• DAC

  Data Access Committee: reviews all requests for access to human studies datasets

• DAR Data Access Request

• Data Access

  A process that involves authorization to access different data repositories; part of a User Narrative for the December 2020 release goal A Work Stream PM Plan constraint: NHLBI, as the project sponsor, will identify a process to enable data access by the BDC team members and for research users

• Data Commons

  Provides tools, applications, and workflows to enable computing large scale data sets in secure workspaces.

• Data Repository Service (DRS) Generic interface (API) to data repositories so data consumers, including workflow systems, can access data in a single, standardized way regardless of where it’s stored or how it’s managed. The primary functionality of DRS is to map a logical ID to a means for physically retrieving the data represented by the ID (from GA4GH).

• Data Steward

  Members of the TOPMed and COPDGene communities who are working with BDC teams.

• dbGaP

  Database of Genotypes and Phenotypes

• DCPPC

  Data Commons Pilot Phase Consortium. The Other Transaction Awardees, Data Stewards, and the NIH.

• Decision Tree

  A decision support tool that uses a tree-like model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility

• Deep Learning

  A machine learning method based on neural networks to learn from data through training to recognize patterns in the data.

• Deliverables

  Demonstrations and products.

• Demos

  Activities and documentation resulting from the DCPPC to build, test and demonstrate completion of goals of the Data Commons Pilot Phase.

• DEV Environment

  Set of processes and programming tools used to create the program or software product

• DMI

  Data Management Incident

• Docker

  Software for running containers, packaged, portable units of code and dependencies that can be run in the same way across many computers. See also Containers.

• Dockerfile

  A text document that contains all the commands a user could call on the command line to assemble an image.

• Dockstore

  An open platform developed by the Cancer Genome Collaboratory and used by the GA4GH for sharing Docker-based tools described with the Common Workflow Language (CWL), the Workflow Description Language (WDL), or Nextflow (NFL)

• DOI

  Digital Object Identifier; a code used to permanently and stably identify (usually digital) objects. DOIs provide a standard mechanism for retrieval of metadata about the object, and generally a means to access the data object itself.

• DUO

  Data Use Ontology - a GA4GH standard for automating access (API) to human genomics data (https://github.com/EBISPOT/DUO)

• DUOS

  Data Use Oversight System, https://duos.broadinstitute.org/

• Ecosystem

  A software ecosystem is a collection of processes that execute on a shared platform or across shared protocols to provide flexible services. Example: The "BDC Ecosystem" - inclusive of all platforms and tools

• EEP

  External Expert Panel. A group of experts who provide guidance and direction to NIH about the program.

• Epic

  A very large user story which can be broken down into executable stories

  *NHLBI’s cost-monitoring level

• eRA Commons

  Designated ID provider for whitelist

• External Expert Panel

  An independent body of experts that inform and advise the work of the BDC Consortium.

• FAIR

  Findable Accessible Interoperable Reusable.

• Feature

  A functionality at the system level that fulfills a meaningful stakeholder need

  *Level at which the CC coordinates

• FireCloud

  Broad Institute secure cloud environment for analytical processing, https://software.broadinstitute.org/firecloud/

• FISMA moderate environment

  Federal Information Security Modernization Act of 2014, amends the Federal Information Security Management Act of 2002 (FISMA), see https://www.dhs.gov/fisma

• FS

  Full Stack

• GA4GH

  Global Alliance for Genomics and Health

• GA4GH APIs

  The Genomic Data Working Group is a coalition assembled to create interoperability standards for storing and sharing genomic data. The GA4GH Genomics API offers Interoperability for exchanging genomic data between various platforms and organizations by sending simple HTTP requests through a JSON equipped RESTful API.

• GCP

  Google Cloud Platform

• GCR

  Governance, Compliance, and Risk

• Gen3

  Gen3 is open source and licensed under the Apache license, which you can use for setting up, developing and operating data commons

• GitHub

  An online hub for storing and sharing computer programs and other plain text files. We use it for storage, hosting websites, communication and project management.

• Gold Master

  A gold master, or GM, is the final version of software or data ready for release to production; a master version from which copies can be made.

• GWAS

  Genome-wide Association Study

• HLBS

  Heart, Lung, Blood, Sleep

• Identity Providers

  A system entity that creates, maintains, and manages identity information for principals while providing authentication services to relying applications within a federation or distributed network; identity providers offer user authentication as a service

• Interoperability

  The ability of data or tools from multiple resources to effectively integrate data, or operate processes, across all systems with a moderate degree of effort.

• Instance

  In cloud computing, refers to a virtual server instance from a public or private cloud network.

• Image

  In the context of containers and Docker, this refers to the resting state of the software.

• IP

  BDC Implementation Plan; outlines how the various elements from the planning phase of the BDC project will come together to form concrete, operationalized BDC platform.

• IRB

  Institutional Review Board; the entity within a research organization that reviews and approves research protocols and clinical research protocols to protect human and animal subjects.

• IRC

  Informatics Research Core

• ISA

  Interoperability Service Agreement

• ITAC

  Information Technology Applications Center

• Jupyter Notebooks

  A web-based interactive environment for organizing data, performing computation, and visualizing output.

• Linux

  An open source computer operating system

• Metadata

  Data about other data

• Milestone

  Marks specific progress points on the development timeline, and they can be invaluable in measuring and monitoring the evolution and risk of a program. © Scaled Agile, Inc.

• MSD

  Minimum set of documents

• MVP

  Minimum viable product

• NHLBI

  National Heart, Lung, and Blood Institute

• NIH

  National Institutes of Health

• NIST Moderate controls

  NIST 800-53 - A collection of security controls and assessment procedures that both U.S. Federal and non-governmental organizations can apply to their information systems, policies, and procedures.

• OTA

  Other Transaction Authority - the mechanism of award that NHLBI chose because it provides a degree of flexibility in the scope of the work that is needed to advance this type of high risk/high reward project

• PI

  Principal Investigator

• Platform

  A piece of the BDC ecosystem. Examples: Terra, Gen3, Seven Bridges, etc.

• PM

  Project Manager

• PMP

  BDC Project Management Plan; breaks down the implementation of BDC from the perspective of the project managers involved in the project including details on roles, specific milestones, and the project schedule.

• PO

  Program Officer

• Portable Format for Biomedical Data (PFB) Avro-based serialization format with specific schema to import, export and evolve biomedical data. Specifies metadata and data in one file. Metadata includes data dictionary, ontology references and relations between nodes. Supports versioning, back- and forward compatibility. A binary format.

• Portfolio for Jira

  Software-as-a-Service project management tool, used to track, roadmap, and visualize various project metrics.

• Python

  Open source programming language, used extensively in research for data manipulation, analysis, and modeling

• Quality Assurance

  The planned and systematic activities implemented in quality management so that quality requirements for a product or service satisfy stated goals and expectations.

• Quality Control

  The operational techniques and activities aimed at monitoring and measuring work processes and eliminating the causes of unsatisfactory outputs.

• RACI

  Responsible, Accountable, Consulted and Informed; tool that can be used for identifying roles and responsibilities during an organizational change process; BDC RACI

• Researcher Auth Service (RAS) Will be a service provided by NIH's Center for Information Technology to facilitate access to NIH’s open and controlled data assets and repositories in a consistent and user-friendly manner. The RAS initiative is advancing data infrastructure and ecosystem goals defined in the NIH Strategic Plan for Data Science.

• RFC

  Request for Comment: A process that documents and enables effective interactions between stakeholders to support shared decision making.

• Risk Register

  A tool used to continuously identify risk, risk response planning and status updates throughout the project lifecycle. This project risk register is the primary risk reporting tool, and is located in the Project Management Plan.

• SC

  Steering Committee

• Scientific use case

  Defined in this project as an analysis of data from the designated sources which has relevance and value in the domain of health sciences, probably implementation and software agnostic.

• SF or SFP

  BDC Strategic Framework [Plan]; defines what the BDC teams have accomplished up to this point, what we plan to accomplish in a timeline fashion, and milestones to track and measure implementation.

• SFTP

  Secure File Transfer Protocol

• Software Developers Kit

  A set of software development tools that allows the creation of applications for a certain software package, software framework, hardware platform, computer system, or similar development platform

• Sprints

  Term of art used in software generation, referring to short, iterative cycles of development, with continuous review of code through daily builds and end-of-sprint demos

• Stack

  Term of art referring to a suite of services that run in the cloud and enable ubiquitous, convenient, on-demand access to a shared pool of configurable computing resources.

• Steering Committee

  Responsible for decision-making and communication in BDC.

• STRIDES

  Science & Technology Research Infrastructure for Discovery, Experimentation, and Sustainability

• Task

  In workflow writing, the term task encompasses all of the information necessary to execute a command, such as specifying input/output files and parameters.

• Team

  Groups of people led by a Principal Investigator (PI), or PIs, who will complete milestones and produce deliverables. Each group has been assigned a name, represented by the elements on the periodic chart.

• Tiger Teams

  A diversified group of experts brought together to investigate, solve, build, or recommend possible solutions to unique situations or problems. Populated with mature experts who know what's at stake, what needs to be done, and how to work well with others; their strengths are diversity of knowledge, a single focus or purpose, cross-functional communications, decision-making sovereignty, and organizational agility.

• Tool

  In Common Workflow Language, the term tool specifies a single command. This definition is not as discrete in other workflow languages such as WDL.

• Tool Registry Service (TRS)

  The GA4GH Cloud Work Stream has released a standard API for exchanging tools and workflows to analyze, read, and manipulate genomic data. The Tool Registry Service (TRS) API is one of a series of technical standards from the Cloud Work Stream that together allow genomics researchers to bring algorithms to datasets in disparate cloud environments, rather than moving data around.

• TOPMed

  Trans-Omics for Precision Medicine. One of the primary data sets of the DCPPC.

• TOPMed DCC

  TOPMed Data Coordinating Center

• Trans-cloud

  A provider-agnostic multi-cloud deployment architecture.

• User Narrative

  Descriptions of a user interaction experience within the system from the perspective of a particular persona. User Narratives are further broken down into Features, Epics, and User Stories. Currently formulated into rough 6-month timelines to benchmark progress.

• User story

  A description of a software feature from a technical/process-oriented perspective; a backlog item that describes a requirement or functionality for a user

  *Finest level of PM Monitoring

• Variant Call Format (VCF)

  File format for storing gene sequence variations. The format has been developed with the advent of large-scale genotyping and DNA sequencing projects, such as the 1000 Genomes Project. Existing formats for genetic data such as General feature format (GFF) stored all of the genetic data, much of which is redundant because it will be shared across the genomes. By using the variant call format only the variations need to be stored along with a reference genome. There is also a Genomic VCF (gVCF) extended format, which includes additional information about "blocks" that match the reference and their qualities (from Wikipedia). See http://www.internationalgenome.org/wiki/Analysis/vcf4.0/.

• VDS

  A composite of complete server hardware, along with the operating system (OS), which is powered by a remote access layer that allows end users to globally access their server via the Internet

• VPC

  Virtual Private Cloud

• Whitelist

  A security measure to permit only an approved list of entities. We recommend instead using the term "allow list".

• Workflow

  A sequence of processes, usually computational in this context, through which a user may analyze data.

• Workflow Description Language (WDL) Way to specify data processing workflows with a human-readable and writeable syntax. Define complex analysis tasks, chain them together in workflows, and parallelize their execution.

• Workspace

  Areas to work on/with data within a platform. Examples: projects within Seven Bridges

• Workstream

  A collection of related features; orthogonal to a User Narrative

• Wrapping

  The process of describing a command-line tool or custom script in Common Workflow Language (CWL) so that it can be easily run in multiple cloud environments, dynamically scale compute requirements based on inputs, and be inserted into distinct analysis pipelines.

• Virtual Machine (VM)

  An isolated computing environment with its own operating system.

The BioData Catalyst Consortium is dedicated to providing a harassment-free experience for everyone, regardless of gender, gender identity and expression, age, sexual orientation, disability, physical appearance, body size, race, or religion (or lack thereof). We do not tolerate harassment of community members in any form. Sexual language and imagery is generally not appropriate for any venue, including meetings, presentations, or discussions.

BDCatalyst-RFC-#: 6 BDCatalyst-RFC-Title: Data Access Working Group Data Upload and Download Policy and Recommendations For Users BDCatalyst-RFC-Type: Process Name of the person who is to be Point of Contact: Kira Bradford, Jessica Lyons Email of the person who is to be Point of Contact: kcbradford@renci.org, Jessica_Lyons@hms.harvard.edu Submitting Team: Data Access Working Group Requested BDCatalyst-RFC posting start date: 2020-03-31 BDCatalyst-RFC-Status: Comment only URL Link to this document: https://bdcatalyst.gitbook.io/biodata-catalyst-documentation/community/request-for-comments/data-upload-and-download-policy-and-recommendations-for-users URL Link to the BDCatalyst-RFC: https://www.nhlbidatastage.org/collaboration/rfcs/bdcatalyst-rfc-6 License: This work is licensed under a CC-BY-4.0 license.

DAWG Data Upload and Download Policy and Recommendations For Users

This document describes and defines data movement (data egress and ingress), explaining the types of data movement currently allowable on each platform, and what kinds of data should be downloaded or uploaded. This document is meant to inform the BioData Catalyst Go Live users of the Data Access Working Group’s (DAWG) data upload and download recommendations and policies. These policies and recommendations herein are specific to BioData Catalyst users. The following terminology is in use throughout this document.

Terminology:

• Ecosystem: BioData Catalyst

• Platform: Piece of the BioData Catalyst ecosystem.

  • Examples: Terra, Gen3, Seven Bridges, PIC-SURE

• Workspace: Areas to work on or with data within a platform.

  • Examples: Projects/workspaces within Seven Bridges or Terra

• Individual-level Data: Data at the level of the individual.

• Controlled-access data: Data that is not publicly accessible and requires specific credentials or approvals for access and use, primarily due to study participant consents.

• External data/user uploaded data: Other data sources not hosted on BioData Catalyst, such as a users own created dataset, or data from another source.

• FISMA moderate: The Federal Information Security Management Act (FISMA) creates standards to ensure that all government partners handle confidential and sensitive data appropriately. FISMA moderate is the designated level that means if data is compromised there could be a serious adverse impact, such as a loss of confidentiality, integrity, or availability of data.

• Security boundary: Refers to the technical infrastructure boundaries of a BioData Catalyst platform and Ecosystem.

Definition of Types of Data Movement on BioData Catalyst

We define data movement as the transfer of data, including controlled-access data, in and out of the FISMA moderate security boundaries of the BioData Catalyst ecosystem. Types of data movement are listed below along with available functionality by Go Live:

• Type 1: Uploading and downloading data to a workspace within a platform

  • Uploading: Moving data that exists outside the BioData Catalyst security boundary inside the security boundary (i.e. uploading data) - available by Go Live (See more information below on Permissible Data Upload)

  • Downloading: Moving data from within the BioData Catalyst security boundary outside of the security boundary (i.e. downloading data) - available by Go Live (See Permissible Data Download for data download limitations)

• Type 2: Moving data from one workspace to another workspace within the same platform - available by Go Live

  • Example: move data from one workspace to another workspace within Terra. For example, a user copies a workspace on Terra and creates a new workspace to run a similar analysis in a different Terra workspace.

• Type 3: Moving data from one platform workspace to another platform workspace - not available by Go Live from all platforms (see Table 1 below). Moving data using core services will be available, such as accessing data available from Gen3 and/or curated data from PIC-SURE and moving it to another platform for analysis.

  • Example: If the user generates result files from running an analysis in one workspace (e.g. Seven Bridges workspace), those work files are not currently able to be accessed by a different platform workspace (e.g. Terra workspace).

• Type 4: Sharing controlled-access data brought into or datasets with controlled-access data produced by users in the BioData Catalyst ecosystem with unauthorized users - prohibited for Go Live. This pertains to controlled access data brought into the ecosystem by the user or data available through BioData Catalyst, including data that has been created or altered for analyses. For user-generated results files, the BioData Catalyst consortium does not currently have policies in place or technical implementations to track this. We do not have the technical implementation to track a user bringing their own data on to the BioData Catalyst ecosystem, and sharing it with others. We also do not have any data provenance technical implementation for tracking how a dataset is transformed. Therefore, our policy is that the user is responsible for any external uploaded or transformed data. Users must adhere to all regulations and data use agreements and are solely responsible for the use of any data uploaded and transformed within the ecosystem.

Data Movement Allowed

The following table describes the current types of data movement allowed for BioData Catalyst users based on which platform they are using.

Table 1: Data Movement allowed in BioData Catalyst

Data Movement from core services to platforms is permissible. These include accessing data available from Gen3 and/or curated data from PIC-SURE and moving it to another platform for analysis.

Permissible Data Upload, Bring Your Own Data

Users are permitted to upload data not available to the BioData Catalyst ecosystem (i.e. external data) to their own workspace. Users may upload data into BioData Catalyst if they have the required approvals for such use. These approvals include: Approval of a Data Access Request for controlled access data that includes a BioData Catalyst cloud use statement and the user's institutional review board policies and guidelines. At all times, it is the user's responsibility to ensure they use the data they upload consistent with applicable Data Use Agreements, Data Use Limitations, IRB and any other restrictions on use.

Permissible and Prohibited Data Download

Due to the sensitive nature of data available through the BioData Catalyst ecosystem, users are only allowed to download certain pieces of data/results as outlined in Table 2. It is acknowledged that the technical infrastructure allows for data download on Biodata Catalyst platforms so the responsibility for compliance with data download requirements lies with the user of BioData Catalyst. Results and data that the user would broadly share, such as in an academic publication, may be downloaded through shared workspaces on the BioData Catalyst ecosystem; however, users are strongly encouraged to keep results and data within the BioData Catalyst ecosystem. Users are prohibited from downloading any controlled access, individual-level data (see Table 2). Users should be aware that if they choose to download permitted data and results, that the act of transferring that data through the BioData Catalyst security boundary may or may not be supported by your Data Use Agreement(s), Limitation(s), or Institutional Review Board policies and guidelines. BioData Catalyst users are solely responsible for adhering to the terms of these policies. Users should be aware that all data downloads are logged and regularly reviewed for compliance. See below examples of data permissible and prohibited for download.

Table 2: BioData Catalyst Permissible and Prohibited Download

This list is not exhaustive of all possible scenarios and is subject to change. If you have questions about permissible data download please contact https://biodatacatalyst.nhlbi.nih.gov/contact.

Sharing Uploaded Data with Collaborators

Allowing users to share data with other collaborators is permissible per BioData Catalyst policy, but platforms are not required to have this sharing capability available on BioData Catalyst for Go Live. The BioData Catalyst user is ultimately responsible for maintaining the confidentiality, integrity, and the availability of any data uploaded or downloaded from the BioData Catalyst ecosystem. It is therefore essential that all users of the BioData Catalyst ecosystem accessing controlled access data understand their responsibilities for ensuring appropriate information security controls and that they work with their institutions to effectively implement those responsibilities. Users can upload their data to a workspace; however, it is the responsibility of the uploader to ensure that data policies and permissions are in place to permit data transfer to any users of the shared workspace. Additionally, the uploader should make all collaborators aware of any Data Use Agreements or Limitations of any newly uploaded data.

User Upload/Download Data Policy Reminder

Users will be made aware and reminded of the data upload and download policy recommendations when working in the BioData Catalyst Ecosystem. The user will see this or similar messaging when working on BioData Catalyst platforms.

“You are transferring data through the BioData Catalyst security boundary. Downloading controlled-access, individual-level data through BioData Catalyst is prohibited and downloading other types of data is strongly discouraged, due to the sensitive nature of the data hosted on the platform. Please see the Permissible and Prohibited Data Download section of the Data Upload and Download Policy for more information. Additionally, transferring data may or may not be supported by your Data Use Agreement(s), Limitation(s), or your Institutional Review Board policies and guidelines. As a BioData Catalyst user, you are solely responsible for adhering to the terms of these policies.”

Overview

BDC recognizes the importance of multimedia resources for ecosystem users, particularly audio/visual recordings. This document provides guidelines on the program's video content approach. Using these guidelines will ensure users get optimized video experiences, from consistent branding that offers insights into the sources of the videos to best practices in video creation that support learning.

Overview of BDC Videos

To share video content - from the consortium, platforms, and users, as described in the following sections - BDC created a YouTube channel:

The BioData Catalyst Coordinating Center (BDC3) , has authority (with direction from the NHLBI) to post (or not post), remove, edit, and otherwise change video content on this channel with or without permission from or notice to video creators, owners, or sharers. Feedback about videos on the BDC YouTube channel should be sent to BDCatalystOutreach@nih.gov.

Categories and Organization of Videos

The BDC YouTube Channel hosts three categories of videos based on their sources and/or approval statuses:

• Consortium-produced / Consortium-approved

• Platform-generated

• User-generated

Learn more about each video category below. Note that each category has its own set of standards that must be adhered to when creating and publishing video content, whether the final outlet is the BDC YouTube channel or another channel.

BDC3 is responsible for organizing videos on the BDC YouTube channel, grouping them into playlists it believes will be most beneficial to ecosystem community members. Playlists may include videos from any or all categories of videos. Viewers can determine the category of a video based on the branding (or non-branding) that appears. The additional information about each video category includes video standards that direct video creators on branding for each category of videos.

Consortium-produced / Consortium-approved Videos

Videos in this category are produced by BDC3, or are produced by Platforms or Users that receive approval from the BDC Consortium (select organizations developing and maintaining the ecosystem). These videos contain pre-approved opening and closing BDC animations and sound.

Consortium-produced / Consortium-approved Video Standards

Videos produced by the Consortium, or by Platforms or Users that submit for approval for recognition as a Consortium-approved video, must adhere to the following standards:

Comply with all requirements and, when possible, follow all best practices outlined in Addendum A: Consortium-produced / Consortium-approved Videos Best Practices. Platforms and users generating videos who wish to submit them for recognition as Consortium-approved must complete the BioData Catalyst Consortium Video Submission Pre-Approval Application. Submit the form BEFORE producing the video to improve the likelihood that the video receives Consortium approval.

Platform-generated Videos

Videos in this category are produced by one of the BDC platforms to support users' understanding of their platform. These videos are not vetted by BDC3, BDC3 Consortium members, or representatives of other BDC platforms. These videos must open with the creator's platform "Powered by" logo (downloadable from the BDC3 internal consortium website).

Platform-generated Video Standards

Unless a Platform plans to seek Consortium-approval status for a video, platforms should use the following standards in the production and posting of their platform-generated videos:

Producers of Platform-generated videos, like all BDC ecosystem users, are always obligated to protect participant privacy and must follow NIH policies for data protection. Platforms are accountable and may be subject to sanctions if policies are violated. Only produce videos that provide information specific to the Platform's BDC instance. Use the Platform's Powered by logo (and only the Powered by logo) for the YouTube thumbnail image. Videos should open with the following information: “In this video we will [discuss/cover/explore] BioData Catalyst Powered by [platform name] and [task/example]” YouTube description language should include: The following language: This is a BioData Catalyst platform-generated video to support ecosystem users' understanding of the BioData Catalyst Powered by [platform name]. The link to the NHLBI BioData Catalyst homepage: https://biodatacatalyst.nhlbi.nih.gov/ Videos should be uploaded using YouTube's auto-generated captions to support 508 compliance. Once the video is uploaded, email the link to: BDCatalystOutreach@nih.gov so BDC3 can make it visible on the BioData Catalyst YouTube channel.

Important Notes

• Only videos offering information specific to the use of ecosystem Platform instances will be shared on the BioData Catalyst YouTube channel. Videos that support the use of Platforms but are not specific to BDC instances may be linked from the ecosystem documentation but will not appear on the BioData Catalyst YouTube channel.

• Platform-generated videos that do not follow the above standards will not be made visible on the BioData Catalyst YouTube channel.

User-generated Videos

These videos are neither approved nor vetted by BDC, the BDC Consortium, BDC Platforms, or the organizations they represent. The opinions and other content in these videos are those of the video creators and sharers alone. These videos may NOT open or close with BDC branding and may only display BDC branding when capturing images of properties where it already appears (i.e., a screencap of an ecosystem platform instance).

User-generated Video Standards

BDC offers user-generated video tutorials and guides. Unless a user plans to seek Consortium-approval status for a video, BDC requires the following for user-generated videos, their creators, and their sharers:

Producers of user-generated videos, like all BDC ecosystem users, are always obligated to protect participant privacy and must follow NIH policies for data protection. User institutions are accountable and may be subject to sanctions if policies are violated. By submitting a video for inclusion, users are attesting that the content of the video follows NIH policies for data protection, agreeing to follow this guidance, and committing to the inclusion of the following statement in video descriptions: This is a user-generated video and is neither approved nor vetted by NHLBI BioData Catalyst (BDC), the members of the BDC Consortium, or the organizations they represent. For more information about BDC, go to https://biodatacatalyst.nhlbi.nih.gov/. For more BDC videos, go to https://www.youtube.com/channel/UCGkmY5oNK8uFZzT8vV_9KgQ. #BioDataCatalyst To share a video, please contact: BDCatalystOutreach@nih.gov

Addendum A: Consortium-produced / Consortium-approved Videos Best Practices

Consortium-produced/Consortium-approved videos must adhere to this addendum. While not required of BDC Platforms and users, BDC encourages them to consider these best practices for the videos they produce.

Gaining Approval: Submitting Your Idea

Planning the video: Considerations before recording

Shooting the video: Best practices

Policy compliance: Federal regulations & BDC3 best practices

Technical aspects: Steps after shooting

Publishing & promoting: Publicizing & sharing video

Library maintenance: Keeping an up-to-date catalog

GTEx Data

The Genotype-Tissue Expression (GTEx) Program is a widely used data resource and tissue bank to study the relationship between genetic variants (inherited changes in DNA sequence) and gene expression (how genes are turned on and off) in multiple human tissues and across individuals. For information on access to GTEx data, refer to GTEx v8 - Free Egress Instructions as part of the AnVIL documentation.

NCPI Data Portal

The NIH Cloud Platform Interoperability Effort (NCPI) is currently working to establish and implement guidelines and technical standards to empower end-user analyses across participating cloud platforms and facilitate the realization of a trans-NIH, federated data ecosystem. Participating institutions include BioData Catalyst, AnVIL, Cancer Research Data Commons, and Kids First Data Resource Center. Learn what data is currently hosted by these platforms by using the NCPI Data Portal.

The BDC user community is essential to advancing science with new and exciting discoveries and informing the development of the ecosystem and its infrastructure. Members of the BDC user community learn how to explore the hosted data, use the services, and employ its tools in exciting and valuable ways that even developers may not know. Therefore, we actively invite user resource contributions to be shared with the community.

Types of Resources

Consider supporting fellow ecosystem users in one of the following ways:

• Written Documentation: Develop step-by-step guides, FAQs, checklists, and so on. Include screenshots to support user understanding.

• Videos: Record a shortcut, tip, or process you think would be helpful to other users. Keep videos short by dividing larger processes into smaller segments and recording separate videos for each.

• Respond to inquiries: Answer questions posed in the BioData Catalyst Forums. Forum content with significant engagement may get incorporated into written documentation or made into videos.

Decide How to Share What You Know

Experienced users who want to share their tips and tricks should consider the following questions.

• Did someone already share my tip? Look through the resources already available to users before investing your time and energy into creating a new one.

  • Check the Frequently Asked Questions page on the BioData Catalyst website

  • View the “Learn” and “Documentation” links available on the BioData Catalyst Services webpage.

  • View the BDC Documentation hosted on GitBook.

  • Explore the links to platform-specific documentation, videos, FAQs, community forums, blogs, tutorials, and upcoming events on the BioData Catalyst Learn webpage

  • Check out the videos on the BDC YouTube channel.

• Which format best suits your resource? Ask yourself, "Would I prefer to watch this on video or have a step-by-step guide to help me?" Then ask yourself which you think other users would prefer. Figuring out which you'd prefer is a great place to start because you are the one who identified the tip. But remember that you are creating something to help other people whose preferences will determine whether a resource gets used.

• Is my tip complex, or does it require several steps? If so, a written how-to guide will probably be easier to follow than a video because someone watching a video may need to stop and restart it often. Still, visual aids will be helpful, so consider using screenshots in your how-to guide.

• Is the guidance I want to share relatively straightforward, but it requires clicking through several pages/places? If so, a short video could be the best way to share your tip. Finding buttons or links can be much easier if shown rather than described.

• If I create a video and make sure to go slowly enough that someone can follow along, will it be longer than 15 minutes? If so, creating a video may not be the right format, or breaking down the content into shorter (more digestible) videos may be preferable.

• Am I comfortable following the requirements outlined for the user-generated video tutorials? If not, please create written documentation (e.g., a how-to guide).

• Do I want to provide help in almost-real-time without needing to formally draft a document or record a video? Visit the Community Forum often to provide answers to questions posed by other users or even just post your tip.

Creating and Sharing Your Contribution

Once you decide upon the best way to share what you learned, you'll need to create your contribution and then share it.

• For a quick tip that you want to distribute swiftly, draft something short that you can easily post to the Community Forum. The following is an example of a quick tip for using PIC-SURE’s Data Access Table:

  • In PIC-SURE, did you know you can use the search bar in the Data Access Table to find studies? Instead of scrolling through the table and looking at the list of available studies manually, you can search for studies. An example could be “MESA” for a specific study name, or a phenotype like “Sickle Cell” to find all sickle cell related studies. It seems obvious, but I’m not sure how many other users are aware of this, and I found it really helpful!

• For Written Documentation, draft your suggestions and include screenshots to help lead users through the process you describe. Once complete, submit the file to BDCatalystOutreach@nih.gov for review and posting to the BioData Catalyst Gitbook. Note that we will accept Google Doc (with at least suggesting edits status preferred) and Microsoft Word formats; PDFs are not accepted.

• For videos, review the User-Generated Videos portion of the BDC Video Content Guidance page. By submitting a video, you agree to those conditions. Once your video is uploaded to your YouTube channel, email the link to BDCatalystOutreach@nih.gov for consideration to be linked to the BDC YouTube channel also.

Finding User-Generated User Resources

• Forum messages will post directly in the community forums.

• Written documentation will live in the BDC Documentation, hosted in Gitbook.

• User-generated videos will be linked in the BDC YouTube Channel.

In the context of agile development and a Consortium with a diverse set of members, the application of various agile-development terms may mean different things to different individuals.

The table below defines the BDC Core Terminology:

Strategic Planning Documents Reviewed & Approved by NHLBI Leadership

1. Navigate to to access Dug Semantic Search.

2. Semantic search is a concept-based search engine designed for users to search biomedical concepts, such as “asthma,” “lung,” or “fever,” and the variables related to and/or used to measure them. For example, a search for “chronic pain acceptance” will return a list of related biomedical concepts, such as chronic pain, headaches, neuralgia, or fibromyalgia, each of which can be expanded to display related variables and CDEs. Semantic search can also find variable names and descriptions directly, using synonyms from its knowledge graphs to find search-related variables.

3. Enter a search term and press “Enter,” or click on the Search button. This will take you to the Semantic Search interface.

Click here to access the website.

Welcome to NHLBI BioData Catalyst® (BDC)

Welcome to the BDC ecosystem and thank you for joining our community of practice. The ecosystem offers secure workspaces to support your data analysis in addition to a number of bioinformatics tools for analysis. The ecosystem currently hosts datasets from the Transomics for Precision Medicine (TOPMed) program. There is a lot of information to understand and many resources (documentation, learning guides, videos, etc.) available, so we developed this overview to help you get started. If you have additional questions, please use the links at the very end of this document, under the "Questions" section, to contact us.

About BDC and Our Community

What is BDC?

The BDC ecosystem is a cloud-based platform providing tools, applications, and workflows in secure workspaces. It is designed to be nimble and responsive to the ever-changing conditions of the biomedical and data science communities. Though the primary goal of the BDC project is to build a data science ecosystem, at its core, this is a people-centric endeavor. BioData Catalyst is also building a community of practice working collaboratively to solve technical and scientific challenges.

What are we doing and why does it matter?

By increasing access to the NHLBI’s datasets and innovative data analysis capabilities, the BDC ecosystem accelerates efficient biomedical research that drives discovery and scientific advancement, leading to novel diagnostic tools, therapeutics, and prevention strategies for heart, lung, blood, and sleep disorders.

Who is developing BDC?

The ecosystem is funded by the National Heart, Lung, and Blood Institute (NHLBI). Researchers and other professionals receive funding from the NHLBI to work on the development of the ecosystem, together often referred to as “The BDC Consortium” or “The Consortium” for short. You can find on the about page of the project’s website and a is available in our documentation.

Learn about our culture.

The BDC community follows a that reflects the Consortium’s dedication to providing a harassment-free experience for everyone.

Find out the meanings of our terms and acronyms.

Like many professional communities, BDC has adopted terms to help us communicate quickly and more efficiently, but that can be a challenge for newcomers. To help, we created the NHLBI BioData Catalyst of terms and acronyms. If ever there is a time when an ecosystem term or acronym is unfamiliar and isn’t in the glossary, please so we can give you the information and add it to the glossary for future newcomers.

The BDC Ecosystem and Services

Learn about the platforms and services available in the ecosystem.

The BDC ecosystem features the following platforms and services.

Explore Available Data

• BioData Catalyst Powered by Gen3 - Hosts genomic and phenotypic data and enables faceted search for authorized users to create and export cohorts to workspaces in a scalable, reproducible, and secure manner.

• BioData Catalyst Powered by PIC-SURE - Enables access to all clinical data, feasibility queries to be conducted, and allows cohorts to be built in real-time and results to be exported via the API for analysis.

Analyze Data in Cloud-based Shared Workspaces

• BioData Catalyst Powered by Seven Bridges - Collaborative workspaces where researchers can find and analyze hosted datasets (e.g. TOPMed) as well as their own data by using hundreds of optimized analysis tools and workflows in CWL, as well as JupyterLab and RStudio for interactive analysis.

• BioData Catalyst Powered by Terra - Secure collaborative place to organize data, run and monitor workflow (e.g. WDL) analysis pipelines, and perform interactive analysis using applications such as Jupyter Notebooks and the Hail GWAS tool.

Use Community Tools on Controlled-access Datasets

• Dockstore - Catalog of Docker-based workflows (from individuals, labs, organizations) that export to Terra or Seven Bridges.

Ecosystem Access, Hosted Data, and System Services

How does data access work?

How do I login?

While all of the platforms within BioData Catalyst use eRA Commons credentials and iTrust performs authorization and authentication, respectively, there are some slight differences between the platforms when getting set up:

• BioData Catalyst Powered by Gen3 - Users do not set up usernames on Gen3. Upon the first time logging in, select “Login from NIH”, then enter eRA commons credentials at the prompt. This ‘User Identity’ is used to track the user on the system.

• BioData Catalyst Powered by PIC-SURE - Similar to Gen3, user identities are used - researchers log into the system by selecting “Log in with eRA Commons.”

• BioData Catalyst Powered by Seven Bridges - Users set up platform accounts. The first time on the system, users select to “Create an account” and then proceed with entering their eRA Commons credentials. The user is then prompted to fill out a registration form with their name, email, and preferred username. Users are also asked to acknowledge that they have read the Privacy Act notice and then they can proceed to the platform.

• BioData Catalyst Powered by Terra - Users initially log in using Google credentials and are asked to agree to the Terms of Service and Privacy Act notice. User activity is tracked via the Google credentials, but users can link their eRA Commons credentials to the account to get access to hosted datasets.

How do I check which data I can access?

What data are available in the ecosystem?

Harmonized data available.

Bring your own data and workflows into the system.

Learn about Genome-wide association study and genetic association testing on BioData Catalyst.

Share your workflows.

Costs and cloud credits.

BioData Catalyst Publications

Let us know about your publications and see how you can cite us.

Questions?

Learn more, ask questions, or request help.

Public website

Go to and click Check My Access.

BioData Catalyst powered by Gen3 platform

Go to , select NIH Login, then log in using your NIH credentials. Once logged in, select the Exploration tab. From the Data Access panel on the left, make sure Data with Access is selected. Note whether you have access to all the datasets you expect.

Data Access

Request Access

BioData Catalyst powered by Seven Bridges platform

1. Click your username in the upper right and select Account Settings.

2. Select the tab for Dataset Access.

3. Browse the datasets and note whether you have access to all the datasets you expect.

   • Datasets you have access to will have green check marks.

   • Datasets you do not have access to will have red check marks.

BioData Catalyst powered by Terra platform

You do not need to check your data access on BioData Catalyst powered by Terra. But before submitting a help desk ticket, ensure that you’ve done the following steps:

BioData Catalyst powered by PIC-SURE platform

eRA Commons Account

Users log into BioData Catalyst platforms with their eRA Commons credentials. For more information, see.

Users create an eRA Commons Account through their institution's Office of Sponsored Research or equivalent. For more information, refer to.

dbGaP

Users who want to access a hosted controlled study on the BioData Catalyst ecosystem must be approved for access to that study in the NIH Database of Genotypes and Phenotypes (). For more information, see and. Note that obtaining these approvals can be a time-intensive process; failure to obtain them in a timely manner may delay data access.

Users have two options for obtaining dbGaP approval depending on whether they already are affiliated with a PI who has dbGaP access to the relevant data:

1. The BioData Catalyst user has no affiliation with an existing dbGaP approved project. In this case the user needs to create their own dbGaP project and then submit a data access request (DAR) for approval by the NHLBI Data Access Committee (DAC). This process often takes 2-6 months depending on whether local IRB approval is required by the dataset the user is requesting, the amount of time it takes for local review of the dbGaP application by the user’s home institution and processing by the DAR committee. See thel or. Once a DAR is approved, it can take a week or longer for the approval to be reflected on BioData Catalyst.

2. The BioData Catalyst user is affiliated with an existing principal investigator, who already has an approved dbGaP application with existing DAR access (for example, the BioData Catalyst user is a post-doctoral fellow in a PI’s lab). A principal investigator with dbGaP DAR access assigns the User as a “Downloader” in dbGaP. See. It can take about 24 hours for “Downloader” approval to be reflected on BioData Catalyst.

TOPMed

BioData Catalyst hosts data from the NHLBI Trans-Omics for Precision Medicine (TOPMed) Consortium. BioData Catalyst users are not automatically onboarded as TOPMed investigators. BioData Catalyst users who are not members of the TOPMed Consortium may apply for released data through the regular dbGaP Data Access Request process.

When conducting TOPMed-related research on BioData Catalyst, members of the TOPMed consortium must follow the and associated processes; for example, operating within Working Groups.

For more information, refer to the following resources:

IRB

Users must ensure that IRB data use agreements (DUAs) are approved and maintained as they are enforced by the BioData Catalyst ecosystem.

BioData Catalyst

Requirements

• An NIH eRA Commons ID (or appropriate NIH Login) is required for submitting a Data Access Request (DAR). If you do not have an eRA Commons account, you must request one through your institution’s Office of Sponsored Research or equivalent. For more information, refer to .

• To submit a DAR, users must have PI status through their institution. Non-PI users must have a PI they work with that can submit a DAR and add them as a downloader.

Data Access Request Process

Step 1: Go to to log in to dbGaP.

Step 2: Navigate to My Projects.

Step 3: Select Datasets.

You can search by Primary disease type, or if you know the dataset you are interested in, you can use Study lookup.

We want to request HCT for SCD, so we will use the accession number phs002385. As you type the accession number, the numbers will start to auto-populate.

Select the study to add it to the Data Access Request. You can request up to 200 studies that you are interested in accessing.

The user can add additional datasets as necessary needed to answer the research question.

Sample Research Use Statement

Title

Long-term survival and late death after hematopoietic cell transplant for sickle cell disease

Research Use Statement

Our project is limited to requested dataset. We have no plans to combine with other datasets.

In 2018, the National Heart Lung and Blood Institute (NHLBI) began work on BioData Catalyst, a shared virtual space where scientists can access NHLBI data and work with the digital objects needed for biomedical research (www.nhlbi.nih.gov/science/biodata-catalyst). This is a cloud-based platform that allows for tools, applications and workflows. It provides secure workspaces to share, store, cross-link and analyze large sets of data generated from biomedical research. Biodata Catalyst addresses the NHLBI Strategic Vision objective of leveraging emerging opportunities in data science to facilitate research in heart, lung, blood and sleep disorders. It offers specialized search functions, controlled access to data and analytic tools via programming interfaces and its interoperability will allow exchange of information with other components of the Data Commons. BioData Catalyst may be accessed by the biomedical researchers and the public at large. The first available datasets in BioData Catalyst include data from NHLBI’s Trans-Omics for Precision Medicine (TOPMed) Program and the Cure Sickle Cell Initiative. Rigor in designing and performing scientific research and the ability to reproduce biomedical research are two of the cornerstones of science advancement. In order to test reproducibility of biomedical data available in BioData Catalyst we accessed NHLBI data from the Cure Sickle Cell Initiative to test and validate the findings of a publication that utilized those data. That report, focused on the effect of donor type and transplant conditioning regime intensity on hematopoietic cell transplant outcomes for sickle cell disease. Hematopoietic cell transplant is potentially curative, yet this treatment is associated with risks for mortality from the treatment procedure. Published reports suggest the life expectancy of adults with sickle cell disease in the United States is shortened by at least two decades compared to the general population. Thus, a fundamental question that is often asked is whether hematopoietic cell transplant over time would offer a survival advantage compared to treatment with disease-modifying agents. In the report1 that examined for factors associated with survival after transplantation, young patients (aged =12 years) and patients who received their graft from an HLA-matched sibling had the highest survival. For those without an HLA-matched sibling the data did not favor one alternative donor type over another.1 The purpose of the current analyses is two-fold: 1) test and validate a publication that utilized data in the public domain and 2) the utility of these data to conduct an independent study. The aim of the later study was to estimate the conditional survival rates after hematopoietic cell transplantation stratified by time survived since transplantation and to compare all-cause mortality risks to those of an age, sex, and race-matched general population in the United States.

Non-technical summary

Investigators in the Cure Sickle Cell Initiative Data Consortium request access to data to examine rigor and reproducibility of data submitted - using a previous publication as reference. Additionally, we will calculate survival after hematopoietic cell transplant by time survived since transplant. For example, we will calculate the 5- and 10-year likelihood of being alive 2 and 5 years after transplantation.

Cloud-Use Statement

The NHLBI-supported BioData Catalyst (www.nhlbiBioDataCatalyst.org) is a cloud-based infrastructure where heart, lung, blood, and sleep (HLBS) researchers can go to find, search, access, share, cross-link, and compute on large scale datasets. It will provide tools, applications, and workflows to enable those capabilities in secure workspaces. The BioData Catalyst will employ Amazon Web Services and Google Cloud Platform for data storage and compute. BioData Catalyst comprises the Data Commons Framework Services (DCFS) hosted and operated by the University of Chicago. DCFS will provide the gold master data reference as well as authorization/authentication and indexing services. The DCFS will also enable security interoperability with the secure workspaces. Workspaces will be provided by FireCloud, hosted and operated by the Broad Institute, Fair4Cures, hosted and operated by Seven Bridges Genomics and PIC-SURE operated by Harvard Medical School. For the NHLBI BioData Catalyst, the NHLBI Designated Authorizing Official has recognized the Authority to Operate (ATO) issued to the Broad Institute, University of Chicago and Seven Bridges Genomics as presenting acceptable risk, and therefore the NCI ATO serves as an Interim Authority to Test (IATT) when used by designated TOPMed investigators and collaborators. Additionally, the NHLBI Designated Authorizing Official has recognized the Authority to Operate (ATO) for Harvard Medical School.

Cloud Provider Information

Cloud Provider:

NHLBI BioData Catalyst, Private, The NHLBI-supported BioData Catalyst (https://biodatacatalyst.nhlbi.nih.gov/) is a cloud-based infrastructure where heart, lung, blood, and sleep (HLBS) researchers can go to find, search, access, share, cross-link, and compute on large scale datasets. It will provide tools, applications, and workflows to enable those capabilities in secure workspaces.

The NHLBI BioData Catalyst will employ Amazon Web Services and Google Cloud Platform for data storage and compute. The NHLBI BioData Catalyst comprises the Data Commons Framework Services (DCFS) hosted and operated by the University of Chicago. DCFS will provide the gold master data reference as well as authorization/authentication and indexing services. The DCFS will also enable security interoperability with the secure workspaces. Workspaces will be provided by FireCloud, hosted and operated by the Broad Institute, Fair4Cures, hosted and operated by Seven Bridges Genomics and PIC-SURE operated by Harvard Medical School.

For the NHLBI BioData Catalyst, the NHLBI Designated Authorizing Official has recognized the Authority to Operate (ATO) issued to the Broad Institute, University of Chicago and Seven Bridges Genomics as presenting acceptable risk, and therefore the NCI ATO serves as an Interim Authority to Test (IATT) when used by designated TOPMed investigators and collaborators. Additionally, the NHLBI Designated Authorizing Official has recognized the Authority to Operate (ATO) for Harvard Medical School.

Google Cloud Platform, Commercial

Google Cloud Platform is a public cloud platform that provides solutions and services such as virtual machines, database instances, storage, and more. We will use the Google Compute, a service that provides resizable compute capacity, to allocate Machine Types in which we will develop the methods and infrastructure necessary to build the NHLBI BioData Catalyst. Google Cloud offers several storage options that work in conjunction with Compute Engine: Google Cloud Storage, and Google Compute Engine Persistent Disks. We expect to use each of these as the provide different capabilities including persistent storage and direct and networked storage for attaching to running machine instances. We will use networking technologies based on Google’s Andromeda architecture, which can create networking elements at any level with software. This software-defined networking allows Cloud Platform's services to implement networking features that fit their exact needs, such as secure firewalls for virtual machines in Google Compute Engine. We will use Google Cloud Identity & Access Management to control user access to these compute resources.

BDCatalyst-RFC-#: 11 BDCatalyst-RFC-Title: NHLBI BioData Catalyst Ecosystem Security Statement BDCatalyst-RFC-Type: Consensus Building Name of the person who is to be Point of Contact: Sarah Davis Email of the person who is to be Point of Contact: sdavis@renci.org Submitting Team: BDC3/NHLBI Requested BDCatalyst-RFC posting start date: 6/14/2021 Date Emailed for consideration: 6/14/2021 BDCatalyst-RFC-Status: Comment only URL Link to this document: URL Link to the website: License: This work is licensed under a .

Overview

The purpose of this RFC is to provide the NHLBI BioData Catalyst Consortium and users of the NHLBI BioData Catalyst ecosystem with a clear statement on security mechanisms of the ecosystem that protect the confidentiality, integrity, provenance, and availability of the hosted data as well as any data that may be uploaded using the ecosystem’s “Bring Your Own Data” (BYOD) functionality.

Figure 1. The NHLBI BioData Catalyst ecosystem leverages separately developed and managed platforms to maximize flexibility for users based on their research needs, expertise, and backgrounds. Utilizing multiple Authorizations to Operate (ATO), these platforms combine to provide secure, cloud-based workspaces, user authentication and authorization, search, tools and workflows, applications, and new innovative features to address community needs.

NHLBI BioData Catalyst Ecosystem Security Statement

The NHLBI and BioData Catalyst Consortium recognizes the importance of protecting both the privacy and security of the data and respecting the consent of the study participants whose data is stored within the BioData Catalyst ecosystem. Tackling these issues produces some challenges beyond those faced by most Federal Information Systems. The BioData Catalyst Consortium has implemented many innovative approaches to enable compliance and ensure that users understand their responsibility to protect data as articulated in specific Data Use Agreements (DUA). These approaches and controls work to protect the confidentiality, integrity and availability of the data; the privacy of the study participants who have contributed data; and data that may be uploaded to BioData Catalyst using the ecosystem’s “Bring Your Own Data” (BYOD) functionality. While the same general security controls are applied to both system and BYOD data, BYOD data is further protected as the ecosystem provides access only to the data’s uploaders and their designated collaborators.

From a Federal Information Security Modernization Act (FISMA) perspective, the BioData Catalyst ecosystem is a set of software systems with distinct security boundaries. Each system owner holds an Authority to Operate (ATO) issued by the NIH. The ATO is the result of a rigorous Security Assessment and Authorization (SA&A) process and third party assessment consistent with guidance from the National Institute of Standards and Technology (NIST). The ecosystem operates via a set of Interconnection Security Agreements (ISA) (Reindl 1979) and utilizes several existing components of security infrastructure (Bridges 2017, Gutiérrez-Sacristán et al. 2018) developed for other NIH platforms. Where the documentation provided as part of the SA&A process describes how security controls are implemented based on the NIST Special Publication 800-53r4 (see Endnote), the ISAs describe the permitted exchange of data and establish ecosystem-wide incident response, logging and auditing expectations that enable the consortium to respond in a unified manner to any suspected cybersecurity incident. The SA&A documentation provides for regular evaluation of the security of the component systems including regular scanning for vulnerabilities and the conduct of an annual penetration test. This level of security represents a baseline, and the BioData Catalyst ecosystem will extend protections over time.

Where the processes, policies, and technical controls protect confidentiality, integrity, and availability of data in accordance with Federal statute and regulation, there are additional ways to ensure that data is used in a manner consistent with study participants’ wishes, as represented by the consent form participants sign when enrolling in a specific study. Respect for these consents is critical to maintaining the public’s trust and requires additional policy, process, and technical controls. The respect for consent in NHLBI BioData Catalyst is enforced using normative NIH policies and processes for data sharing and using the existing infrastructure provided by the National Center for Biomedical Information’s (NCBI) Database of Genotypes and Phenotypes (dbGaP). All NHLBI-provided data within the NHLBI BioData Catalyst ecosystem are registered in dbGaP; in this process, data are assigned “consent groups” that describe in a machine-readable format the parameters of the consent for the data. These range from the most expansive “General Research Use” to more restrictive, such as only allowing for research outcomes related to “Health Medical Biomedical topics” or even to specific diseases, such as Chronic Obstructive Pulmonary Disease. Further, while secondary analysis of data is not considered human subjects research as described in the Common Rule (45 CRF Part 46), some datasets require the review of a research proposal by an Independent Review Board (IRB) or a Letter of Collaboration (LOC) with the originating study Principal Investigator as determined by the informed consents or special considerations that the submitting institution has determined are needed. These measures provide additional protection for datasets with particular sensitivity or special criteria for use.

For instance, IRB review is required when 1) informed consents that were signed by the study participants state that IRB oversight for secondary use of the data is required, and/or 2) the study IRB of record determines that the data may contain sensitive information that requires IRB oversight for secondary research. For collaboration letters, the informed consents indicate that the secondary use of the data by researchers outside the study will work with the study and therefore formal collaborations need to be set in place. While these are rarely used, they provide additional protection under special circumstances, such as where an indigenous population or sovereign nation requires direct control of how their data is used. Because consent is expressed at the individual level there may be a variety of consents for a study, either because the study offered choices to their participants or because the study consent evolved over an extended longitudinal study, such as the Framingham Heart Study. These variations in consent are reflected as multiple “consent groups” within a study and may mean that an investigator may only receive permission for subsets of study participants.

Endnote

While NIST-800-53r4 is a thorough standard for many kinds of systems, it has some gaps for the most-modern systems encountered. There are additional standards to apply, either from the High column of NIST-800-53r4 or addendums as “best practice”.

In particular, the Standard does not give guidance on modern applications (“appsec”) -- especially when the Infrastructure is serverless and the entire security surface is the application itself. For instance, there are requirements for regular scanning but scanning tools do not scan APIs or modern Single Page Apps (SPA) sufficiently. A “web app scan” with tools like IBM’s Appscan would return no vulnerabilities on an API without actually testing it, yet it would satisfy RA-5 of NIST-800-53 to run such a scan. Nor does the standard require that the Infrastructure as a Service layer (AWS, GCP, Azure) abide by a continual scanning posture for misconfiguration -- only Networks and VMs are specified. That is one example of where NIST-800-53r4 doesn’t work for modern applications.

There’s also no guidance from NIST at large about running an API where external parties are building “clients'' to that API. Does allowing clients extend the security boundary, and thus all 3rd party applications are to be evaluated as such? Is there a different consideration for these 3rd party applications? The standard is silent there. Companies like Apple and Google through their AppStores require all 3rd party apps to undergo evaluation to their own security standards and such an idea might be applicable here. NHLBI might consider adding some extra controls, what the AllOfUs Program calls FISMA+, for enhanced security.

BioData Catalyst Powered by PIC-SURE has integrated clinical and genomic data from a variety of heart, lung, blood, and sleep related datasets. These include NHLBI Trans-Omics for Precision Medicine (TOPMed) and TOPMed related studies, BioLINCC datasets, and COVID-19 datasets.

View a summary of the data you have access to by viewing the Data Access Table.

This table displays information about the study and associated data, including the full and abbreviated name of the study, study design and focus, the number of clinical variables, participants, and samples sequenced, additional information with helpful links, consent group information, and the dbGaP accession number (or phs number). You are also able to see which studies you are authorized to access in the Access column of the table. For information from dbGaP on submitting a data access request, refer to Tips for Preparing a Successful Data Access Request documentation. Note that studies with a sickle cell disease focus contain links to the Cure SCi Metadata Catalog for additional information.

You can also check the data you have access to by going to the BioData Catalyst Data Access page on the BioData Catalyst website and clicking Check My Access.

The BioData Catalyst ecosystem hosts several datasets from the NIH NHLBI Collaborating Network of Networks for Evaluating COVID-19 and Therapeutic Strategies (CONNECTS) program. These COVID-19 related studies follow the guidelines for implementing common data elements (CDEs) and for de-identifying dates, ages, and free text fields. For more information about these efforts, you can view the CDE Manual and De-Identification Guidance documents on the CONNECTS COVID-19 Therapeutic Trial Common Data Elements webpage.

Table of COVID-19 Studies Included in the CONNECTS Program Available in PIC-SURE

Requirements

To obtain access to BioData Catalyst Powered by PIC-SURE, you must have an NIH eRA Commons account. For instructions and to register an account, refer to the eRA website.

Login

After you have created an eRA Commons account, you can log in to BioData Catalyst Powered by PIC-SURE by navigating to https://picsure.biodatacatalyst.nhlbi.nih.gov and selecting to log in with eRA Commons. You will be directed to the NIH website to log in with your eRA Commons credentials. After signing in and accepting the terms of the agreement on the NIH RAS Information Sharing Consent page, allow the BioData Catalyst Powered by Gen3 service to manage your authorization.

Upon login, you will be directed to the Data Access Dashboard. This page provides a summary of PIC-SURE Authorized Access, PIC-SURE Open Access, and the studies you are authorized to access.

The BioData Catalyst ecosystem hosts several datasets from the NHLBI Trans-Omics for Precision Medicine (TOPMed) program. The PIC-SURE platform has integrated the clinical and genomic data from all studies listed in the Data Access Dashboard. Through the ingestion process, occasionally PIC-SURE will ingest phenotypic data for the TOPMed studies prior to the genomic data.

Harmonized Data (TOPMed Harmonized Clinical Variables)

There are limited amounts of harmonized data available at this time. The TOPMed Data Coordinating Center (DCC) curation team has identified 44 variables that are shared across 17 NHLBI studies and normalized the participant values for these variables.

The 44 harmonized variables available are listed in the table in Appendix 2. For more information on this initiative, you can view the additional documentation from the TOPMed DCC GitHub repository or on the NHLBI Trans-Omics for Precision Medicine website.

Table of Studies Included in the TOPMed Harmonized Dataset Available in PIC-SURE

What is the PIC-SURE API?

Databases exposed through the PIC-SURE API encompass a wide heterogeneity of architectures and data organizations underneath. PIC-SURE hides this complexity and exposes the different databases in the same format, allowing researchers to focus on the analysis and medical insights, thus easing the process of reproducible sciences. The API is available in two different programming languages, python and R, allowing investigators to query databases in the same way using either of those languages. The PIC-SURE API tutorial notebooks can be directly accessed on .

PIC-SURE Access Token

To access the PIC-SURE API, a user-specific token is needed. This is the way the API grants access to individual users to protected-access data. The user token is strictly personal; do not share it with anyone. You can copy your personalized access token by selecting the User Profile tab at the top of the screen.

Here, you can Copy your personalized access token, Reveal your token, and Refresh your token to retrieve a new token and deactivate the old token.

Analysis in the BioData Catalyst Ecosystem

From the Analyze Data in Cloud-based Shared Workspaces section, select Launch for your preferred analysis platform.

BioData Catalyst Powered by Seven Bridges

Jupyter notebook examples in R and python can be found under the Public projects tab by selecting PIC-SURE API.

From the Data Studio tab, select an example that fits your research needs. Here, we will select PIC-SURE JupyterLab examples.

This will take you to the PIC-SURE API analysis workspace, where you can view the examples in python. Copy this workspace to your own project to edit or run the code yourself.

BioData Catalyst Powered by Terra

Select the Public tab and search for “PIC-SURE”. Workspaces for both the python and R examples will be displayed. You must clone the workspaces to edit or run the code within them.

PIC-SURE provides two ways to search: PIC-SURE Open Access and PIC-SURE Authorized Access. enables the user to explore aggregate-level data without any dbGaP data authorizations. feature allows the user to explore participant-level data and requires authorization to access at least one study through an active dbGaP Data Access Request (DAR).

Table Comparison of PIC-SURE Open and Authorized Access

The PIC-SURE v2 API is a meta-API used to host any number of resources exposed through a unified set of generalized operations.

PIC-SURE Repositories:

• PIC-SURE API: This is the repository for version 2+ of the PIC-SURE API.

• PIC-SURE Wiki: This is the wiki page for version 2+ of the PIC-SURE API.

• BioData Catalyst PIC-SURE: This is the repository for the BioData Catalyst environment of PIC-SURE.

• PIC-SURE-ALL-IN-ONE: This is the repository for PIC-SURE-ALL-IN-ONE.

Additional PIC-SURE Links:

• DCPPC Presentation on PIC-SURE as a meta-API

• Avillachlab-Jenkins Repository: A link to the Avillach Lab Jenkins repository.

• Avillachlab-Jenkins Dev Release Control: A repository for Avillach Lab Jenkins development release control.

Client Libraries

The following are the collected client libraries for the entire PIC-SURE project.

• R Client Library

• Python Client Library

PIC-SURE User Interface

The PIC-SURE User Interface acts as a visual aid for running normal queries of resources through PIC-SURE.

PIC-SURE User Interface Repositories:

• PIC-SURE HPDS UI: The main High Performance Data Store (HPDS) UI repository.

Additional PIC-SURE User Interface Links:

• PIC-SURE UI Flow: Links to a google drawing of the PIC-SURE UI flow.

PIC-SURE Auth Micro-App (PSAMA)

The PSAMA component of the PIC-SURE ecosystem authorizes and authenticates all actions taken within PIC-SURE.

PSAMA Repos:

• PIC-SURE Auth MicroApp Repository

Additional PSAMA Links:

• PSAMA Core Logic: This is where the core of the PSAMA application is stored in GitHub

High Performance Data Store (HPDS)

HPDS is a datastore designed to work with the PIC-SURE meta-API. It grants researchers fast, dependable access to static datasets and the ability to produce statistics-ready dataframes filtered on any variable they choose at any time.

HPDS Repositories:

• PIC-SURE HPDS: The main HPDS repository.

• PIC-SURE HPDS Python Client: Python client library to run queries against a PIC-SURE HPDS resource.

• PIC-SURE HPDS R Client: R client library to run queries against a PIC-SURE HPDS resource.

• PIC-SURE HPDS UI: The main HPDS UI repository.

• HPDS Annotation: This repository describes steps to prepare and annotate VCF files for loading into HPDS.

1. Search bar: Enter any phenotypic variable, study or table keyword into the search bar to search across studies. Users can also search specific variables by accession number, if known (phs/pht/phv).

2. Study Tags: Users can filter the results found through their search by limiting to studies of interest or excluding studies.

3. Variable Tags: Users can filter the results found through their search by limiting to keywords of interest or excluding keywords that are out of scope. For example, a user could filter to categorical variables, variables containing the term ‘blood’, and/or exclude variables containing the term ‘pressure’.

   How are variable tags generated? Each variable has a set of associated tags, which are generated during the PIC-SURE data loading process. These tags are generated based on information associated with the variable, including the name of the study, study description, dataset name, PIC-SURE data type (continuous or categorical), and variable description. For a search in PIC-SURE, tags associated with a variable are displayed. Note that tags applicable to less than 5% or more than 95% of the search results are not displayed since these are not useful for filtering results.

4. Search Results table: View all variables associated with your search term and/or study & variable tags.

5. Results Panel: Panel with content boxes that describe the cohort based on the variable filters applied to the query.

6. Data Summary: Displays the total number of participants in the filtered cohort which meet the query criteria. When first opening the Open or Authorized Access page, the number will be the total number of participants that you can access.

7. Added Variable Filters summary: View all filters which have been applied to the cohort.

8. Filter Action: Click on the filter icon to filter cohort participants by specific variable values.

9. Reset button: Allows users to start a new search and query by removing all added filters and clearing all active study and variable tags.

Table of BioData Catalyst dbGAP/TOPMed Identifiers

Table of PIC-SURE Identifiers

Video Walkthroughs

Playlist

Videos

Login to the BioData Catalyst Gen3 Platform

In order to navigate and access data available on the Gen3 platform, start by visiting the login page. You will need an eRA Commons account as well as access permissions through the Database of Genotypes and Phenotypes (dbGaP). If you are a researcher, login by selecting NIH Login and using your eRA Commons account. BioData Catalyst consortia developers can login using their Google accounts. Make sure to use the correct login method that contains access to your available projects.

Once logged in, your username will appear in the upper right-hand corner of the page. You will also see a display with aggregate statistics for the total number of subjects, studies, aliquots and files available within the BioData Catalyst platform.

NOTE: These numbers may differ from those displayed in the dbGaP records as they include TOPMed studies as well as the associated parent studies.

Types of Hosted Data

Phenotypic

DCC Harmonized clinical data:

A number of clinical variables have been harmonized by the Data Coordinating Center (DCC) in order to facilitate cross-study analysis. Faceted search over the DCC Harmonized Variables is available via the Exploration page, under the "Data" tab.

Unharmonized clinical data:

Unharmonized clinical files are also available on the Gen3 platform and contain all of the raw phenotypic information for the hosted studies. Unlike the DCC Harmonized Variables, these files are located and searchable under the "Files" tab in the Exploration page.

Genomic

The Gen3 platform hosts genomic data provided by the Trans-Omics for Precision Medicine (TOPMed) program and the 1000 Genomes Project plus synthetic tutorial data from Terra. At present, these projects include CRAM and VCF files together with their respective index files. Specifically for TOPMed projects, each project will contain at least one multi-sample VCF that comprises all subjects within the consent group. CRAM and VCF are based on an individual level, whereas multi-sample VCFs are based on the study consent level.

All files are available under the "Files" tab in the Exploration page. More detailed information on currently hosted data on the Gen3 platform can be found here.

Gen3 Pages

The BioData Catalyst Gen3 platform contains five pages described below:

• Dictionary: An interactive data dictionary display that details the contents and relationships between clinical and biospecimen data

• Exploration: The facet filter custom cohort creation tool

• Query: The GraphQL query tool to retrieve specific data within the graph model

• Workspace: The launch page for Gen3 workspaces that includes Jupyter Notebooks and RStudio

• Profile: The information page for each user, displaying access and the location for credential file downloads

Overview

The Query page can search and return metadata from either the Flat Model or the Graph Model of a commons. Using GraphQL, these searches can be tailored to filter and return fields of interest for the data sets being queried. These queries can be made immediately after data submission as this queries the model directly.

Profile Page

The Profile page contains two sections: API keys and Project access.

API key(s)

To download large amounts of data, an API key will be required as a part of the gen3-client. To create a key on your local machine, click Create API key, which will activate the following pop-up window:

Click Download json to save the credential file to your local machine. After completion, a new entry will appear in the API key(s) section of the Profile page. It will display the API key key_id and the expiration date (one month after the key creation). The user should delete the key after it has expired. If for any reason a user feels that their API key has been compromised, the key should be deleted before subsequently creating a new one.

Project Access

This section of the Profile page lists the projects and the methods of access for the data within in the Gen3 BioData Catalyst system. If you do not see access to a specific study, check that you have been granted access within dbGaP. If access has been granted for over a week, contact the BioData Catalyst Help Desk: bdcat-support@datacommons.io

Using Exploration

The Exploration page located in the upper right-hand section of the toolbar allows users to search through data and create cohorts. The Exploration portal contains a dynamic summary statistics display, as well as search facets leveraging the DCC Harmonized Variables.

Data Accessibility

Users can navigate through data on the Exploration page by selecting any of the three Data Access categories.

• Data with Access: A user can view all of the summary data and associated study information for studies the user has access to, including but not limited to Project ID, file types, and clinical variables.

• Data without Access:

  • Locks next to the project ID signify to users that they do not have subject-level access but they can still search through the available studies but only view summary statistics. Users can request access to data by visiting the dbGaP homepage.

  • Projects will also be hidden if the select cohort contains fewer than 50 subjects (50 ↓, "You may only view summary information for this project", example below); in this case grayed out boxes and locks both appear. An additional lock means users have no access.

• All Data: Users can view all of the data available in the BioData Catalyst Gen3 platform, including studies with and without access. As a result, studies not available to a user will be locked as demonstrated below.

By default, all users visiting the Exploration page will be assigned to Data with Access.

The Data Tab

Under the "Data" tab, users can leverage the DCC harmonized variables to create custom cohorts. When facets are selected and/or updated to cover a desired range of values, the display will reflect the information relevant to the new applied filter. If no facets have been selected, all of the data accessible to the user will be displayed. At this time, a user can filter based on three categories of clinical information:

• Project: Any specifically defined piece of work that is undertaken or attempted to meet a single investigative question or requirement.

• Subject: The collection of all data related to a specific subject in the context of a specific experiment.

• Harmonized Variables: A selection of different clinical properties from multiple nodes, defined by the Consortium.

NOTE: The facet filters are based on the DCC Harmonized Variables, which are a selected subset of clinical data that have been transformed for compatibility across the dbGaP studies. TOPMed studies that do not contain harmonized clinical data at this time will be filtered out when a facet is chosen, unless the no data option is also selected for certain facets.

Exporting Data from the Data Tab

After a cohort has been selected, the user has four different options for exporting the data.

Export

The options for export are as follows:

• Export All to Terra : Initiate a Portable Format for Bioinformatics (PFB) export of all clinical data and file GUIDs for the selected cohort to BioData Catalyst powered by Terra. At this time the max number of subjects that can be exported to Terra is 120,000.

• Export All to Seven Bridges: Initiate a Portable Format for Bioinformatics (PFB) export of all clinical data and file GUIDs for the selected cohort to BioData Catalyst powered by Seven Bridges.

• Export to PFB : Initiate a PFB export of all clinical data and file GUIDs for the selected cohort to your local storage.

• Export to Workspaces : Export a manifest to the user's workspace and make the case-associated data files available in the workspace under the /pd/data directory.

NOTE: PFB export times can take up to 60 minutes, but often will complete in less than 10 minutes.

The Files Tab

The Files tab displays study files from the facets chosen on the left-side panel (Project ID, Data Type, Data Format, Callset, and Bucket Path). Each time a facet selection is made, the data summary and displays will update to reflect the applied filters.

Locating Unharmonized Clinical Data

The Files tab also contains files that are either case-independent or project-level. This is important for files that are part of the Unharmonized Clinical Data category under the Data Type field. Unharmonized clinical files are made available in two distinct data formats:

• TAR : Contain a complete directory of phenotypic datasets as XML and TXT files that are direct downloads of unharmonized clinical data from dbGaP on a study consent level project.

• AVRO: These files are the same as the unharmonized clinical data from dbGaP as the TAR files, but in form of a PFB file.

• XML: These files contain either dictionary or variable reports of the phenotypic datasets that are in the TXT files. These supporting files do contain information on a study-level and not on a subject-level.

• TXT: These files contain subject-level phenotypic datasets.

NOTE: The unharmonized clinical data sets contains all data from the dbGaP study, but it is not cross-compatible across all studies within BioData Catalyst.

Exporting/Downloading Data from the Files Tab

Once the user has selected a cohort, there are five options for accessing the files:

• Download Manifest: Download the file manifest and use this manifest to download the enlisted data files using the gen3-client.

• Export to Workspace: The files can be exported to a Gen3 workspace.

• Export All PFB: Initiate a PFB export of the selected files.

• Export All to Terra: Initiate a PFB export of the selected files to BioData Catalyst powered by Terra.

• Export All to Seven Bridges: Initiate a PFB export of the selected files to BioData Catalyst powered by Seven Bridges.

• GUID Download File Page: Aside from the 5 button options, users can download files by first clicking on the link(s) under the GUIDs column, followed by the Download button in the file information pages (see next section below).

File Information Page

A user can visit the File Information Page after clicking on any of the available GUID link(s) in the Files tab page. The page will display details such as data format, size, object_id, the last time it was updated and the md5sum. The page also contains a button to download the file via the browser (see below). For files that are 5GB or more, we suggest using the gen3-client.

Free text search for Submitter IDs and File Names

Both the Data and File tabs contain a text-based search function that will initiate a list of suggestions below the search bar while typing.

In the Data tab, Submitter IDs can be searched under the Subject tab.

In the File tab, File Names can be searched under the File tab.

Click either on a single or on multiple suggestions in the list appearing underneath the search bar to create a cohort and export/download the data. The selections can be again clicked to be removed from the created cohort.

Current Project IDs

A list of current project IDs can be found in the Data tab, under Filters>Project>Project Id. The current project IDs are:

• Parent

• TOPMed

• Open_Access

• Tutorial

Parent and TOPMed Studies

Distinguishing Between Parent and TOPMed Studies

The Parent and TOPMed study types have been categorized on Gen3 by their Program designation. An example of this designation by Program is presented below.

The Program types can be further identified by whether there is an underscore (_) at the end of the study:

• Parent studies will include an underscore at the end of the study name.

  • Example: parent-WHI_HMB-IRB_

• TOPMed studies will not include an underscore at the end of the study name.

  • Example: topmed-BioMe_HMB-NPU

Relationship Between Parent and TOPMed Studies

There are three distinct relationships possible between Parent and TOPMed studies. The first two relationships are streamlined:

• Parent only: The Parent study does not have a TOPMed counterpart study. This usually means that there are no genomic data, such as WXS (whole exome sequencing) or WGS (whole genome sequencing), located within the study; only phenotypic data.

• TOPMed only: This TOPMed study does not have a Parent counterpart study. These studies will contain both genomic data, WXS or WGS, and phenotypic data.

• Parent study with a counterpart TOPMed study: The Parent study will contain the phenotypic data, while the TOPMEd study will contain the genomic data. Under dbGaP, these studies would be kept separate from one another and the user would need to create the linkages. In the Gen3 platform, these studies have been linked together under the Parent study, based on the participant IDs found in dbGaP. This allows our system to produce valuable information and cohort creation as it combines both phenotypic and genomic data.

Parent and TOPMed Study Contents

The most notable difference between the Program categories is the type of hosted data.

Parent

• Genomic data: None

• Phenotypic data: Like with TOPMed studies, any phenotypic data found within the Graph Model, will only be DCC harmonized variables. For the raw phenotypic data from dbGaP, again, it can be found in the reference_file node.

TOPMed

• Genomic data: Available data can include CRAM, VCFs and Cohort-level VCF files

• Phenotypic data: TOPMed studies without an associated Parent study will include phenotypic data in the data graph by way of DCC harmonized variables. Additionally, raw phenotypic data from dbGaP can be found in the reference_file as tar files that share this common naming scheme: RootStudyConsentSet_phs######.<study_shorthand>.v#.p#.c#.<consent_codes>.tar.gz

Open_Access - 1000 Genomes project

The 1000 Genomes Project is an international research effort (2008-2015) to establish the most detailed catalogue of human variation and genotype data. On the Gen3 platform, the Program open_access contains:

• Genotypic data: Available data can include CRAM and VCF files.

• Phenotypic data: The data graph will contain phenotypic data by way of DCC harmonized variables. Additionally, raw phenotypic data can be found in the reference_file as VCF and TXT files.

Tutorial

On the Gen3 platform, the Program tutorial contains:

• Genotypic data: Available data can include CRAM and VCF files.

• Phenotypic data: The data graph will contain phenotypic data by way of DCC harmonized variables. Additionally, raw phenotypic data can be found in the reference_file as VCF and GDS files.

When navigating to a Workspace, users are presented with multiple workspace options.

The Gen3 platform offers two workspace environments: Jupyter Notebooks and R Studio.

There are six workspaces:

Virtual machines (VM):

• Small Jupyter Notebook VM

• Large Jupyter Notebook Power VM

• R Studio VM

Pre-made workflow workspaces:

• Autoencoder Demo

• CIP Demo

• Tensorflow-Pytorch.

To start a workspace, select Launch. You will see the following launch loading screen.

Once the VM is ready, the initial screen for the workspace will appear. For scripts and output that need to be saved when the workspace is terminated, store those files in the pd/ directory.

This workspace will persist once the user has logged out of the Gen3 BioData Catalyst system. If the workspace is no longer being used, terminate the workspace by selecting Terminate Workspace at the bottom of the window. You will be returned to the Workspace page with all of the workspace options.

Overview

The Dictionary page contains an interactive visual representation of the Gen3 data model. The default graph model view, as pictured below, displays all of the nodes and relationships between nodes in a hierarchical structure. The model further specifies the node types and links between nodes, as highlighted in the legend located at the top right side of the page.

Graph View

Users can click on any of the graph nodes in order to learn more about their respective properties. By clicking on a node, the graph will highlight that specific node and all associated links that connect it to the Program node. A "Data Model Structure" list will also appear on the left side toolbar. This will display the node path required to reach the selected node from the Program node.

When a second node in the path is selected, it will then gray out the other possible paths and only highlight the selected path. It will also change the "Data Model Structure" list on the left side toolbar.

The left side toolbar has two options available:

• Open properties: Will download the submission files for all the nodes in the "Data Model Structure" list. This option can also be found on the node that was first selected.

• Download templates: Will open the node properties in a new pop-up window; an example is displayed in the following screenshot.

This property view will display all properties in the node and information about each property:

• Property: Name of the property.

• Type: The type of input for the node. Examples of this are string, integer, Boolean and enumerated values (enum), which are displayed as preset strings.

• Required: This field will display whether the property is required for the submission of the node into the data model.

• Description: This field will display further information about the property.

• Term: This field can be populated with external resources that have further information about the property.

Table View

The Table view is similar to the Properties view, and nodes are displayed as a list of entries grouped by their node category.

Clicking on one of the nodes will open the Properties view of the node.

Dictionary Search

The Dictionary contains a text-based search function that will search through the names of the properties and the descriptions. While typing, a list of suggestions appears below the search bar. Click on a suggestion to search for it.

When the search function is used, it will default to the graph model and highlight nodes that contain the search term. Frames around the node boxes indicate whether the searched word was identified in the name of the node (full line) or in the node's description and properties' names/descriptions (dashed line).

Clicking on one of these nodes, it will only display the properties that have this keyword present in either the property name or the description.

Click Clear Search Result to clear the free text search if needed.

The search history is saved below the search bar in the "Last Search" list. Click on an item here to display the results again.

What is a Portable Format for Bioinformatics?

A Portable Format for Bioinformatics (PFB) allows users to transfer both the metadata from the the Data Dictionary as well as the Data Dictionary itself. As a result, data can be transferred while keeping the structure from the original source. Specifically, a PFB consists of three parts:

• A schema

• Metadata

• Data

For more information and an in-depth review that includes Python tools for PFB creation and exploration, refer to the PyPFB github page and install the newest version.

Schema

A schema is a JSON formatted Data Dictionary containing information about the properties, such as value types, descriptions, and so on.

To view the PFB schema, use the following command:

pfb show -i PFB_file.avro schema

Example Output

...
  {
    "type": "record",
    "name": "gene_expression",
    "fields": [
      {
        "default": null,
        "name": "data_category",
        "type": [
          "null",
          {
            "type": "enum",
            "name": "gene_expression_data_category",
            "symbols": [
              "Transcriptome Profiling"
            ]
          }
        ]
      },
      {
        "default": null,
        "name": "data_type",
        "type": [
          "null",
          {
            "type": "enum",
            "name": "gene_expression_data_type",
            "symbols": [
              "Gene Expression Quantification"
            ]
          }
        ]
      },
      {
        "default": null,
        "name": "data_format",
        "type": [
          "null",
          {
            "type": "enum",
            "name": "gene_expression_data_format",
            "symbols": [
              "TXT",
              "TSV",
              "CSV",
              "GCT"
            ]
          }
        ]
      },
      {
        "default": null,
        "name": "experimental_strategy",
        "type": [
          "null",
          {
            "type": "enum",
            "name": "gene_expression_experimental_strategy",
            "symbols": [
              "RNA-Seq",
              "Total RNA-Seq"
            ]
          }
        ]
      },
      {
        "default": null,
        "name": "file_name",
        "type": [
          "null",
          "string"
        ]
      },
      {
        "default": null,
        "name": "file_size",
        "type": [
          "null",
          "long"
        ]
      },
      {
        "default": null,
        "name": "md5sum",
        "type": [
          "null",
          "string"
        ]
      },
      {
        "default": null,
        "doc": "The GUID of the object in the index service.",
        "name": "object_id",
        "type": [
          "null",
          "string"
        ]
      }
...

NOTE: To make the outputs more human-readable, the above information was then piped through the program jq. Example: pfb show -i PFB_file.avro schema | jq

Metadata

The metadata in a PFB contains all of the information explaining the linkage between nodes and external references for each of the properties.

To view the PFB metadata, use the following command:

pfb show -i PFB_file.avro metadata

Example Output

...
    {
      "name": "exposure",
      "ontology_reference": "",
      "values": {},
      "links": [
        {
          "multiplicity": "MANY_TO_ONE",
          "dst": "subject",
          "name": "subjects"
        }
      ],
      "properties": [
        {
          "name": "years_smoked",
          "ontology_reference": "Person Smoking Duration Year Count",
          "values": {
            "source": "caDSR",
            "cde_id": "3137957",
            "cde_version": "1.0",
            "term_url": "https://cdebrowser.nci.nih.gov/CDEBrowser/search?elementDetails=9&FirstTimer=0&PageId=ElementDetailsGroup&publicId=3137957&version=1.0"
          }
        },
        {
          "name": "years_smoked_gt89",
          "ontology_reference": "Person Smoking Duration Year Count",
          "values": {
            "source": "caDSR",
            "cde_id": "3137957",
            "cde_version": "1.0",
            "term_url": "https://cdebrowser.nci.nih.gov/CDEBrowser/search?elementDetails=9&FirstTimer=0&PageId=ElementDetailsGroup&publicId=3137957&version=1.0"
          }
        },
        {
          "name": "alcohol_history",
          "ontology_reference": "Alcohol Lifetime History Indicator",
          "values": {
            "source": "caDSR",
            "cde_id": "2201918",
            "cde_version": "1.0",
            "term_url": "https://cdebrowser.nci.nih.gov/CDEBrowser/search?elementDetails=9&FirstTimer=0&PageId=ElementDetailsGroup&publicId=2201918&version=1.0"
          }
        },
        {
          "name": "alcohol_intensity",
          "ontology_reference": "Person Self-Report Alcoholic Beverage Exposure Category",
          "values": {
            "source": "caDSR",
            "cde_id": "3457767",
            "cde_version": "1.0",
            "term_url": "https://cdebrowser.nci.nih.gov/CDEBrowser/search?elementDetails=9&FirstTimer=0&PageId=ElementDetailsGroup&publicId=3457767&version=1.0"
          }
        },
...

Data

The data in the PFB are the values for the properties in the format of the Data Dictionary.

To view the data within the PFB, use the following command:

pfb show -i PFB_file.avro

To view at a certain number of entries in the PFB file, use the flag -n to designate a number. For example, to view the first 10 data entries within the PFB, use the following command:

pfb show -i PFB_file.avro -n 10

Example Output

...
{
  "id": "6c5e21d5-da76-49a5-9f82-7e3a726d44c6",
  "name": "lab_result",
  "object": {
    "cer451q1": null,
    "oxldl1": null,
    "f81c": null,
    "renins1c": null,
    "cystatc1": null,
    "triglycerides": -0.40415245294570923,
    "glucos1c": 6.5463337898254395,
    "glucos1u": null,
    "ldl": 2.0789523124694824,
    "hdl": 2.7123606204986572,
    "creatin1": null,
    "total_cholesterol": 3.039848566055298,
    "chlcat1c": null,
    
...

    "uabcat1c": null,
    "inslnr1t": 1.8090298175811768,
    "vldlp31c": null,
    
...

    "unit_hematocrit_vfr_bld": null,
    "age_at_total_cholesterol": 80,
    "unit_total_cholesterol": null,
    "age_at_triglycerides": 80,
    "unit_triglycerides": null,
    "age_at_hdl": 80,
    "unit_hdl": null,
    "age_at_ldl": 80,
    "unit_ldl": null,
    
...

    "unit_mcv_entvol_rbc": null,
    "submitter_id": "HG00325_lab_res",
    "state": "validated",
    "project_id": "tutorial-synthetic_data_set_1",
    "created_datetime": "2020-01-27T13:54:06.745386+00:00",
    "updated_datetime": "2020-01-27T13:54:06.745386+00:00"
  },
  "relations": [
    {
      "dst_id": "f4fdda57-80f4-4995-bea2-161c3242c525",
      "dst_name": "subject"
    }
  ]
}

About Seven Bridges

BioData Catalyst Powered by Seven Bridges offers researchers collaborative workspaces for analyzing genomics data at scale. Researchers can find and analyze the hosted TOPMed studies by using hundreds of optimized analysis tools and workflows (pipelines); creating their own workflows; or interactive analysis. On the platform, researchers can utilize collaborative workspaces for analyzing genomics data at scale, and access hosted datasets along with Common Workflow Language (CWL) and GENESIS R package pipelines for analysis. This platform also enables users to bring their own data for analysis and work in RStudio and Jupyterlab Notebooks for interactive analysis.

Key Features

• Private, secure, workspaces (projects) for running analyses at scale

• Collaboration features with the ability to set granular permissions on project members

• Direct access to BioData Catalyst without needing to set up a Google or AWS billing account

• Access hosted TOPMed studies all in one place and analyze data on the cloud at scale

• Tools and features for performing multiple-variant and single-variant association studies including:

  • Annotation Explorer for variant aggregations

  • Cloud-optimized Genesis R package workflows in Common Workflow Language

• Cohort creation by searching phenotype data

  • Use PIC-SURE API for searching phenotype data

  • Search by known dbGaP identifiers

• Rstudio and Jupyterlab Notebooks built directly into the platform for easy interactive analysis and manipulation of phenotype data

• Hosted TOPMed data you can combine with your own data on AWS or Google Cloud

• Billing and administrative controls to help your research funding go further: avoid forgotten instances, abort infinite loops, get usage breakdowns by project.

Introduction

This tutorial guides users through the process of transferring files between the two workspace environments of NHLBI BioData Catalyst: NHLBI BioData Catalyst Powered by Seven Bridges and NHLBI BioData Catalyst Powered by Terra.

Most researchers select one of the workspaces as their primary analysis environment and their labmates and collaborators typically work with them on the same workspace environment. However, there are cases where some collaborators work on Seven Bridges and others work on Terra. In this case, researchers need to share data files between the two workspaces to facilitate collaboration. When researchers run analyses on Seven Bridges, the results, or derived data, is only available on Seven Bridges. Likewise, when researchers run analyses on Terra, the results are only available on Terra. This tutorial provides step-by-step guidance on how to share derived data between the workspace environments. These instructions can also be used to share private data that has been uploaded to Seven Bridges or Terra.

Both open access data and controlled access data can be shared across workspace environments. Importantly, if a researcher intends to share controlled access data, they must ensure that all recipients have the necessary dbGaP permissions for those files. In some cases, this may mean the researchers must be listed as collaborators on their respective dbGaP applications. These instructions are intended for sharing files under 1 terabyte (TB) in size. If you want to share data larger than 1 TB, contact the BioData Catalyst Help Desk to discuss your use case.

Initial Considerations

Platform Accounts

The first consideration is platform accounts. Moving data between Seven Bridges and Terra is currently a manual process and requires that one of the researchers involved in sharing has an account on both platforms. It is recommended that the recipient of the shared data is the person to have accounts on both Seven Bridges and Terra.

Let’s consider an example case: Sebastian who is working on Seven Bridges and Teresa who is working on Terra. If Sebastian wants to share data with Teresa so that she can use the data on Terra, Teresa first needs to set up an account on Seven Bridges. Now Teresa has an account on Terra and an account on Seven Bridges. Sebastian will share the data with Teresa on Seven Bridges by adding her as a member of the project with the data he wants to share, with Copy permissions. For information on permissions, refer to the Seven Bridges Set permissions documentation. Once Teresa is added as a member of the project, she can move the data from the Seven Bridges project to a workspace on the Terra platform, following the instructions in the section titled Moving Data From Seven Bridges to Terra.

If Teresa (Terra) wants to share data with Sebastian (Seven Bridges) so that he can use the data on Seven Bridges, Sebastian first needs to create an account on Terra. Now Sebastian has an account on Seven Bridges and an account on Terra. Teresa can share the data with Sebastian on Terra by sharing the workspace with the data she wants to share with Sebastian. For information on sharing workspaces, refer to the Terra How to share a workspace documentation.

To create a Terra account, refer to the Terra documentation.

To create a Seven Bridges account, refer to the Seven Bridges documentation. If you are new to Seven Bridges, you may find this Getting Started Guide helpful.

Billing

The second consideration is making sure the researcher moving data between the two workspaces has billing groups set up on both workspaces to cover cloud costs if necessary. Contact the BioData Catalyst Help Desk if you have questions about how to get a billing group on Seven Bridges or Terra.

Moving Files From Terra to Seven Bridges

The following steps describe how to use the Seven Bridges platform to pull data securely from a Terra workspace into a Seven Bridges project.

Refer to the Terra documentation for Moving data to/from a Google bucket (workspace or external), specifically the section Upload and download data files in a terminal using gsutil. This method:

• Works well for all size transfers.

• Ideal for large file sizes or 1000s of files.

• Can be used for transfers between local storage and a bucket, workspace VM or persistent disk and a Google bucket, as well as between Google buckets (external and workspace).

You will use the terminal in JupyterLab on the Seven Bridges workspace environment. The reason for this is that although Seven Bridges can run on the Google Cloud Platform, the Google bucket API is not exposed in the same manner as it is on Terra. Therefore you will start a JupyterLab notebook on Seven Bridges, using the project you would like to be the destination for the copied data. Refer to the Seven Bridges documentation for launching Jupyter Lab notebooks on Seven Bridges and accessing the terminal in a JupyterLab environment.

After launching the notebook, the next step is to open the terminal and install the program gsutil which is a python program that lets end users add data to or copy data from a Google Cloud bucket. After opening the terminal, run the following commands:

pip install gsutil
gsutil config

Installing gsutil takes only a few seconds.

The config command provides a secure URL for you to navigate to in the browser. You will authenticate with the same credentials that were used to login to Terra. The shortcut to access the printed URL in the JupyterLab terminal is to press shift and right click, which will display options to copy the URL. Copy and then navigate to the URL in a new browser tab, which will direct you to Google authentication:

Google will provide an authentication code that you will copy and paste into the terminal.

Next, you will type in the Google Project id. This is found on the right side of the Terra Workspace Dashboard.

Next, run the command below to display the different Google buckets that are attached to the project id.

gsutil ls

The Google bucket name for the Terra project can be found in the lower right corner of the Terra Workspace.

Running gsutil ls on the Google bucket name will display the folders and files from the Terra workspace.

To copy a folder to the Seven Bridges workspace environment, run the following command:

gsutil cp -R gs://[Google-Bucket-Name] /sbgenomics/output-files/

There are a couple important things to mention about the gsutil cp command. First, the -R flag for gsutil cp is used to recursively copy a folder and all of its subfolders and files. Most users will likely want to use the -R flag. This flag should be omitted if copying individual files or if using a wild card such as “*.vcf”.

Additionally, /sbgenomics/output-files should be the destination folder when bringing in data from Terra, as this will ensure the files or folders get populated back to the Seven Bridges project. Refer to the Save analysis outputs documentation for information about working with files in Data Cruncher environments. After the JupyterLab instance is shut down, your files will automatically be populated in your project-files tab on Seven Bridges.

Moving Data From Seven Bridges to Terra

In this section we will discuss pushing data from a Seven Bridges project to a Terra workspace.

The process of moving data from Seven Bridges to Terra is the same setup as the previous section with some modifications to the gsutil copy command. Instead, we reverse the arguments.

gsutil cp -R /sbgenomics/output-files/vcfs_to_transfer \ gs://[Google-Bucket-Name]

You will still use the -R flag but the destination is a Terra bucket. The Terra workspace’s Google bucket name/id can be found on the Terra workspace Dashboard tab. You can verify that the folder has been copied by navigating to the Files section of the Data tab in your Terra workspace.

Clicking on the folder, you will see that all three files have been copied.

One of the key steps to becoming an advanced user and being able to fully understand and leverage the power of BioData Catalyst Powered by Seven Bridges is to learn how to detect and correct errors that prevent the successful execution of your analyses. The Troubleshooting tutorial presents some of the most common errors in task execution on the platform and shows you how to debug and resolve them. There is also a corresponding public project on the platform called "Troubleshooting Failed Tasks" which has examples of the failed analyses presented in the written tutorial.

• Find the written tutorial here.

• Find the platform public project with examples here.

Just starting out on NHLBI BioData Catalyst Powered by Seven Bridges, and need to get up to speed on how to use the platform? Our experts have created a Getting Started Guide to help you jump right in. We recommend users begin learning how to use BioData Catalyst Powered by Seven Bridges by following the steps in this guide. After reading this guide, you will know how to create an account on NHLBI BioData Catalyst Powered by Seven Bridges, learn the basics of creating a workspace (project), run an analysis, and search through the hosted data.

To read our Getting Started Guide, please refer to our documentation page .

is a collection of user documentation which describes all of the various components of the platform, with step-by-step guides on their use. Our Knowledge Center is the central location where you can learn how to store, analyze, and jointly interpret your bioinformatic data using BioData Catalyst Powered by Seven Bridges.

From the Knowledge Center, you can access platform documentation. This content is organized into sections that deal with the important aspects of accessing and using BioData Catalyst Powered by Seven Bridges.

You can also read the Release Notes in the Knowledge Center, keeping you up-to-date on all of the latest updates and new features for BioData Catalyst Powered by Seven Bridges.

This guide aims to help you learn how to take advantage of all the various features and functionality for performing analyses on Seven Bridges and ensure that you can set up your analyses in the most efficient way possible to save time and money.

The following topics are covered in this guide:

• The basics of working with CWL tools and workflows on the platform.

• How to specify computational resources on the platform and how to use the default options selected by the execution scheduler.

• How to run Batch analyses and take advantage of parallelization with scatter.

• The basics of working with Jupyterlab Notebooks and Rstudio for interactive analysis.

You can refer to the guide .

BioData Catalyst Powered by Terra is a user-friendly system for doing biomedical research in the cloud. Terra workspaces integrate data, analysis tools, and built-in security components to deliver smooth research flows from data to results.

The following entries in this section of the BioData Catalyst documentation are a starting point for learning how to use Terra in the context of the BioData Catalyst ecosystem. You can also dive deeper into Terra by visiting the Terra website and the Terra Support Center. Wherever possible, we highlight specific articles, tutorial videos, and example workspaces that will help you learn what you need to know to accelerate your research.

If you can't find what you are looking for, we are happy to help. See the Troubleshooting and Support section for more information.

Please note that Terra is designed for and tested with the Chrome browser.

Now that you can log in, you’ll want to make sure that you have access to a Billing Account and Billing Project. This will allow you to charge storage and analysis costs through a Google account linked to Terra. A Terra Billing Project is Terra's way of connecting a workspace where you accrue costs for things, back to a Google Billing account where you pay for it. You must have a Google Billing Account established before creating a Terra Billing Project. Outlined here are the steps necessary to set this up, as well as instructions on how to add or be added to an existing account/billing project.

Detailed instructions for setting up your billing can be found by following the links below. If you are a BioData Catalyst Fellow, your procedure for billing set up is a bit different, but you may find some of the information below still relevant (sharing a billing project with another user, for example). Step 1: Get Cloud credits for BioData Catalyst Step 2: Wait for approval & review the ​Billing overview for BioData Catalyst users Step 3: Credits approved. Now create a new Terra billing project Step 4 (optional): ​Sharing Billing Projects among colleagues

Logging in to Terra for the first time is a quick and straight-forward process. The process is easiest if you already have an email address hosted by Google. If you want to use an email address that is not hosted by Google, we have instructions for that as well. Article: ​How to register for a Terra account Article: ​Setting up a Google account with a non-Google email We also recommend our article on navigating in Terra to get familiar with basic menus and options in Terra, as well as this video introduction to Terra.

Read on in the next two subsections for primers on how to set up billing and how to manage costs.

Workspaces are the fundamental building blocks of Terra. You can think of them as modular digital laboratories that enable you to organize and access your data in a number of ways for analysis.

To learn about the basics of operating a Terra workspace, we recommend these resources: Article: Working with workspaces Video: Introduction to using workspaces in Terra

Read on in this section to get familiar with:

• Data storage and management

• Collaboration

• Security