> For the complete documentation index, see [llms.txt](https://bdcatalyst.gitbook.io/biodata-catalyst-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://bdcatalyst.gitbook.io/biodata-catalyst-documentation/data-management/data-submission-instructions.md).

# Instructions for Data Submission to BDC

## Introduction

The five steps and their subparts outlined below provide instructions for submitting data and making it available through BDC. These instructions will be updated as new information and processes are made available. Before the data submission can take place, the submitter will need an [eRA commons login](https://www.era.nih.gov/). Your institute’s office of sponsored research, or similar, will need to assist in setting up the login if you do not have one.

## Instructions for Data Submission to BDC

<details>

<summary><strong>Step 1: Initiate the Intent to Submit</strong></summary>

## Step 1: Initiate the Intent to Submit

This step has two data submitter action items, and the first is different for NHLBI intramural investigators than for extramural investigators.

:arrow\_right: **Data Submitter Action Item 1: for NHLBI Intramural Investigators**\
Email <NHLBIDIRBDCSubmission@mail.nih.gov> for submission information.

:arrow\_right: **Data Submitter Action Item 1: for Extramural Investigators**\
Use the following email template, complete it with information specific to your study, and send it to <bdcatalystdatasharing@nih.gov>.

**Email template:**

To: <bdcatalystdatasharing@nih.gov>\
Subject: BioData Catalyst Data Submission \[Grant Number / Award Number]

| Study Name                                        |   |
| ------------------------------------------------- | - |
| Institution Name                                  |   |
| PI Name                                           |   |
| Grant Number/Award Number/ZIA Number              |   |
| Expected date for data upload/submission          |   |
| Does this submission include genomics data?       |   |
| Does this submission include biospecimen samples? |   |
| Does this submission include imaging data?        |   |

:white\_check\_mark: **Result**

After sending the email, you will receive an automated response with the documents to use in Step 2.

Additionally, depending on what information is included you may receive additional instructions:

* If the study has biospecimen samples, you may be directed to the data submission at NHLBI’s BioLINCC repository.
* If the study has genetic data, or your cohort has genetic data in other sub-studies, you may be directed to the NHLBI Genomic Program Administrator (GPA).
* If the study or cohort doesn’t have any genetic data, you may be directed to the NHLBI non-Genetic GPA.

:arrow\_right: **Data Submitter Action Item 2: Complete the Institutional Certification (IC) and Data Submission Information (DSI) Sheet**

* For genetic data, complete the IC/DSI forms provided with the automated response and email them to <nhlbigeneticdata@nhlbi.nih.gov>.
* For non-genetic data, contact <NHLBI_nongeneticdata@mail.nih.gov>.

:white\_check\_mark: **Results**

* You will receive a response from the GPA confirming receipt of your email.
* Extramural submitters will also receive a response from the BDC Data Management Core (DMC) (<nhlbi.dmc.concierge@rti.org>) to provide further assistance or answer data submission-related questions.

</details>

<details>

<summary><strong>Step 2: Study Registration in dbGaP</strong></summary>

### Step 2: Study Registration in dbGaP

All research data submitted to BDC must be registered through the NIH database of Genotypes and Phenotypes (dbGaP). The DMC will contact you and provide specific guidance. Study registration has two parts but only one action for data submitters. Data submitters will need eRA commons credentials to log in to complete the actions.

:blue\_circle: The GPA will register the study in the dbGaP Submission System by entering information provided from your Institutional Certification and Data Submission Information Sheet, resulting in the generation of the study accession number (phs number). If needed, the GPA may contact you for additional information or clarification or asking for a data sharing plan and data use agreement.

:white\_check\_mark: **Results**

* You will receive an automated email from dbGaP to review and complete Study Submission; see example screenshot of the dbGaP email below.

<figure><img src="https://4002529329-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LwOmaDlbanAQ-7fhd89%2Fuploads%2F4dVPdWZbiN8kjk15REN1%2FPicture1.png?alt=media&amp;token=b9e74a16-30db-40b9-8a14-cbb48c747561" alt=""><figcaption></figcaption></figure>

:arrow\_right: **Data Submitter Action Item 1:** After receiving the automated email from dbGaP, review and complete the dbGaP submission process using guidance available in the dbGaP Study Configuration Process for Submission of Data to BDC; see screenshot of the dbGaP Study Submission portal below. Study Config consists of a web form that collects a description of the study data, methods, and findings, inclusion/exclusion, study history, references, attributions, and terms that will be indexed to enable users to search for your study in dbGaP Advanced Search.

{% hint style="info" %}
**Note**: Gather all information ahead of the web form entry, as the current form does not have a “save” button for partial entry.
{% endhint %}

Click below to download the example files needed for dbGaP submission.

{% file src="/files/vyNcM1CUqJ0nikvSzg12" %}

<figure><img src="https://4002529329-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LwOmaDlbanAQ-7fhd89%2Fuploads%2Fa79az2MhfQBHX5k5QEFo%2Fimage.png?alt=media&amp;token=aa7b2cbe-6a47-48ce-8251-7514499bd3ec" alt=""><figcaption></figcaption></figure>

:white\_check\_mark: **Results**

* Once you finish your study configuration, dbGaP will curate your submission and may contact you for questions. Once dbGaP completes its curation process, you will receive an email from dbGaP to approve and complete your study registration.
* Note: While waiting for dbGaP curation, proceed with the steps to prepare and submit data to BDC (steps 3 and 4 below) to reduce the time to ingest and release the data.

</details>

<details>

<summary><strong>Step 3: Data Preparation</strong></summary>

### Step 3: Data Preparation

Data preparation can happen before, during, or after the study registration process at dbGaP and must be completed to submit data to BDC. This step has one action item for all data submitters and a second action item for submitters of omics and phenotypic data types.

:arrow\_right: **Data Submitter Action Item 1:** Organize submission into two subfolders: `/data` and `/documentation`.

The `/data` directory contains the files that contain the scientific data generated for the study. Hosted data at BDC is organized by consent type and are organized into separate cloud storage buckets. Organizing the data files that correspond to participants with similar consent groups (as described in the dbGaP 2b file) now will save you time when it comes time to upload the data.

The `/documentation` directory contains the prepared supplemental documentation to accompany the data submission (“data package”) according to the [Instructions for Preparing Clinical Research Study Datasets for Submission to the NHLBI](https://www.nhlbi.nih.gov/grants-and-training/policies-and-guidelines/guidelines-for-preparing-clinical-study-data-sets-for-submission-to-the-nhlbi-data-repository), including:

* Protocols
* Survey Instruments\*
* BDC compliant Data Dictionaries
  * **Clinical Data** - Find the dbGaP Data Dictionary description [here](https://www.ncbi.nlm.nih.gov/gap/docs/submissionguide/#8-what-is-a-phenotype-data-dicti) and download a sample Data Dictionary file below
    * as separate files for each of the data files (not as a tab within the data file)\* -  download and reference the Data Dictionary Requirement PDF below
  * **Imaging Data** - Find dbGaP Data Dictionary description [here](https://www.ncbi.nlm.nih.gov/gap/docs/submissionguide/#15-how-do-i-submit-medical-image) and a sample Data Dictionary file below

{% file src="/files/y0DmspXr3MkBELSQz9Ur" %}

{% file src="/files/2hQXtNBqybyAXUssAfbV" %}

{% file src="/files/7EOmGCumVVmsoB1spCcf" %}

* File/folder naming
  * Use lowercase file extensions (.csv instead of .CSV).
  * Presence of any spaces and special characters (\~, (, ), &, $, @, =, ;, :, +, ?) will result in an error during data uploading. Replace spaces with underscores (\_) or hyphens (-).
* Subject consent files (2a/2b in dbGaP submission above) for QC
* Datasets Readme\*
  * Specify data file name and variable name for “subject ID” and “age” (note if age variable has been winsorized or removed).
  * Datasets organization - if the datasets are organized in multiple sub-folders, need a Readme file to describe the relationship of the sub-folders, if they are independent (e.g., multiple phases or visits), main studies with ancillary studies, or overlapping (e.g., /raw data and /harmonized data, where the /harmonized data is a subset of the /raw data). Download an example PDF "Sample-CT\_Readme.pdf" below for an image study.
  * Compressed and archive file formats - e.g. ZIP, GZ, TAR. No nested archives. The Readme file may describe the directory structure of the compressed/archive files within the bucket, and/or the directory structure within the compressed/archive files. Describe the de-identification methods, see example file [here](https://bdcatalyst.gitbook.io/biodata-catalyst-documentation/data-management/data-submission-instructions/de-identification-readme) for clinical data, and [DICOM](https://drive.google.com/file/d/1oKFvlI_3NtmfbS8xCD9JDg1LOvimmhBs/view?usp=drive_link), [ECG](https://drive.google.com/file/d/1sHhhOx-aPPkfriBS8QXS0r2kUJstyQL2/view?usp=drive_link), [Echo](https://drive.google.com/file/d/1rHz7WgcBfwE-wvbjZCxFX6uf5IiaL73G/view?usp=drive_link)\*

{% file src="/files/8ROXxKcBZETUjDDYoMJQ" %}

* Data/Metadata model if applicable
* Additional Supplemental documentation to reproduce study results

*\* Supported documentation types for data dictionaries and models are .csv, tab-delimited, xml, json, and other machine-readable formats. PDF and SAS file formats are not machine-readable and are discouraged from submission unless submitted with machine-readable versions as well. File names* *should not include any spaces or* *special characters.* Find more information at NIH\_NHLBI\_BDC\_Data\_Upload\_Guide PDF below.

:arrow\_right: **Data Submitter Action Item 2**: Only for Omics and Phenotypic data types, prepare the data files per the [dbGaP Study Submission Guidance](https://www.ncbi.nlm.nih.gov/gap/docs/submissionguide/#aconfig).

</details>

<details>

<summary><strong>Step 4: Data Submission to BDC</strong></summary>

### **Step 4: Data Submission to BDC**

Data submission has two action items for data submitters. This process can happen in parallel with Data Submitter Action Item 1 from Step 2. The data submission process begins by filling out the BDC contact form: <https://biodatacatalyst.nhlbi.nih.gov/help/contact-bdc>.

:arrow\_right: **Data Submitter Action Item 1: Request bucket creation:**

* Send request to the DMC staff by providing:

  1. dbGaP PHS accession number (phs#) received from step 2, and
  2. study consent group information (dbGaP submission 2a/2b files prepared in step 2).

  DMC staff will use the information to create Amazon Web Services (AWS) cloud storage bucket(s) for your study.

:arrow\_right: **Data Submitter Action Item 2: Access the cloud bucket created for your study.** You will receive a secure email from the Information Technology Applications Center (ITAC) team at NHLBI that provides the URL to activate access with user ID and password (see screenshot below):

<figure><img src="https://4002529329-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LwOmaDlbanAQ-7fhd89%2Fuploads%2FSF9WI8sIcnRheXK7W5Lu%2FPicture3.png?alt=media&amp;token=857fd0be-5506-4936-b182-df4d59056b4d" alt=""><figcaption></figcaption></figure>

Follow the links and instructions in the email to activate the Amazon Web Service (AWS) S3 web interface using the link provided in the email.

<figure><img src="https://4002529329-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LwOmaDlbanAQ-7fhd89%2Fuploads%2FrfsuUebafGxgQMGqaBsK%2FPicture4.png?alt=media&amp;token=9e57774f-e356-46c0-910a-922d55027eaa" alt=""><figcaption></figcaption></figure>

Find more information in the Access the BDC AWS cloud bucket (PDF) below. If you have any questions or issues about accessing the buckets, contact <nhlbi.dmc.concierge@rti.org>.

{% file src="/files/kaF9h2Af2eSmTCgyo82e" %}

:arrow\_right: **Data Submitter Action Item 3: Upload datasets to the cloud bucket created for your study.** After access, *“search “S3” in the search box and click the S3 to find the study buckets,* upload datasets for each consent group to the corresponding buckets (e.g., xxxx-c1) as described in the dbGaP 2b file.

<figure><img src="https://4002529329-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LwOmaDlbanAQ-7fhd89%2Fuploads%2FWO6H6R5QWP684EMiDLxt%2FPicture5.png?alt=media&amp;token=3e6d55dd-9e99-4416-9194-432a22217c48" alt=""><figcaption></figcaption></figure>

Once selected the specific bucket for a consent group, use the “Upload” button to upload data files. Please upload data files to the `/data` subfolder (NOT the root level), and the documentation files to the `/documentation` folder.

<figure><img src="https://4002529329-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LwOmaDlbanAQ-7fhd89%2Fuploads%2FUHFQXR3jBV8qBfetx6ly%2FPicture6.png?alt=media&amp;token=e5c9182c-6ba7-430b-ad42-e793dc871da9" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary><strong>Step 5: Next Steps</strong></summary>

### Step 5: Next Steps

Once your data package is uploaded successfully, the data go through quality checks before ingestion and release. If issues are found, the DMC will contact you and assist in resolving the issues before ingestion and release. There are three data submitter action items associated with this step.

:arrow\_right: **Data Submitter Action Item 1: If the DMC contacts you about QC issues with the uploaded data, respond to their inquiries to resolve the issues.**

:arrow\_right: **Data Submitter Action Item 2: If requested by the DMC, resubmit the data package after all issues are resolved.**

After data clears the data quality checks, the ingestion and release process can take as few as 4-6 weeks. After the data is released, the DMC will notify you that your study is available for use by authorized individuals in BDC ([study inventory](https://biodatacatalyst.nhlbi.nih.gov/resources/data/studies/)).

:arrow\_right: **Data Submitter Action Item 3: You are encouraged to log in and view your study data in BDC.**

</details>

## Need Assistance?

Contact the BDC Data Management Core (DMC) via <https://biodatacatalyst.nhlbi.nih.gov/help/contact-bdc> and select **Data Submission** in the "Select type of contact" field.