Overview
The synapser package provides an interface to Synapse, a collaborative workspace
for reproducible data intensive research projects, providing support
for:
- integrated presentation of data, code and text
- fine grained access control
- provenance tracking
The synapser package lets you communicate with the
Synapse platform to create collaborative data analysis projects and
access data using the R programming language. Other Synapse clients
exist for Python, Java,
and the web browser.
If you’re just getting started with synapser, have a
look at the Getting Started guides: Installation, Authentication, and Configuration.
Good example projects are:
- TCGA Pan-cancer (syn300013)
- Development of a Prognostic Model for Breast Cancer Survival in an Open Challenge Environment (syn1721874)
- Demo projects (syn1899339)
This page gives a short tour of what you can do with
synapser. Each section links to a tutorial that covers the
topic in depth.
Installation
Install synapser from R-universe:
install.packages(
"synapser",
repos = c("https://sage-bionetworks.r-universe.dev", "https://cloud.r-project.org")
)Alternatively, edit your ~/.Rprofile and configure your
default repositories:
after which you may run install.packages without
specifying the repositories:
install.packages("synapser")synapser sets up the Synapse Python client it needs the
first time it is loaded, so you don’t need to install Python yourself.
On Linux, complete the extra setup steps in the Installation vignette first. That vignette
also covers installing from GitHub and installing synapser 2.x and
earlier, which remain on the Synapse R repository (RAN). For
troubleshooting, see the Troubleshooting guide.
If you’re upgrading from synapser 2.x, read Migrating to synapser 3.0 first.
Connecting to Synapse
To use Synapse, you’ll need to register for an account. The Synapse website can authenticate using a Google account. If you authenticate using a Google account, you’ll need to create a personal access token to log in to Synapse through the programmatic clients. See the Authentication vignette for more information.
Once that’s done, you’ll be able to load the library and login:
For more ways to manage your Synapse credentials, please see the Authentication vignette, and the native reference documentation:
?synLogin
?synLogoutAccessing Data
Synapse identifiers start with syn, followed by numbers
(for example, syn123). Upload a local file with
synStore(), and download a file with
synGet():
file <- File(path = "/path/to/data.csv", parent_id = "syn123") |> synStore()
downloaded_file <- synGet(
file$id,
file_options = FileOptions(download_location = "/path/to/downloads/")
)
downloaded_file$pathBy default, files are downloaded to the Synapse cache. See Data Upload, Download, and Delete and Files for more, including versioning. To move many files at once, see Uploading data in bulk, Downloading data in bulk, and Downloading data by Synapse ID. The manifest file used for bulk uploads is described in Manifest CSV.
Organizing Data in a Project
Projects and folders are containers that organize your data:
project <- Project(name = "My uniquely named project") |> synStore()
folder <- Folder(name = "raw_data", parent_id = project$id) |> synStore()To list what a project or folder contains without downloading anything, sync its metadata:
project <- project |> synSyncFromSynapse(download_file = FALSE, recursive = FALSE)
for (child in project$folders) {
print(child$name)
}See Projects, Folders, and Structuring Your Synapse Project.
Annotating Synapse Entities
Annotations are key-value metadata that make your data searchable.
Set them with synSetProperties() and save them with
synStore():
file <- synGet(file$id, file_options = FileOptions(download_file = FALSE)) |>
synSetProperties(annotations = list(assay = "RNA-seq", fileType = "csv")) |>
synStore()
file$annotationsSee Annotations, Add, Update and Remove Annotations, and Properties and annotations.
Provenance
Synapse’s provenance tools record what was used, and what was
executed, to create a file. Attach an Activity to an entity
and store it:
file <- synGet(file$id, file_options = FileOptions(download_file = FALSE)) |>
synSetProperties(
activity = Activity(
name = "clustering",
description = "whizzy clustering",
used = list(UsedEntity(target_id = "syn1234")),
executed = list(UsedURL(name = "analysis code", url = "https://github.com/my-org/my-analysis"))
)
) |>
synStore()See Activity/Provenance
and the Activity reference documentation
(?Activity).
Tables
Synapse Tables store tabular data that can be queried using a SQL-like query language. See Tables and Virtual Tables.
Datasets
A Dataset is a curated, versionable collection of files that can be queried like a table. Group datasets into a Dataset Collection. See Datasets and Dataset Collection.
Wikis
Wiki pages let you describe your project and its data in Markdown:
wiki_page <- WikiPage(
owner_id = project$id,
title = "My Wiki Page",
markdown = "# My Wiki Page\n\nThis is a wiki page."
) |> synStore()See Wikis.
Evaluations
An Evaluation is a queue that accepts and scores submissions, for example in a computational challenge. Submit an entity to an evaluation queue:
submission <- Submission(
entity_id = file$id,
evaluation_id = "9999999",
name = "My Submission"
) |> synStore()See Evaluations and Submissions, SubmissionStatuses, SubmissionBundles.
Sharing Access to Content
By default, data you upload is private to you. Share it by setting permissions for a user, a team, or everyone, for example to give the public read access:
project |> synSetPermissions(principal_id = 273949L, access_type = list("READ"))File Views
A file view can be defined by its scope. It allows querying for files within the scope using a SQL-like syntax. Please visit the EntityView vignette for more information.
Accessing the API Directly
These methods enable access to the Synapse REST(ish) API taking care of details like endpoints and authentication. See the REST API documentation.
?synRestGet
?synRestPost
?synRestPut
?synRestDeleteSynapse Utilities
Utilities that used to live in the separate synapserutils
package are now available directly in synapser:
- Copy files, folders, and projects with
synCopy(). - Upload data to Synapse in bulk with
synSyncToSynapse()(see Uploading data in bulk). - Download data from Synapse in bulk with
synSyncFromSynapse()(see Downloading data in bulk).
More information
For more information see the Synapse User Guide.