The intent is that these may act as a guideline for those wishing to enhance the reusability of their data holdings.
Wilkinson and colleagues, The FAIR Guiding Principles for Scientific Data Management and Stewardship (2016)
By the end of this chapter you will be able to decide what belongs in a team’s slow-moving environment layer and what in its fast-moving package layer. You will also be able to state plainly why the committed Dockerfile rather than the pushed image is the reproducibility guarantee, and to deposit a compendium so that it outlives the laptop, the registry, and the forge it was built on.
NoteBefore you start
Reading time. About 60 minutes.
Prerequisites. Chapter 6 (Chapter 6) supplies the renv lockfile this chapter extends to a team. The rule for that extension, the union model, under which the project lockfile records every package any contributor has installed, is introduced in this chapter. Chapter 7 (Chapter 7) supplies the digest-pinned Docker image this chapter archives, whose guarantee it relocates from the image to the Dockerfile, and a first look at the two-layer team model this chapter develops. Chapter 8 (Chapter 8), which the core path marks optional, is the source of the single-file SIF image used in the deposit; the archiving section explains the one command the deposit needs, so that chapter is not required. Chapter 3 (Chapter 3) supplies the L0-L3 ladder every honest claim in this chapter is stated against. Chapter 4 (Chapter 4) supplies the commit rule this chapter’s restricted-data section makes stricter. No new statistical knowledge is assumed; reading a Dockerfile and a short shell session is.
By the end of this chapter you will be able to answer:
Can you explain the two-layer team model and decide which layer a given package decision belongs to under the union model?
Can you state why the committed Dockerfile, and not the pushed image, is the reproducibility guarantee, and choose a registry and a forge with vendor independence in view?
Can you deposit a research compendium for the long term, minting a persistent identifier and archiving the environment as a single file, and relate the steps to the FAIR principles?
Can you recover a decayed environment from the date an analysis was run, and state the sustainability limits of a young framework honestly rather than papering over them?
Can you adapt the sharing-and-archiving workflow to a package, a blog, and a manuscript, naming what varies and what does not?
Outline.
Where this sits on the stack: what archiving captures and what it only validates.
Orientation: a laboratory whose environment nobody can reconstruct two years later.
The analyst’s contribution: three judgments no tool automates.
Sharing with a team: the two-layer model.
Sharing with the world: archiving, and what changes when the data are restricted.
Stewardship over time: recovering a decayed environment.
Across the archetypes: package, blog, manuscript.
Worked example: one laboratory’s quarter of collaboration and one deposit.
The cohort, continued: checking this book’s own compendium against a deposit checklist, today.
Collaborating with an LLM.
NoteCheck your understanding: from earlier chapters
A laboratory is about to deposit a compendium whose README claims ‘L3 verified’. The repository is under git. Its Dockerfile begins FROM rocker/r-ver:4.4.1 and installs packages with install.packages() from the live repository, and there is no renv.lock. The CI workflow runs only R CMD check, and its badge is green. Which rung does the compendium actually reach, and which earlier remedy would raise it by each rung in turn?
NoteAnswer
The compendium in this warm-up, a Dockerfile with no lockfile, reaches only L0, locatable through git. The rungs are cumulative (Chapter 3), so without a lockfile it has not reached L1, and the Dockerfile cannot make it L2. Three steps raise it. A committed lockfile written by renv::snapshot() gives L1 (Chapter 6). Restoring that lockfile inside an image whose base is pinned by digest, with a dated package snapshot, gives L2 (Chapter 7); a tag such as 4.4.1 can be rebuilt to different bytes. L3 needs verification rather than more capture: a CI job that rebuilds the image and runs a baseline test within a stated tolerance (Chapter 12). A green R CMD check compares nothing with a recorded result.
13.1 Where this sits on the stack
This chapter fixes no new version of anything on the determinant stack, and neither of its halves, sharing with a team and archiving with the world, adds a rung. It still does two things to the stack, and they differ in kind.
Figure 13.1 marks both. Archiving pins the input data layer more tightly than any earlier chapter, which is the second sense of ‘captures’ in the Conventions. The compendium of Chapter 5 made the data locatable in a team’s repository, which a forge outage or a departed contributor can erase. A deposit moves them to a persistent identifier that continues to resolve after the project ends. The layer is marked as captured here for that reason, although no new version or byte is fixed. Stewardship, by contrast, pins nothing new about the packages or the operating system; the lockfile of Chapter 6 and the digest-pinned image of Chapter 7 already did that capturing. What stewardship does is confirm, using rang or an archived image, that those pins still resolve after they have aged. It validates layers already captured elsewhere.
plot_determinant_stack( captures =c('Input data'), validates =c('Package versions','Operating system, system libraries'), title ='Where archiving and stewardship sit on the stack')
Figure 13.1: The determinant stack of Chapter 3, with input data marked as captured here and the package-version and operating-system layers marked as validated. This chapter pins no new byte: the packages are captured in Chapter 6 and the operating system in Chapter 7. What it adds to the data layer is durability, lifting it from merely locatable on a forge to findable under a persistent identifier, which is why input data is captured rather than validated here; what it adds to the package and system layers is the stewardship check, using an archived image or a date-based reconstruction, that those earlier pins still resolve, which is why they are validated rather than captured again.
13.2 Orientation
The last question a reproducible analysis must answer is a social and temporal one. Every earlier chapter was concerned with a single machine and a single analyst. We saw how to pin the packages (Chapter 6), fix the environment (Chapter 7), and render the report (Chapter 9). We then saw how to validate the whole with tests and continuous integration (Chapter 12). The present chapter concerns what happens when the analysis must leave that machine. It must first reach a collaborator down the hall, and eventually reach a stranger, a reviewer, or the analyst’s own future self, years after the original laptop has been recycled.
Consider a concrete case. A five-person laboratory runs a multi-site observational cohort. The principal investigator fixes the computational environment at the project’s start. Over the following year, one contributor adds survival-analysis packages, a second adds table-formatting packages for the manuscript, and a third adds spatial packages for a geographic sub-study. Each installs freely, on their own machine, without asking the others’ permission. Two years after the paper appears, a journal requests the analysis for a reproducibility audit. The original environment must then be reconstructed from whatever survives in the repository. Nothing in the single-analyst workflow of the earlier chapters tells us how the laboratory should have coordinated during that year. Nor does it tell us how the compendium should have been deposited so that the audit could succeed.
These are two distinct problems, and we shall keep them apart. The first is collaboration: sharing a compendium with a team while it is under active development, so that contributors do not drift into mutually incompatible environments. The second is archiving and stewardship: depositing the finished compendium so that it remains findable and executable long after the project ends, and recovering it when the pins have decayed. Both sit at the top of the reproducibility ladder and add no new rung. Sharing a compendium changes nothing about its level: an L2 compendium shared with a team is still an L2 compendium, and a verified L3 compendium deposited in an archive is still L3. What sharing and archiving add is the assurance that the level already reached survives contact with other people and with the passage of time. That assurance is the subject of this chapter. Being the last chapter, it is also where we draw the threads of the whole book together.
We begin with the three judgments that remain the analyst’s own. We then take up sharing with a team through the two-layer model, and sharing with the world through archiving, including what changes when the data are restricted. Stewardship over time, and the recovery of a decayed environment, follow. Lastly, we consider the archetypes, a worked example, and a check of this book’s own compendium against a deposit checklist.
NoteCheck your understanding: two problems, not one
The laboratory’s L2 compendium is shared with a new contributor, who extends the package layer, and is later deposited in Zenodo. Does either event change the level the compendium has reached? What does each event add if not a level?
NoteAnswer
Neither event changes the level. Sharing the compendium with a new contributor, and depositing it with a DOI, are both social and temporal acts, not capture acts. The compendium was L2 before either happened and remains L2 afterward. What sharing adds is that the fixed environment survives contact with a second person without drifting. What archiving adds is that the fixed environment survives the passage of time and the disappearance of the original machine.
13.3 The analyst’s contribution
Before turning to the mechanics, we note that three judgments in this chapter are the analyst’s own, and no tool makes them.
Where to draw the governance boundary, between what the team fixes and what each member may vary. The framework offers the two-layer model as a default. However, only a person who understands the project can decide which layer a given decision belongs to. Fixing too much makes the team rigid and slows every contributor; fixing too little lets the environment fracture. The base image, the R version, the system libraries, and the packages every project in the lab uses are almost always the lead’s to fix; the packages one project’s analysis needs are almost always the members’ to add. The awkward cases in between (a version of a system library that one sub-study needs and the others do not) are decided by the analyst, not the tool.
When to deposit, at what depth, and at what level to claim. An archival deposit is a public and permanent act, and the honest disclosure of the level reached is part of the deposit. Depositing a compendium and labeling it ‘reproducible’ without stating whether it has been verified, or whether the environment was pinned by digest or merely by tag, misrepresents the work. The analyst decides when the compendium is ready and which artifacts accompany it. The analyst also decides which data the consent and the data-use agreement allow to accompany it, and, above all, what the accompanying claim honestly says.
Where the work lives, and how dependent that choice makes it. A registry and a forge are conveniences, but each is also a dependency on a commercial vendor whose terms may change. The analyst weighs the convenience of a popular platform against the vendor independence of an institutional or self-hosted alternative. The analyst does so knowing that a compendium tied to a service that disappears is a compendium at risk.
NoteThe vocabulary of this chapter
Registry. A server that stores and distributes container images, from which collaborators pull a pre-built environment. GitHub Container Registry (GHCR) and Docker Hub are the two in common use.
Forge. A platform that hosts a version-controlled repository and runs its continuous integration. GitHub and GitLab are the two principal forges; GitLab is frequently self-hosted by universities.
Two-layer team model. A division of a project into a slow-moving environment layer, fixed once by a team lead, and a fast-moving package layer, extended by every contributor. The environment layer is the team image: the operating system, the R version, the system libraries, and the packages every project in the lab uses, pinned by digest. The package layer is the project lockfile.
Union model. The rule that the project lockfile records the union of every package any contributor has installed, so that a rebuilt image carries what the whole team needs, not only what one member used.
Persistent identifier (DOI). A durable, resolvable name for a deposited artifact, such as a Digital Object Identifier, that continues to resolve even if the artifact moves.
FAIR. A set of principles holding that a research object should be Findable, Accessible, Interoperable, and Reusable, formulated for data and later adapted for research software.
OCI tarball or SIF file. A container image exported as a single file, checksummable and depositable as an archival artifact. An OCI image is exported as a tar archive, whereas SIF is the self-contained single-file format used by the Apptainer runtime.
Zenodo. A general-purpose research repository, operated at CERN, that mints a DOI for any deposited file set.
Controlled access. A repository arrangement in which data are released only to approved users under an agreement, rather than to anyone who downloads.
Data-availability statement. The statement in an article or README of what was deposited, where, and how any data not deposited may be obtained.
NoteCheck your understanding: whose judgment, again
A compendium is ready to deposit, by every mechanical check: the tests pass, the lockfile is current, the image builds. Name one judgment from this section that a checklist still cannot make, and say why it cannot.
NoteAnswer
A checklist cannot decide what level to claim in the accompanying statement. It can confirm that a lockfile and a Dockerfile exist, which is mechanical. However, it cannot decide whether the analyst has actually re-run the pipeline from a clean rebuild and compared the outputs. Nor can it know what the consent and data-use agreement permit to accompany the deposit. Those are judgments about this specific project’s history and its governing documents, and no tool has access to either.
13.4 Sharing with a team: the two-layer model
We begin with collaboration. The design starts from one observation: the two things a team shares change at very different rates. The computational environment (the operating system, the R version, the system libraries, and the packages every project in the lab uses) is chosen at most a handful of times over the life of a project. The set of packages one project needs, by contrast, changes almost weekly as the analysis grows. A model that forces both to be renegotiated on the same schedule will either freeze the packages to the cadence of the environment, which is too slow, or churn the environment to the cadence of the packages, which is too disruptive. The resolution, adopted by the framework this book uses and set out by its authors, is to separate the two into layers with different change rates and different governance.
Layer 1 is the environment, and it is lead-governed. A designated team lead selects a base image at the project’s start and builds on it a team image carrying the packages every project in the lab uses, as Chapter 7 sketched. The lead resolves that image to an immutable content digest and fixes it for every collaborator. This is the slow-moving half. It corresponds to pillar one, the computational environment, and it is where the L1-to-L2 transition of the ladder is secured. The lead builds this environment once, and the whole team reuses it. The vignette that describes the pattern calls this ‘build once, use everywhere’. It reports that reusing a pre-built team image reduces the setup of a new project from the fifteen or twenty minutes of a full build to a matter of seconds. For reproducibility, the larger gain is that every collaborator works from the same fixed floor.
Layer 2 is the project’s package set, and it is contributor-governed under the union model. Each member installs, on top of the team image, the packages their part of the work requires, freely and without coordination. The project lockfile at any moment records the union of everything anyone has installed. The shared image is rebuilt from that lockfile periodically rather than on every change. When it is, it carries the full set every contributor needs, not merely the subset one member happened to use. A collaborator who pulls the latest main branch and rebuilds therefore inherits the survival packages, the table packages, and the spatial packages all at once. This holds whether or not their own work touches any of them. This is the fast-moving half, corresponding to pillar two, and it is governed by every member.
The union model buys the elimination of a common and maddening failure, the analysis that runs for its author and nowhere else. The price is a little environmental bloat. A package installed once for an exploratory notebook remains in the lockfile for everyone until it is deliberately removed. In our view this is a sound trade. On the one hand, the bloat is bounded and visible, a few extra megabytes in an image that is already a gigabyte or more, and it can be pruned when it matters. On the other, the failure it prevents is a contributor blocked because a colleague forgot to record a dependency. That is the precise ‘works on my machine’ pathology that the whole enterprise exists to abolish. We would rather carry an unused package than lose a day to a missing one.
13.4.1 The Dockerfile, not the image, is the guarantee
The pushed team image is a convenience, a cache that saves build time; it is not the reproducibility guarantee. The guarantee is the Dockerfile, together with the lockfile it restores from, both committed to version control. The reason is that a pushed image is opaque and impermanent. It is a binary blob on a registry that may be deleted, or pruned under a retention policy. It may be pushed for one processor architecture and silently emulated on another, and it cannot be inspected line by line to see how it was made. The Dockerfile is transparent and durable: it is a text recipe, readable and diffable, that reconstructs the image from named and pinned inputs. If the registry vanishes tomorrow, a compendium whose guarantee is the Dockerfile can be rebuilt; a compendium whose guarantee was the pushed image cannot.
The framework’s vignette makes the same point in a note that merits restating in our terms. Pushing the image to a registry is optional, and if the image is not available, building from the Dockerfile reproduces the environment. We should qualify that claim, in keeping with this chapter’s own discipline. A rebuild reproduces the same content (the same base image, R version, and package versions) only when the base image is pinned by content digest and the package snapshot is dated, as Chapter 7 urged. A recipe pinned by a mutable tag, or drawing on a rolling snapshot, reconstructs at best a functionally equivalent environment whose content may have drifted. Even a correctly digest-pinned, dated-snapshot rebuild is not guaranteed byte-for-byte identical at the image level, because per-layer timestamps ordinarily differ between builds. Matching content is the guarantee a pinned recipe gives, and bit-identical bytes require separate reproducible-build tooling (Chapter 7). A team that treats the pushed image as authoritative, and lets the Dockerfile drift out of step with it, has converted its compendium from something reconstructable into something that merely happens to exist. The committed lockfile still holds L1, but the environment layer is no longer captured by anything the compendium contains, so the compendium has regressed below L2. That regression is what honest disclosure is meant to catch.
13.4.2 Registries and the forge
Two distribution choices attend team sharing, and both are concerns about vendor independence. Neither changes the level a compendium has reached.
The registry is where the team image is published: GitHub Container Registry, Docker Hub, or an institutional registry such as one hosted inside GitLab. Docker Hub is the most familiar. However, its free tier rate-limits image pulls and permits only a single private repository, constraints that a small laboratory reaches quickly. GHCR integrates with the GitHub forge and has more generous terms for public research. An institution with confidentiality requirements may mandate a private or self-hosted registry. The choice is a trade between convenience and control, and, we would stress, it is reversible. Because the guarantee is the Dockerfile, a team can re-push its image to a different registry at any time without altering the compendium.
The forge is where the repository and its continuous integration live: GitHub, or, very often in the health sciences, an institutional GitLab. This choice matters because research compendia are disproportionately authored on self-hosted GitLab. Many universities and laboratories, particularly those subject to on-premises data-governance rules, run their own GitLab and cannot push to a public forge. A framework whose premise is reproducibility for that audience must not lock its users into one forge. The practical difficulty is concentrated entirely in continuous integration. GitHub Actions and GitLab CI use structurally different configuration dialects. Supporting both, without maintaining two parallel and divergent sets of workflow files, therefore requires that the CI logic be single-sourced, typically in the Makefile. Each forge’s configuration is then reduced to a thin shim that calls it. The analyst should therefore treat the forge as a replaceable host and keep the reproducibility logic in artifacts that travel between forges unchanged.
NoteCheck your understanding: what survives a registry outage
A colleague argues that the team should push its image to Docker Hub after every package installation, ‘so the image is always the source of truth’. What is wrong with treating the pushed image as the source of truth, and what should be the source of truth instead?
NoteAnswer
The pushed image is an opaque, impermanent binary that may be deleted, rate-limited, pruned, or built for the wrong architecture. It cannot be inspected to see how it was made. If Docker Hub removes it, an analysis that depended on it is lost. The source of truth should be the committed Dockerfile together with the lockfile it restores from. That is a transparent, diffable, durable recipe that reconstructs the same pinned environment, though not byte-identical image layers, from pinned inputs. The image is a convenience cache; the recipe is the guarantee. Pushing after every install also wastes effort, since the image need only be rebuilt periodically from the union lockfile.
13.5 Sharing with the world: archiving
Sharing with a team keeps a live project consistent. Archiving asks a different question: how does a finished compendium survive after the project ends, the collaborators disperse, and the laboratory’s servers are decommissioned? A repository on a forge is not an archive. Forges delete inactive accounts, rename themselves, change ownership, and go out of business; a URL that resolves today may not resolve in a decade. An archive, by contrast, undertakes to preserve a deposit and to keep a name for it resolving indefinitely.
The instrument is a deposit in a research repository that mints a persistent identifier. Depositing the compendium in Zenodo, the general-purpose repository operated at CERN, produces a Digital Object Identifier. The DOI continues to resolve even as the underlying storage moves, and it can be cited in the article the compendium supports. The DOI is what lifts a compendium from merely locatable on a forge, which is fragile, to durably findable, which is the archival requirement.
A deposit should carry the environment as well as the source. Here the container runtimes of Chapter 8 supply the archival artifact needed. Apptainer, the high-performance-computing runtime, builds a single Singularity Image Format file from a digest-pinned Docker image. That SIF, or equivalently an OCI image exported as a tarball, is checksummable and depositable as one object. The deposit needs one command, apptainer build env.sif docker://<image>@sha256:<digest>, which pulls the image by its digest, flattens its layers, and writes them to a single read-only file. A reader who skipped Chapter 8 needs nothing more for this chapter, and a reader without Apptainer can write the tarball with Docker alone, using docker save. A compendium archived as its source tree, a pinned Dockerfile, and the built environment as a single file is reproducible by two independent routes. A reader may rebuild from the recipe or, if the recipe’s inputs have decayed, may run the archived environment directly. Depositing both is belt and suspenders, and for a result that matters it is worth the disk. What a deposit may carry of the data is a separate question. It is governed by the terms under which the data were obtained, rather than by the analyst’s wish to be reproducible. For patient-level data the usual answer is that the data stay out of an open deposit, and Section 13.5.2 treats what is deposited in their place.
These practices realize a set of principles that the community has crystallized under the acronym FAIR. Wilkinson and colleagues proposed that a research object should be Findable, Accessible, Interoperable, and Reusable, and framed the principles for research data (Wilkinson et al., 2016). Barker and colleagues subsequently adapted them for research software, whose reuse involves execution as well as reading (Barker et al., 2022). A DOI serves findability. A repository with a stated access procedure serves accessibility, which for restricted data means a controlled route rather than an open download. A documented, standard container format serves interoperability, and a pinned, rebuildable environment serves part of reusability. Reusability also requires a clear usage license (R1.1 in the original principles), which a pinned environment does not by itself supply. In this light, the archetype-agnostic compendium of Chapter 5 is most of the way to a FAIR object by construction. This holds provided it is deposited rather than merely committed and carries an explicit license; licensing practice is treated separately.
13.5.1 Licensing, CITATION.cff, and Software Heritage
A pinned environment and a minted DOI satisfy three of FAIR’s four letters without touching the fourth’s hardest requirement. Reusability, principle R1.1 in the original statement, asks for ‘a clear and accessible data usage license’ (Wilkinson et al., 2016). Nothing about findability, accessibility, or interoperability supplies one. A compendium can be perfectly findable and still carry no statement of what a reader is permitted to do with it. In the absence of a license, that defaults, under copyright, to nothing. ‘The cohort, continued’ below returns to what license this book’s own compendium carries.
Two further instruments round out what a deposit should carry, beyond the license file itself.
CITATION.cff. A CITATION.cff file is a plain-text, machine-readable citation record placed at a repository’s root, using a small set of key: value fields (cff-version, authors, title, version, and date-released among them) (GitHub Docs, 2024). GitHub parses the file, when it sits on the default branch, into a ‘Cite this repository’ link on the repository page. The link generates an APA or BibTeX citation on request (GitHub Docs, 2024). Zenodo can read the same file when it archives a GitHub release and use it to pre-fill the deposit’s own citation metadata, so a CITATION.cff written once serves both the forge and the archive. What it pre-fills is only what the file says, the license field included. A compendium with one license for its prose and another for its code cannot record in that field which license covers which files, so the license on the resulting deposit is checked against LICENSE before it is published. Without a CITATION.cff, a reader who wants to cite the compendium constructs a citation by hand from whatever metadata happens to be visible on the page. This is the same fragility that motivated the DOI in the first place.
Software Heritage. Zenodo is a deposit repository: an institution, CERN, operates it. Its continuation is a reasonable expectation, but it is an expectation and not a guarantee, since a deposit repository could in principle be discontinued by the institution that runs it. Software Heritage undertakes something narrower and more absolute for source code specifically. Its stated mission is a permanent, universal archive of publicly available source code. A depositor cannot withdraw what is archived; removal happens only through a takedown procedure that the project’s staff carry out. The project’s SWHID identifier scheme was formalized as an ISO/IEC standard in 2025 (OpenChain Project, 2025). Every archived object, whether a file, a directory, a commit, or a full repository snapshot, receives a SoftWare Hash IDentifier of the form swh:1:<type>:<hash>. Here <type> names the kind of object (cnt for a file’s content, dir for a directory, among others) and <hash> is a 40-character value computed from the object’s own bytes (Software Heritage, 2024). The identifier is computed from the content rather than assigned by a registry. Two copies of the same file anywhere in the archive therefore resolve to the same SWHID. This is a different kind of guarantee from a DOI: a DOI names a deposit, and a SWHID names the bytes.
This book’s own repository illustrates the gap between having these instruments and using them well. A CITATION.cff is present at the repository root, generated by zzcollab when the project was created. Its license field reads CC-BY-4.0, the license of the prose, while LICENSE places the code under MIT and GPL-3; the file’s abstract states all three licenses in words, but a deposit service reading the license field sees only the one identifier. Its orcid field is still empty, so the record names its author without a persistent identifier. The file’s presence is enough for GitHub to render the ‘Cite this repository’ link and for Zenodo to read it on deposit. Whether its content credits the right people under the right license is a question only a reading of the file answers.
Two broader efforts merit the analyst’s attention. The Whole Tale platform, described by Brinckman and colleagues, bundled code, data, and computational environment into a single executable research object that a reader could run in the browser without local installation (Brinckman et al., 2019). Its public service did not respond when we checked in November 2026. It remains an ambition the compendium-plus-archived-image pattern approaches by other means. And the Turing Way, a community-authored handbook, collects the collaborative and stewardship practices of reproducible research into one continually updated reference (The Turing Way Community, 2022). Some readers will want the social side of collaboration, from code review to project governance to credit, in more depth than a single chapter allows. They should begin there.
13.5.2 When the data are restricted
Everything said so far about depositing assumes that every file in the compendium may be published. In health research that assumption is usually false. A compendium built on patient records contains, or was built from, such data. Their release is governed by law, by the consent the participants gave, and by the data-use agreement under which the analyst received the extract. A deposit in an open repository is a publication to anyone, indefinitely, and a file placed there cannot be recalled.
ImportantRestricted data
The rule is the commit rule of Chapter 4, made stricter. Patient-level data do not go into an open deposit unless three conditions hold. They must have been formally de-identified, and the consent and the agreement must both permit public release. The responsible data steward or IRB must also have confirmed it in writing. In their place, a deposit may take four routes:
the code, the tests, the lockfile, the Dockerfile, and the archived image, always;
de-identified data, only where the data’s steward has made that determination;
a synthetic stand-in, labeled as such;
for data that may be shared under agreement but not opened to the world, a citation to a controlled-access repository such as dbGaP.
None of these substitutes removes the obligation to read the consent form and the data-use agreement first. The deposit’s data-availability statement must say, exactly, which of these routes was taken.
The paragraphs below work through each of the four routes named above, and the two documents that govern the choice, in enough detail to act on.
The code and the environment, always. The analysis code, the tests, the lockfile, the Dockerfile, and the archived image carry no participant data. Depositing them openly lets anyone inspect the analysis, and lets anyone who obtains the data by the legitimate route re-run it. Two checks come first. The archived image must not contain the data. A COPY . . executed while the extract sat in the build context places it inside the image. A .dockerignore that excludes the data directory, and an inspection of the built image’s file system, are therefore part of the deposit. And the other artifacts must not carry individual-level rows by another route. Rendered reports, cached chunk output, saved workspaces, logs, and test fixtures are all places where records leak. Tables with very small cell counts can disclose as surely as a row does.
De-identified data, where permitted. Under the HIPAA Privacy Rule, health information ceases to be individually identifiable by one of two routes. In the first, a qualified expert applies statistical methods and documents that the risk of identification is very small. In the second, the eighteen listed identifiers are removed and the covered entity has no actual knowledge that the remainder could identify anyone (45 CFR 164.514, n.d.). A limited data set, which keeps dates and some geography, is not de-identified and may be shared only under a data-use agreement (45 CFR 164.514, n.d.). NIH guidance adds that data de-identified in the legal sense may still warrant controlled access (National Institutes of Health, 2026c). De-identification is a determination made by the data’s steward, not something an analyst achieves by deleting the name column.
Synthetic data, as a stand-in. A synthetic file with the same columns, types, and plausible ranges as the analysis data lets a reader run the pipeline from end to end. The reader can confirm that the code executes, though not that the published estimates regenerate. It should be labeled synthetic in the file and in the README, with the script that generated it. A synthetic file produced by a model fitted to the real records needs the same disclosure review as the records themselves, since such a model can reproduce rare individuals. One simulated from stated parameters, as this book’s running cohort is, does not.
Controlled-access repositories. When data may be shared but not publicly, a controlled-access repository holds them and releases them only to approved users under an agreement. NIH’s dbGaP, for genotype and phenotype data, is an example: public study descriptions sit beside an authorized-access route with its own security procedures and code of conduct (National Center for Biotechnology Information, n.d.). The open deposit of code then cites the controlled-access accession, so that the two halves are linked by identifier. The NIH Data Management and Sharing Policy has been in force since 25 January 2023. It requires access controls for data with explicit limits on later use (National Institutes of Health, 2020). NIH’s privacy guidance adds that those limits should travel with the data and be communicated to the repository (National Institutes of Health, 2026c).
Consent and agreement terms, read before the analysis begins. What may be deposited is settled by documents signed long before the deposit: the consent form, the IRB protocol, and the data-use agreement. The analyst reads them for redistribution, derived data, public posting, and retention. An agreement that requires the extract to be destroyed at the end of the project governs every copy of it, including those in backups, caches, and container images. NIH guidance asks that consent explain how data will be shared and that use agreements carry their restrictions with the data (National Institutes of Health, 2020, 2026c).
The data-availability statement. The article and the README state what was deposited, where, and how the rest may be obtained, with the same exactness the honest-level claim demands. A statement of the following form is typical:
The analysis code, package lockfile, container recipe, and archived container image are openly available at [DOI], with a synthetic dataset of the same structure on which the pipeline runs from end to end. The patient-level data were obtained from [registry] under a data-use agreement that does not permit redistribution; qualified researchers may apply to [data steward] for access. The synthetic data do not reproduce the reported estimates.
The last sentence bears on the level claim. A reader without access to the data can confirm that the pipeline executes. Only someone with access can confirm that it regenerates the published numbers, so an L3 claim must say who performed the comparison and on what data.
NoteCheck your understanding: which route, and why not the obvious one
A colleague proposes removing the name and medical record number columns from a patient-level extract and depositing the rest openly, calling the result ‘de-identified.’ What is wrong with this plan, and which of the four routes above should be used instead?
NoteAnswer
Deleting two columns is not de-identification in the regulatory sense. Dates, small geography, and rare combinations of remaining variables can still re-identify a person. De-identification needs either a qualified expert’s documented determination that the risk is very small, or removal of all eighteen HIPAA identifiers with no actual knowledge that the rest could identify anyone. The analyst’s own judgment is neither. Absent that determination in writing from the data steward or IRB, the extract stays out of an open deposit. The appropriate route is a synthetic stand-in with the same structure, deposited openly and labeled as synthetic. The real extract is made available, if the agreement allows it at all, only through a controlled-access repository or a direct request to the data steward.
13.5.3 Working inside a trusted research environment
The four routes above assume that the analyst holds a copy of the data and must decide what to release. For data whose provider never releases a copy, that assumption fails. A trusted research environment, also called a secure data environment, a secure processing environment, or a data enclave, reverses the direction of travel. The data stay on the provider’s infrastructure, and the analyst brings code to the data rather than data to the code. The NIH All of Us Researcher Workbench is a US example. Its documentation states that cohort building and analysis take place solely on the Workbench, which supplies RStudio and Jupyter among its tools (National Institutes of Health, All of Us Research Program, n.d.-b). In the European Union, the European Health Data Space Regulation entered into force on 26 March 2025. Secondary use under it requires a permit from a health data access body. Processing may then take place only in a secure processing environment, from which users can download only non-personal data (Article 73). The rules on secondary use apply to most data categories from March 2029 (European Commission, n.d.; European Parliament and Council of the European Union, 2025). In the United Kingdom, the SATRE specification, an open community standard whose release 2.0.1 is dated July 2026, sets out what such an environment should provide (SATRE Community, 2026).
The Five Safes. A common way to describe the controls of such an environment is the Five Safes framework, which originated at the UK Office for National Statistics (UK Data Service, n.d.). Safe projects are approved by the data owner as serving the public good. Safe people are trained and authorized researchers. A safe setting is an environment that itself prevents unauthorized use. Safe data have been treated to protect confidentiality. Safe outputs are those screened and approved as non-disclosive before release.
Output checking. The last of the five is the one that shapes the analyst’s daily work. Nothing leaves the environment, whether a table, a figure, a log, or a fitted model, until it has passed statistical disclosure control. SATRE makes two things mandatory for a provider: a documented process for disclosure control of outputs, and a process that assigns responsibility for output checking. It also recommends that each project’s intended outputs be established at the outset (SATRE Community, 2026). The rules a checker applies are concrete. Under the All of Us dissemination policy, no participant count from 1 to 20 may be published, nor any figure, percentages included, from which such a count could be derived (National Institutes of Health, All of Us Research Program, n.d.-a). This is the small-cell hazard named under the first route above, now enforced by someone other than the analyst.
A reproducible pipeline is what makes this review feasible. A checker can judge an output only when its provenance is plain: which script produced it, from which cohort, and with what suppression. A pipeline that writes every exportable table from a script, applies the threshold in code, and records what it suppressed turns each request into a small, checkable package. A revision the checker asks for is then a re-run, not a hand edit of a table whose origin nobody can reconstruct.
What reproducibility can mean inside. The environment belongs to the provider. The analyst cannot pin its operating system, may not be permitted to run Docker, and may reach only the package sources the provider chooses to offer. SATRE lists access to public package repositories, and tight control of which packages are available, as options for the provider rather than obligations (SATRE Community, 2026). The ladder still applies, with a change of owner at the environment layer. The lockfile records the package versions, restored against whatever mirror the environment offers. The operating system and R runtime are the provider’s, so the compendium records the provider’s image or environment identifier, read from inside the environment, in place of a Dockerfile of its own. A claim of L2 is honest only to the extent that this identifier names a fixed environment. What then enters the open deposit is the code, the lockfile or that identifier, a synthetic extract of the same structure, and the release record of what passed output checking and when. SATRE lists a machine-readable record of the disclosure checks applied to each output as an option (SATRE Community, 2026). L3 is the rung that cannot travel. The comparison with the published numbers can be run only inside the environment, by an approved user, so the level claim must say who ran it and where. The data-availability statement names the environment and its application route, as the controlled-access route above names an accession.
13.5.4 Funder and journal obligations
Everything the four routes above implement is a response to an obligation that exists before the analysis begins: a funder’s data-sharing policy and, later, a journal’s reporting requirement. Both predate the deposit and shape what the deposit must contain.
The funder side. The NIH Data Management and Sharing Policy applies to proposals due on or after 25 January 2023. It requires that an applicant submit a Data Management and Sharing Plan at the time of application, not drafted after data collection is underway (National Institutes of Health, 2020). Applications submitted on or after 25 May 2026 use an updated Plan format (National Institutes of Health, 2026b), and career development (K) awards no longer need a Plan (National Institutes of Health, 2026a). The Plan goes in the ‘Other Plan(s)’ attachment of the research plan; only a brief summary and the associated costs go in the budget and its justification (National Institutes of Health, n.d.). NIH program staff assess the Plan. Peer reviewers may comment on its budget, but those comments do not affect the overall score. Once an award is made, the approved Plan becomes a term and condition of that award. Noncompliance can bring added special terms and conditions or termination of the award, and can affect future funding decisions (National Institutes of Health, 2020). For the analyst, the choice among the four routes above was therefore made, in outline, when the grant was written; the deposit executes the Plan.
The journal side. A code-availability statement is a narrower instrument than the data-availability statement of this section, above. It says where the analysis code lives, under what license, at what version, and what it does not include. It does so independently of whatever the data-availability statement says about the data. The two statements answer different questions, and a manuscript needs both. A reader who wants to re-run the pipeline reads the code statement, and a reader who wants to obtain the inputs reads the data statement.
Some reporting guidelines now name code availability as a checklist item rather than leaving it to a journal’s general policy. TRIPOD+AI is the 2024 extension of the TRIPOD reporting guideline to prediction models built with regression or machine learning. Under its ‘Open science’ section it lists item 18f, ‘Provide details of the availability of the analytical code,’ and adds a footnote. The footnote specifies that this covers the analysis code, including any data cleaning, feature engineering, model building, and evaluation (Collins et al., 2024). A separate item, 18e, asks the same question of the data. Naming code availability as its own item, distinct from data availability, is what makes TRIPOD+AI concrete where a general journal policy is often vague: an author completing the checklist cannot satisfy 18f by describing only the data.
A code-availability statement that meets this bar states four things. The first is where the code is deposited, with a persistent identifier if one was minted. The second is under what license it is released. The third is which version, commit, or tagged release corresponds to the reported results, since a repository’s default branch moves on after publication. The fourth is what, if anything, the deposit withholds, and why (a proprietary dependency, institution-licensed software, or data referenced only by an accession number). ‘Available on GitHub,’ without a version and a license, answers none of the four.
For randomized trials, the two guidelines this audience completes most often took the same step in 2025. SPIRIT 2025, for trial protocols, and CONSORT 2025, for trial reports, each gained a new ‘Open science’ section (A.-W. Chan et al., 2025; Hopewell et al., 2025). In SPIRIT 2025, item 5 asks where the protocol and the statistical analysis plan can be accessed. Item 6 asks where and how the de-identified participant data, with their data dictionary, the statistical code, and any other materials will be made accessible. CONSORT 2025 asks the same of the finished report. Its item 3 now asks where the statistical analysis plan, as well as the protocol, can be accessed. Its new item 4 asks where and how the individual de-identified participant data, with their data dictionary, the statistical code, and any other materials can be accessed. Two consequences follow. The protocol promises, under SPIRIT item 6, what the report must deliver under CONSORT item 4, so the code-sharing route is fixed before enrollment, much as the NIH Plan fixes the deposit route. And the statistical analysis plan must itself be accessible, so the compendium should hold or cite it beside the code that implements it.
Journal policies vary more than the guidelines do. Table 13.1 summarizes three that biostatistics and epidemiology authors commonly meet, each read from the journal’s own pages on 8 November 2026. Policies change, so the target journal’s current instructions, not this table, govern a submission.
Table 13.1: Code-availability policies of three journals, as stated on each journal’s author pages on 8 November 2026.
Journal
Code and data
Review of the materials
Journal of the American Statistical Association
Code, data, and a workflow that reproduces the work are required at invited revision: for Applications and Case Studies since 2016, for Theory and Methods since 2021. An Author Contributions Checklist form is required at submission (Journal of the American Statistical Association, n.d.).
Sharing of data and code is strongly encouraged where ethically feasible. Papers are marked D when the data are freely available, C when the code is, and R when both are (Biostatistics, n.d.).
None described on the author-guidelines page.
American Journal of Epidemiology
A data-availability statement is required. Sharing of data and code is strongly encouraged where ethically feasible (American Journal of Epidemiology, n.d.).
Archiving preserves the artifacts; stewardship is the longer discipline of keeping them runnable as the world underneath them changes. Even a carefully pinned compendium decays. A base image tag may be re-pushed. A package version pinned in a lockfile may age out of the repository that once served it, and a system library may be dropped from a new operating-system release. The question stewardship answers is this. When the pins have decayed and the archived image was lost or never built, can the environment still be recovered?
For the R ecosystem a useful recovery tool is rang, described by Chan and colleagues (C. Chan & Schoch, 2023). (rang was archived on CRAN on 2026-08-22. The historical versions remain installable from the CRAN archive or from its GitHub repository. The technique it illustrates is worth knowing even if the package itself needs an alternative source or a successor tool by the time a reader needs it.) The ordinary workflow pins prospectively, by recording a lockfile at development time and restoring from it. rang, by contrast, reconstructs an environment retrospectively, from a date. Given the date an analysis was run, it resolves the package versions current on the archives at that date. It then generates a Dockerfile that can often reconstruct a period-correct environment even when no original lockfile survives. The two approaches are complementary on a time axis. The framework of this book enforces consistency prospectively over a project’s active life, and rang provides a recovery path retrospectively when that consistency has lapsed. A prudent steward records the run date in the compendium so that this recovery path stays open.
ImportantThe honest level
Sharing and archiving themselves reach no rung, since preserving a level is not capturing or verifying one. The single failure this chapter exists to prevent is depositing a compendium no better than locatable, with a DOI and a tidy README, and calling it ‘fully reproducible’ in the article the deposit outlives.
We must extend the same honesty to the tools this book has taught, and to the framework it uses as its running example. Frameworks that compose the whole determinant stack are young. Measured against the 9 layers of the determinant stack in Chapter 3, the framework firmly pins roughly four: the operating system and system libraries, the R runtime, the package versions, and the analysis code, with much of the session configuration besides. It leaves the rest (the hardware, the numerical libraries, the input data, and intrinsic non-determinism) to the base image, to the analyst, or to convention. Its own system review describes it candidly as a strong scaffolding tool rather than a finished guarantor of reproducibility. The review also notes the sustainability risk that attends a small maintainer base: a single principal author is a bus factor of one, and the tool’s future depends on that one person. The appropriate response is to disclose its reach accurately, the same discipline the honest-level callout demands of any analysis. The evidence that disclosure cannot be taken for granted is empirical. Stodden and colleagues tested a journal policy requiring code and data on request. They obtained artifacts for 44 percent of 204 articles and reproduced the findings of 26 percent (Stodden et al., 2018). A gap between the reproducibility a project claims and the reproducibility it delivers is common.
NoteCheck your understanding: recovering a decayed environment
An analysis was run in 2024 and deposited as a source tree and an renv.lock, but no Docker image was ever built, and by 2029 several of the pinned package versions are no longer installable from CRAN. A reviewer asks you to reproduce it. Which tool addresses this situation, what single piece of recorded information does it most need, and why is this a stewardship failure rather than a capture failure?
NoteAnswer
The rang package (C. Chan & Schoch, 2023) addresses it by reconstructing a period-correct environment from a date. It resolves the package versions current at that date and generates a Dockerfile, even when the pinned versions are no longer installable directly. The single most useful recorded fact is the date the analysis was run, from which rang resolves the historical package set. It is a stewardship failure rather than a capture failure because the pins were captured correctly at the time (the compendium was L1). What lapsed was the preservation of a runnable environment over time, which an archived image or a recorded run date would have protected.
13.7 Across the archetypes
We turn now to how the machinery varies, or does not, across the archetypes. The team and archiving machinery is archetype-agnostic: the two-layer model, the union lockfile, the registry and forge choices, the DOI deposit, and the rang recovery path apply to every archetype; Table 5.2 in Chapter 5 is the reference. What varies is the final act of publication. A data-analysis or simulation compendium is deposited as itself: the compendium with its DOI is the published object. A manuscript compendium is deposited beside the article it supports, cited by DOI from its data-availability statement. A package compendium has a further, ecosystem-native channel, release to CRAN or an R-universe, whose incoming checks are themselves a validation gate. A blog compendium is deployed as a website, with tools such as workflowr(Blischak et al., 2019) binding each rendered page to the commit that produced it. The archival obligation still applies to the compendium behind the posts. In every case the underlying discipline is identical: fix the environment, govern it as a team, deposit with a persistent identifier, and state the level honestly. Only the last mile differs.
NoteCheck your understanding: the archetype that changes nothing
A simulation study, built entirely from its own generated data, is being prepared for deposit. Which part of this chapter’s archiving machinery does it need, and which part, unlike the manuscript and package archetypes, does it not need to worry about?
NoteAnswer
It needs everything archetype-agnostic. It needs the two-layer model while the study was developed by a team, and the committed Dockerfile as the guarantee. It also needs a DOI-minting deposit carrying the code and the archived environment, and an honest statement of the level reached. It does not need Section 13.5.2’s machinery at all. Data generated by the study’s own code from stated parameters carries no participant, no consent, and no data-use agreement to honor.
13.8 Worked example
To make the abstractions concrete, we follow the multi-site cohort laboratory of the Orientation through one quarter of collaboration and one archival deposit.
At the project’s start the lead selected a base image, resolved it to a digest, and built the team environment once. Over the quarter, three contributors extended the package layer independently. The union model records the result: the project lockfile always holds the union of every contributor’s additions. The following chunk reconstructs the running union over the quarter and tabulates it. It uses only base R together with tibble and knitr, and computes the cumulative union deterministically, so it regenerates identically on every render.
Table 13.2: The renv union model over one quarter of team collaboration. Each contributor installs packages independently; the project lockfile records the cumulative union, which the shared image is periodically rebuilt from. Because it is a union rather than a running total, a package a contributor installs that another already has does not enlarge it, so the union can grow by fewer than the number of packages added. The union size never shrinks, so every contributor who rebuilds inherits the full set.
Contributor
Packages added
Lockfile union (n)
Lead (initial base)
here, renv, rmarkdown
3
Ana (survival)
survival, survminer, here
5
Ben (tables)
gtsummary, gt
7
Cai (spatial)
sf, tmap
9
We see in the table the invariant that makes the model work. The union size is monotone, never shrinking as contributors are added, so a member who rebuilds the image at any point inherits everything the team needs up to that point. Because it is a union and not a running total, a package a contributor installs that the team already has does not enlarge it. Ana adds 3 packages, but one of them, here, was already in the lockfile from the initial base. The union therefore grows by only 2, from 3 to 5. Ana never installs sf, yet after the quarterly rebuild her environment carries it, because Cai did, and the lockfile records the union.
When the manuscript was accepted, the laboratory deposited the compendium. The cohort’s patient-level extract came from the participating sites under a data-use agreement that forbids redistribution. The deposit therefore follows Section 13.5.2. It carries the code, the tests, the lockfile, the Dockerfile, the archived environment, and a synthetic extract with the structure of the real one. It also carries a README whose data-availability statement names the sites’ data steward as the route to the real data. The extract itself is in none of these files. The session below is shown as a static illustration rather than executed. It checks that the image holds no study data, publishes it to a registry, and records its immutable digest. It then exports the environment as a single archival file and mints a DOI for the compendium. The $ prompt is not part of the commands, and lines beginning #> are output.
# Rebuild the team image once, from the pinned# Dockerfile and the union lockfile, then publish it.$ make docker-build# Before the image leaves the laboratory, confirm that# no study data were copied into it; the expected# result is an empty listing.$ docker run --rm cohort-env:latest \ find / -xdev-path'*raw_data*'-type f$ docker tag cohort-env:latest \ ghcr.io/mylab/cohort-env:2026-q3$ docker push ghcr.io/mylab/cohort-env:2026-q3# Record the immutable digest the team builds against.# This, not the mutable tag, is the reproducible name.$ docker inspect --format'{{index .RepoDigests 0}}'\ ghcr.io/mylab/cohort-env:2026-q3#> ghcr.io/mylab/cohort-env@sha256:4813816ad695...# Export the environment as a single archival file for# deposit, built from the digest-pinned image.$ apptainer build cohort-env.sif \ docker://ghcr.io/mylab/cohort-env@sha256:4813816ad695...# Deposit the compendium (code, tests, lockfile,# Dockerfile, synthetic extract, README; no patient# data) plus the archived environment in Zenodo, which# mints the persistent identifier the article's# data-availability statement will cite. The command is# schematic; real deposition uses the Zenodo REST API or# the web upload interface, and the DOI is a placeholder.$ zenodo-upload cohort-compendium-v1.0.zip cohort-env.sif # schematic#> Deposited. DOI: 10.5281/zenodo.XXXXXXX
Three features of the session deserve note. First, the image is pushed by a dated tag but recorded by digest; the tag is convenient for humans, the digest is the immutable name a collaborator or a reviewer builds against. Second, the deposit carries both the compendium and the archived environment. Reproduction therefore has two independent routes, rebuild-from-recipe and run-the-archived-image, and does not depend on the registry surviving. Third, neither route includes the patient data. A reader of the deposit can run the pipeline on the synthetic extract and see that it executes. Regenerating the published estimates requires the real extract, obtained under the sites’ agreement, and the article’s level claim says so.
NoteCheck your understanding: two routes, one reason
The deposit above includes both a rebuildable Dockerfile and an archived SIF file of the same environment. Why both, rather than whichever is more convenient?
NoteAnswer
Because each route can fail independently, and the deposit is meant to outlive both failures. The Dockerfile route fails if the base image’s registry disappears or a pinned package is pulled from CRAN before rang or a successor tool can recover it. The archived-image route fails if the SIF or OCI file itself is lost or becomes unreadable by a future runtime. Depositing both means a reader needs only one of the two to still work. The archived image is often a gigabyte or more, so the second route is not free, but for a result that matters it is worth its size.
13.9 The cohort, continued
The worked example above is a composite laboratory. We now apply a smaller, real question to this book’s own running cohort compendium. If it had to be deposited today, what would already be true, and what would still need doing?
The first two checks concern licensing. A LICENSE file is present, and it covers three kinds of material. The prose is CC BY 4.0, the code printed in the book is MIT, and the repository code, including every script that builds the cohort, is GPL-3. That is the explicit license FAIR’s reusability principle asks for, and the one Section 13.5.2’s deposit example above was silent about. A DESCRIPTION file is present and records License: GPL-3, matching the repository-code section of the LICENSE file, so a package repository or an automated check reading DESCRIPTION sees the same answer for the code that a human reading LICENSE would. CITATION.cff does not: its single license field reads CC-BY-4.0, the prose license, and a deposit service that takes its license from that field would record it for the code as well. Before a deposit, the license recorded in the deposit’s own metadata is checked by hand against LICENSE.
The last two checks ask git itself, through git check-ignore, whether a path would be ignored. The cohort extract analysis/data/raw_data/cohort_raw.csv is ignored. The more informative result is the second. The file registry_extract_2027.csv does not exist and is named nowhere in .gitignore, yet git reports that a file of that name placed in the same directory would be ignored as well. This is what protects real data. A rule that names one file protects that file and nothing else, and the next extract a teammate saves beside it under a new name would be staged by the next git add . without a warning. The directory pattern analysis/data/raw_data/*, with an exception for the directory’s README.md, protects whatever is put in the directory, including files no one has yet thought to name. A text search of .gitignore for the string raw_data cannot tell these two cases apart, which is why the chunk asks git rather than reading the file, and why it tests a filename that appears nowhere in it. The CSV itself is generated from a committed script rather than tracked. This is the practice that the version-control chapter (Chapter 4) and Section 13.5.2 both prescribe for data with identifier-shaped columns. The chunk cannot certify whether the data are actually sensitive, only which paths git would exclude. That judgment is the analyst’s, informed by what the data are.
On the capture ladder, the compendium is at L1. What it still lacks among the artifacts checked here is a Dockerfile and a .dockerignore. The .dockerignore matters as soon as a Dockerfile exists, because .gitignore keeps files out of git and not out of a Docker build context; Exercise 8 below follows a file through that gap. What the compendium still lacks for an archival deposit proper is the part no file-system check can see. It lacks a minted DOI, an archived container image, and a written statement of the level this compendium has reached at the time of deposit.
NoteCheck your understanding: what the checks did not check
Suppose every check above returned TRUE for this compendium, a Dockerfile and a .dockerignore included. Would that mean the compendium is ready to deposit as L3? Why or why not?
NoteAnswer
No. The checks are structural: a license file exists, a DESCRIPTION agrees with it, the capture files exist, and git would exclude the data directory. None of them is evidence that the analysis has been rebuilt from a clean environment and its outputs compared against a committed baseline within a stated tolerance. That is what an L3 claim requires. A compendium can pass every structural check above and still be no higher than L2, because files that capture the environment are not evidence that anyone rebuilt it. Structural readiness for deposit and the level honestly claimed in the deposit are two different questions, and this section answers only the first.
13.10 Collaborating with an LLM
A language model is a capable assistant for the mechanical parts of sharing and archiving. It is an unreliable one for the parts that require judgment about this specific project.
ImportantRestricted data
One rule comes first (habit 3). Never paste or upload protected health information, identifiable records, or data held under a data use agreement, nor outputs derived from them, into a model or a coding agent unless three conditions all hold. The institution has approved that endpoint for that class of data. The IRB protocol and the consent permit the processing. The data use agreement, or the provider’s terms, permits it. If any of the three is unclear, the answer is no.
The third condition is the one most often missed. Institutional approval of a tool does not override the terms under which a data provider released the data. NIH, for example, treats sharing controlled-access data through prompts with public generative AI tools, which it glosses as third-party tools, as a breach of the data use certification. It treats a model trained on such data as a data derivative, to be shared only with other approved users and not kept after the project closes (National Institutes of Health, 2025).
Describe the structure instead: the column names and types, or a small synthetic example generated from the data dictionary, never edited from real rows. A coding agent sees whatever files are in its working tree, so run it in a checkout or container where the real data is not mounted.
An agent also acts, so confine it to a branch and keep merges, pushes, and deposits for yourself (Section 2.10.1).
For a deposit, the derived outputs that fall under the rule include row-level logs and small-count tables. The NIH notice cited above names public generative AI tools, which it glosses as third-party tools, and makes no exception for a tool an institution has licensed. A prompt about a deposit names the file names and the directory layout rather than any file’s contents. We give three exchanges, each with what to watch for and what the analyst must verify.
Prompt. ‘Write the shell commands to tag our image, push it to GHCR, record its digest, and export it as a SIF for archival deposit.’
Watch for. Check whether the commands push and record the mutable tag rather than resolving and recording the digest. Recording the tag silently discards the reproducibility the exercise exists to secure. Check also every registry path and flag against the registry’s own documentation and the analyst’s setup.
Verification. Run each command and confirm the recorded reference is a @sha256:... digest, not a :tag. Pull the image fresh on a second machine using the recorded digest and confirm it resolves. Do not deposit until the digest round-trips.
With a coding agent, these commands are executed rather than read. A push to a public registry and an export bound for deposit are actions the analyst takes after reading the commands, not ones an agent runs on its own authority (Section 2.10.1).
Prompt. ‘Draft the data-availability and reproducibility statement for our manuscript, citing the deposited compendium.’
Watch for. The model does not know what level the compendium actually reached, whether the analyst verified the outputs or merely pinned them, or what the data-use agreement permits. Check for confident language (‘fully reproducible’, ‘all results can be regenerated’) that the compendium has not earned. Check also for a sentence such as ‘all data are available in the deposited compendium’ attached to a deposit that holds none.
Verification. Rewrite the level claim by hand against the honest-level standard. Say verified only if zzc verify or its equivalent was run and its baseline test passed; say pinned if the environment was pinned but not re-run. Name the DOI actually minted, not a placeholder. Check every sentence about the data against what the deposit holds and against the agreement, and name the access route for anything withheld.
A better prompt. ‘Draft a data-availability statement and a code-availability statement for our manuscript. Inputs: the attached deposit file listing, the DOI <minted DOI>, the level reached (L1: a lockfile, no baseline re-run), and the agreement’s redistribution clause, pasted below. Claim no level above L1 and no data the listing does not show. Give two short paragraphs, then list each sentence with the file or clause that supports it, so that I can check them one by one.’
An agent given access to the deposit service could go on to publish the record and mint the DOI. A published deposit cannot be recalled (Section 13.5.2), so publishing it, like the level claim, stays with the analyst (Section 2.10.1).
Prompt. ‘Our 2024 analysis has an renv.lock but several pinned versions no longer install. How do we reproduce it?’
Watch for. Check whether the answer edits the lockfile to newer versions, which changes the analysis rather than reproducing it. If the answer recommends rang, check it against the caveat in ‘Stewardship over time’: the package was archived on CRAN on 2026-08-22, and it is built for the case in which no lockfile survives. Here one does, so the versions to install are already known.
Verification. Confirm that the proposed route installs the recorded versions, from the CRAN archive or from a dated repository snapshot, with the recorded run date used only to choose the snapshot. Diff the installed versions against the lockfile. Test the reconstructed environment against a known output from the original analysis before trusting it.
13.11 Exercises
Take an existing single-analyst compendium of your own and identify, for each package it uses, whether the decision to include it belongs to the environment layer or the package layer of a hypothetical team version. Justify the two or three borderline cases.
Explain, in a paragraph a collaborator could act on, why the committed Dockerfile rather than the pushed image is the reproducibility guarantee. Then describe one concrete situation in which relying on the pushed image would cause a reproduction to fail.
Your institution runs a self-hosted GitLab and forbids pushing to public forges. List the components of a compendium’s reproducibility workflow that must change when moving from GitHub to GitLab, and the components that do not, and identify which single place the CI logic should live so that the change is minimal.
Deposit a small practice compendium in a research repository that mints a DOI (Zenodo offers a sandbox for exactly this). The compendium must hold no restricted data: use public or synthetic data only, such as this book’s simulated cohort. Record the DOI. Then write a data-availability statement that says what the deposit contains and names the level the compendium actually reached, defending your level claim against the honest-level standard. Then rewrite the statement for the case in which the analysis data were a registry extract held under a data-use agreement. Say what would have been deposited instead and how a reader could obtain the real data.
Take a compendium with an renv.lock and, using rang, generate a Dockerfile that reconstructs its environment from the date it was created. Compare the reconstructed package set against the lockfile and report any differences, explaining why a date-based reconstruction may not match a lockfile exactly.
For one archetype other than the manuscript (a package, a blog, or a simulation), write the specific ‘sharing with the world’ step it requires beyond the archival deposit, and explain what validation gate, if any, that step adds.
Run the checks from ‘The cohort, continued’ yourself, from the repository root rather than from analysis/report/ (so set repo <- '.'). For the ignore checks, also run git check-ignore --no-index -v in a terminal on analysis/data/raw_data/cohort_raw.csv and on a second file name of your own invention in the same directory, and note which line of .gitignore git reports as the match. Report the booleans you get. Then explain, in two or three sentences, why the result for your invented file name, not the one for cohort_raw.csv, tells you whether real patient-level data saved in that directory would stay out of git, and name one check in the chunk that would still have to change before the compendium could hold such data safely.
This exercise returns to the commit rule of Chapter 4 (Chapter 4): never commit restricted data, because git keeps every version forever even after a file is deleted. The same rule applies one layer up. Suppose a .dockerignore lists *.csv but a teammate’s branch adds analysis/data/raw_data/extra.tsv the week before a deposit, and nobody updates the ignore patterns. Walk through what happens to that file at each of three points: the docker build, the docker push, and the Zenodo deposit of the exported SIF. Then say which single artifact, if inspected before the deposit, would have caught the leak.
Write a code-availability statement, meeting TRIPOD+AI item 18f’s standard, for this book’s running cohort compendium as it stands today. Name the repository, the license (recall ‘The cohort, continued’ above), the commit or tagged release the statement should point to, and anything withheld. Then write the parallel data-availability statement for a hypothetical version of the same analysis built on a registry extract held under a data-use agreement. Say in one sentence how the code statement would differ from the data statement in that case.
13.12 Solutions
Attempt an exercise before expanding its solution. Most of these are performed on a compendium of the reader’s own, so their outputs vary; the boxes record the criterion to apply and the reasoning behind it.
NoteExercise 1
No single correct answer. The criterion to apply is how often the package changes and how many contributors need it.
The environment layer is the team image. Beyond the operating system, R, and the system libraries, it holds the packages every project needs and nobody changes: the tidyverse, knitr and rmarkdown, the plotting stack, anything requiring system libraries that are expensive to install. Baking these into the image means each contributor pays the install cost zero times rather than once each. The layer moves only when the team lead rebuilds.
The package layer holds what one contributor needs for one piece of work: a specialized survival package, a mapping library used in a single figure, anything adopted mid-project. These arrive through the lockfile, which every contributor extends.
The borderline cases are the substance of the exercise, and three kinds recur.
A package that started in the package layer and is now used by everyone belongs in the environment layer at the next rebuild. The signal is the union model: when a package appears in the lockfile because several contributors installed it independently, it has outgrown the fast layer.
A package with heavy system dependencies used by only one contributor is ambiguous. Installing it from the lockfile is slow and may fail without the system libraries; baking it in imposes the size on everyone. The usual resolution is to put the system libraries in the image and leave the R package in the lockfile.
A package pinned to an unusual version for one analysis belongs in that project’s package layer, however widely the package itself is used. The team image serves every project in the lab and should carry the version most of them use. One project’s lockfile can record the unusual version and restore it into the project library, which takes precedence over the image’s copy, without imposing it on the other projects. Within a single project the union lockfile still records one version per package, so that project’s contributors must agree on it.
NoteExercise 2
A paragraph a collaborator could act on:
The Dockerfile is a recipe and the image is a cake. The recipe says how the environment was made, in text, in version control, reviewable line by line, and it will still say so in ten years. The image is an opaque binary that someone must keep hosting. It can be deleted, pruned, rate-limited, or rebuilt under the same tag, and nothing in it explains how it was produced. So commit the Dockerfile and treat the pushed image as a cache: convenient, worth having, and not the record.
A concrete failure from relying on the pushed image. The registry removes the image under a retention policy for inactive repositories. Alternatively, the organization account lapses, or the tag is overwritten by a later push. An analysis whose only environment record was that image is now unreconstructable. The failure appears at exactly the moment someone tries to reproduce the work, which is usually years after anyone was watching.
A second and more insidious failure merits mention because it produces no error. The image is rebuilt and pushed under the same tag with a newer base. A reader pulls the tag, gets a different environment from the one the analysis ran in, and obtains slightly different numbers with nothing to indicate anything moved. This is the argument for pinning by digest as well as committing the recipe.
A third arises if the collaborator works on Apple Silicon. The pushed image may have been built for one architecture only. The pull then succeeds on the author’s machine and fails, or runs under emulation with different numerical behavior, on someone else’s.
NoteExercise 3
What must change. The CI configuration file, from .github/workflows/*.yml to a single .gitlab-ci.yml at the root, in the dialect of stages and scripts rather than jobs and reusable actions. The remote URL and the authentication method. The registry, from GHCR to the GitLab container registry, and therefore the image reference in any pull. The secrets, which must be re-created as GitLab CI variables. Anything relying on a GitHub-specific feature: pull-request comment bots, GITHUB_TOKEN, Actions from the marketplace.
What does not change. Everything that constitutes the reproducibility of the compendium. The Dockerfile, the renv.lock, the directory structure, the DESCRIPTION, the tests, the analysis code, the data dictionary, and the checks themselves are all forge-agnostic. (Restricted data stay off both forges; moving forges does not change where they live.) The reproducibility machinery is independent of the hosting platform. A compendium that must be substantially rewritten to move forges was coupled to its forge in ways it should not have been.
Where the CI logic should live. In a Makefile, or an equivalent single entry point, so that both forges’ CI files reduce to a shim naming an image and calling make check, make check-renv, and the render target that matches the archetype (make docker-render for an .Rmd report; there is no plain make render). The migration then touches one thin file rather than the check logic, and the same commands run locally. That incidental benefit makes the arrangement worth adopting even for a project that will never move.
NoteExercise 4
The deposit and DOI will differ for each reader. The substance is the statement and its defense.
For a compendium built on simulated data, with a lockfile and a Dockerfile but no verification record:
The complete research compendium, comprising the analysis code, the simulated data and the script that generates them, the package lockfile, and the container recipe, is archived at https://doi.org/10.5281/zenodo.XXXXXXX. The data are synthetic and describe no real person. The computational environment is pinned by a digest-referenced container image and an renv.lock lockfile, so the analysis can be re-executed in the environment in which it was developed (L2 on the capture ladder used in the accompanying methods). We have not independently verified that re-execution regenerates the reported outputs, so no L3 claim is made.
For the registry case, the deposit loses the data and the statement gains an access route:
The analysis code, package lockfile, container recipe, and archived container image are archived at https://doi.org/10.5281/zenodo.XXXXXXX, with a synthetic dataset of the same structure on which the pipeline runs from end to end. The patient-level data were obtained from [registry] under a data-use agreement that does not permit redistribution; qualified researchers may apply to [data steward] for access. The synthetic data do not reproduce the reported estimates. The environment is pinned (L2); re-execution on the registry data has not been independently verified.
A common wrong answer keeps ‘the complete research compendium’ in the second statement. It is not complete, and saying so is part of the statement’s honesty.
Defending the claim against the honest-level standard means checking three things and being willing to have the answer come out lower than one would like. Is there a lockfile? Then L1. Is there, in addition, an artifact pinning the runtime and system libraries? Then L2. Is there evidence (a CI log or a verification report) that a baseline test passed within a stated tolerance after a clean rebuild? Only then L3. A recorded hash of an output file is not that evidence. A matching hash shows that the file has not changed, but L3 is defined by agreement with a committed baseline within a tolerance, which a hash cannot express.
The last clause of the statement above is the one that makes it honest, and it is the clause most authors omit. Omitting it invites the reader to assume verification that nobody performed.
The comparison typically shows most packages matching and a minority differing, with the differing set growing as the gap widens between the snapshot date and the dates on which packages were actually installed.
Why a date-based reconstruction may not match a lockfile exactly. Four reasons, and it is worth separating them.
First and most important, a lockfile records what was installed, and a date reconstruction records what was current on a date. These differ whenever a package was installed on one date and the lockfile written on another. That is the normal case for a project of any duration.
Second, packages installed from sources other than CRAN (from GitHub at a particular commit, or from Bioconductor) have no CRAN-dated equivalent. The reconstruction therefore either misses them or substitutes something.
Third, archived packages present a gap. A package removed from CRAN is recoverable from the archive by version, but its dependency resolution at a past date may not be.
Fourth, the reconstruction resolves the transitive closure afresh, so a dependency that has since changed its own requirements can pull a different set than the lockfile recorded.
The conclusion the chapter draws: reconstruction by date is what one falls back on when no lockfile exists, and it is an approximation rather than a substitute. A project with a lockfile should restore it.
NoteExercise 6
Taking each in turn; the reader needs only one.
R package. The sharing step beyond the deposit is publication to a package repository, CRAN or Bioconductor, or at minimum a tagged release installable with remotes::install_github(). The validation gate this adds is substantial and externally imposed. CRAN runs R CMD check across multiple platforms and R versions, including development R, and rejects the package on failure. This is a stronger gate than any the author configured, precisely because the author did not configure it. It is the closest thing in the R ecosystem to the independent verification of Chapter 12.
Blog. The step is deployment to a public URL with a feed, and registration where readers will find it. The gate added is thin: a deploy check confirming the site builds and links resolve. Notably, no gate checks that the code in the posts still runs. That is why an old post is the most reliably broken artifact in this book’s taxonomy.
Simulation study. The step is depositing the full results alongside the reduced pipeline, together with the seed and the replicate count, since the full run is too expensive to reproduce on demand. The gate added is the correspondence check: the reduced pipeline under the recorded seed must regenerate the leading replicates of the deposited full results. That holds only when a replicate’s draws are fixed by its index: a seed or random-number stream per replicate (the per-task streams of Chapter 11), or one serial stream that the reduced pipeline consumes in exactly the same order. A full run made in parallel on streams assigned per worker does not qualify, because the stream a replicate drew from depends on the worker count. The Monte Carlo standard error of each reported summary belongs in the deposit as well, so that a reader can tell a changed result from Monte Carlo noise. The check is what lets a reader verify the machinery without paying for the study.
NoteExercise 7
The solution for this exercise is in the instructor guide, which is distributed separately to course instructors. Self-study readers can compare their booleans with the named vector the chunk in ‘The cohort, continued’ prints; it is computed at render time, so it, not any typed value, is the key. In outline, the argument runs as follows. A rule that names cohort_raw.csv would report that file as ignored and say nothing about the next extract saved beside it, so only a file name that appears nowhere in .gitignore tests the protection real data depend on. When git check-ignore -v reports the directory pattern analysis/data/raw_data/* as the match for the invented name, any file saved there stays out of git. That is still not enough for real data: the .dockerignore check returns FALSE, and one that excludes the directory must exist before any Dockerfile copies the project into an image, because .gitignore does not govern a Docker build context.
NoteExercise 8
At docker build. The build context includes every file under the Dockerfile’s directory not excluded by .dockerignore. Because *.csv does not match extra.tsv, the file is copied into the image by any COPY . . or equivalent instruction. It thereby becomes part of the image’s file system layers.
At docker push. The image, .tsv included, is uploaded to the registry as-is. The registry has no knowledge of which files are data and which are code; it stores the layers it is given. The leak has now left the laboratory’s own machines.
At the Zenodo deposit.apptainer build exports the pushed image, .tsv and all, into the SIF file, which is then deposited and assigned a DOI. The leak is now public and, per this chapter’s own account of archival deposits, unrecallable.
What would have caught it. An inspection of the built image’s file system before the push. This is the check this chapter’s worked example performs with docker run --rm ... find / -xdev -path '*raw_data*' -type f, run against the actual data directory rather than assumed from the .dockerignore pattern. A pattern-matching ignore file is a plan; a listing of what the built image actually contains is a verification. Only the second one catches a file the plan did not anticipate. This is the Verification callout’s own rule, ‘how to see a claim actually holds,’ applied to the narrower claim that no data entered the image.
NoteExercise 9
Code-availability statement. A statement meeting 18f’s standard, as this book’s repository stands:
The analysis code for this compendium is available at [repository URL], at the commit tagged for this submission (not the default branch, which continues to change). The code is a CITATION.cff-bearing R research compendium; the code itself is released under GPL-3, the code printed in the accompanying book under MIT, and the prose and figures under CC BY 4.0. The deposit withholds nothing: the cohort analyzed is synthetic, generated by a committed, version-controlled script, and no restricted data were used.
The patient-level data analyzed were obtained from [registry] under a data-use agreement that does not permit redistribution. Qualified researchers may apply to [data steward] for access under the same agreement terms. A synthetic dataset of the same structure, on which the pipeline runs end to end, is deposited alongside the code and does not reproduce the reported estimates.
How the two differ. The code statement answers ‘can a reader obtain and run what produced these results,’ and its answer here is simply yes, with a version and a license. The data statement answers a harder question, ‘can a reader obtain what the code was run on,’ and in the restricted case the honest answer is a controlled route rather than a download. The two statements can therefore diverge sharply even for the same manuscript. This is why a reporting guideline that names them as separate checklist items is doing real work.
Reports the immutable digest to record, not the mutable tag
docker run --rm <img> find / -xdev -path '*raw_data*'
Confirms no restricted data were copied into the built image
apptainer build out.sif docker://<img>@sha256:...
Exports a digest-pinned image as a single archival file
renv.lock + Dockerfile, committed
The reproducibility guarantee; the pushed image is a cache, not the record
Two-layer model
The team image (operating system, R, system libraries, lab-wide packages) is lead-governed and slow; the project lockfile is contributor-governed and fast, under the union model
Union lockfile
Records every contributor’s packages cumulatively; never shrinks unless deliberately pruned; rebuild periodically, not on every install
rang::resolve(pkgs, snapshot_date = ...)
Reconstructs a period-correct environment from a date when no lockfile survives
Recorded run date
The single fact a retrospective recovery (rang) needs most
Bring code to the data; record the provider’s environment identifier; L3 only inside
CONSORT 2025 item 4, SPIRIT 2025 item 6
Say where the statistical code (and de-identified data) can be accessed
13.14 Further reading
Wilkinson and colleagues (Wilkinson et al., 2016) formulate the FAIR principles for research data; Barker and colleagues (Barker et al., 2022) adapt them for research software, whose reuse involves execution. Read both to understand what an archival deposit is trying to achieve, and why software reuse asks more than data reuse.
Brinckman and colleagues (Brinckman et al., 2019) describe the Whole Tale platform, which bundles code, data, and environment into a single executable research object. It is the most ambitious realization of the compendium-as-runnable-object idea.
The Turing Way (The Turing Way Community, 2022) is the community handbook for reproducible and collaborative research. For the social side of team collaboration, project governance, credit, and code review, in more depth than this chapter offers, it is the place to begin.
Chan and colleagues (C. Chan & Schoch, 2023) present rang, the retrospective recovery tool for decayed R environments. Read it together with the pinning of Chapter 6 to see prospective capture and retrospective recovery as two halves of stewardship.
Blischak and colleagues (Blischak et al., 2019) describe workflowr, which binds each rendered page of a versioned analysis website to its commit, the natural tool for the blog archetype’s deployment.
Stodden and colleagues (Stodden et al., 2018) provide the empirical case that a reproducibility policy is a weak predictor of an actually reproducible artifact, the evidence behind this book’s insistence on honest disclosure over asserted reproducibility.
The NIH Data Management and Sharing Policy (National Institutes of Health, 2020) and NIH’s principles for protecting participant privacy (National Institutes of Health, 2026c) set out what a funded project must plan for when its data come from human participants; read them before writing a data-availability statement for such a project.
13.15 In conclusion
The book’s argument comes down to four points. Reproducibility, we have maintained throughout, is not a badge a project either holds or lacks. It is a level a project reaches. The discipline is to reach as high as the work warrants and to say plainly how high that was.
First, reproducibility comes in degrees, and the ladder names them: locatable (L0), pinned packages (L1), pinned environment (L2), and verified (L3). Every chapter of this book has been an ascent of that ladder. We began by finding the source and pinning the packages with a lockfile. We then fixed the environment with a container or a content-addressed alternative, and finally regenerated the outputs and confirmed they match. The ladder is the common scale on which any analysis, built with any toolchain, can be placed and compared.
Second, the rungs are reached by two different kinds of work, and conflating them is the source of much confusion. Capture fixes the inputs and sets the level from L0 to L2. Pinning the packages and the environment is capture, which carries a compendium as far as L2. The last rung, L3, is reached not by capture but by an act of verification: a clean rebuild after which a baseline test passes within a stated tolerance. Running zzc verify once, with such a baseline test in place, is therefore what lifts a pinned L2 compendium to a verified L3 one. Automated re-checking is distinct from that act. Continuous integration reruns the verification on every change and certifies that the L3 state still holds, but it adds no rung of its own. Removing it does not lower the level; it only stops the automated confirmation. Keeping these registers distinct is what lets a project report, honestly and mechanically, exactly where it stands.
Third, the unit of reproducible work is the research compendium, and it is not only a manuscript. The same machinery, the same pillars, and the same ladder serve a data analysis, a simulation study, an R package, a blog, and a scholarly manuscript alike. The archetypes differ only in their directory layout, in whether a report is rendered, and in the last mile of publication: a package submission, a deployed website, or a deposit beside an article.
Fourth, and above all, the goal is reproducibility matched honestly to the level actually reached. The tools this book has taught, and the framework it has used as its running example, are strong but partial. They firmly secure some determinants and leave others to convention, and an honest practitioner discloses the difference. A verified L3 compendium and a locatable L0 script are both legitimate outputs of a scientific process; what is not legitimate is to present the second as the first. The reader who finishes this book with the judgment to name the level a given analysis has reached, across whatever archetype the work takes, and the honesty to claim no more, has learned the one thing the book most wished to teach.
The introduction named seven habits that recur across every chapter since. They are worth gathering here, once more, as the book’s last word rather than its first.
NoteSeven habits, and a motto
Claim only the level you have verified.
Commit before you compute.
Pin what you ran, not what is current.
Never commit, deposit, or paste restricted data.
Make every gate able to fail, and watch it fail once.
Compare against a recorded baseline, with a stated tolerance.
Rebuild from clean, on a machine that is not yours.
Write the honest level down.
45 CFR 164.514: Other requirements relating to uses and disclosures of protected health information. (n.d.). Retrieved October 11, 2026, from https://www.law.cornell.edu/cfr/text/45/164.514
Barker, M., Chue Hong, N. P., Katz, D. S., et al. (2022). Introducing the FAIR principles for research software. Scientific Data, 9, 622. https://doi.org/10.1038/s41597-022-01710-x
Blischak, J. D., Carbonetto, P., & Stephens, M. (2019). Creating and sharing reproducible research code the workflowr way. F1000Research, 8, 1749. https://doi.org/10.12688/f1000research.20843.1
Brinckman, A., Chard, K., Gaffney, N., et al. (2019). Computing environments for reproducibility: Capturing the WholeTale. Future Generation Computer Systems, 94, 854–867. https://doi.org/10.1016/j.future.2017.12.029
Chan, A.-W., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., et al. (2025). SPIRIT 2025 statement: Updated guideline for protocols of randomised trials. BMJ, 389, e081477. https://doi.org/10.1136/bmj-2024-081477
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., Smeden, M. van, Boulesteix, A.-L., Camaradou, J. C., Celi, L. A., Denaxas, S., Denniston, A. K., Glocker, B., Golub, R. M., Harvey, H., Heinze, G., … Moher, D. (2024). TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 385, e078378. https://doi.org/10.1136/bmj-2023-078378
European Parliament and Council of the European Union. (2025). Regulation (EU) 2025/327 of 11 february 2025 on the european health data space and amending directive 2011/24/EU and regulation (EU) 2024/2847. Official Journal of the European Union, L series. https://eur-lex.europa.eu/eli/reg/2025/327/oj/eng
Hopewell, S., Chan, A.-W., Collins, G. S., et al. (2025). CONSORT 2025 statement: Updated guideline for reporting randomised trials. PLOS Medicine, 22(4), e1004587. https://doi.org/10.1371/journal.pmed.1004587
Journal of the American Statistical Association. (n.d.). JASA reproducibility guide. Retrieved November 8, 2026, from https://jasa-acs.github.io/repro-guide/
National Center for Biotechnology Information. (n.d.). dbGaP: Database of genotypes and phenotypes. Retrieved October 11, 2026, from https://dbgap.ncbi.nlm.nih.gov/home
National Institutes of Health, All of Us Research Program. (n.d.-a). Data and statistics dissemination policy. All of Us Research Hub. Retrieved November 8, 2026, from https://www.researchallofus.org/?p=4698
Stodden, V., Seiler, J., & Ma, Z. (2018). An empirical analysis of journal policy effectiveness for computational reproducibility. Proceedings of the National Academy of Sciences, 115(11), 2584–2589. https://doi.org/10.1073/pnas.1708290115
The Turing Way Community. (2022). The TuringWay: A handbook for reproducible, ethical and collaborative research. Zenodo. https://doi.org/10.5281/zenodo.3233853
Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18