Skip to content

Commit a717da4

Browse files
committed
Restructure docs in deployment order and migrate to OKF v0.2
Convert the documentation bundle to the Open Knowledge Format v0.2 and reorganize it to follow the order in which CyVerse is actually deployed. Format (OKF v0.2) - Every concept document carries YAML frontmatter: type, title, description, tags, status, generated, and sources where the content is derived from something. - Every directory carries an index.md listing, with the bundle-root index declaring okf_version. - Bundle history is recorded in docs/log.md. - references/ holds provenance notes for derived material. Structure - architecture/ what the system is, what it needs, what depends on what - platform/ the products and services offered to users - deployment/ phases 1-7, ordered by dependency - operations/ running a deployed system - api/, development/, references/ - deployments/, database/, guides/, and services/ are retired; their content moved into the tree above. New content - deployment/from-scratch.md: end-to-end runbook for a two-node deployment, written from an anonymized pilot deployment record. All hostnames, zone names, DNs, realm and client names, and credentials are placeholders; secrets are generated per install. - Previously undocumented steps: HAProxy, iRODS provider and DE integration, iCAT, database migrations, cert-manager, Harbor, Argo, OpenLDAP, OpenSearch, NATS, bootstrap, verification, troubleshooting. - architecture/component-inventory.md and network-requirements.md. Editorial - Deduplicated: mail (exim4 + local-exim), the golang-migrate procedure repeated in every database document, the deployment-order list that appeared in three places, the database list that appeared twice, and two retired overview pages now covered by directory indexes. - Corrected: databases ordered before the services that need them; the rsyslog path; dirname vs basename for the kubeconfig directory; the helm uninstall invocation; VRRP as IP protocol 112 rather than a TCP port; QMS extensions created in the qms database; recomputed sizing totals with the source's 100/110 TB discrepancy noted. - Fixed the docs build: zensical.toml named a theme that is not installable, so the published build was failing, and extra_css named a stylesheet that does not exist. - Marked Elasticsearch deprecated in favor of OpenSearch, and ingress-nginx transitional in favor of Traefik. - Replaced site-specific hostnames, DNs, and sample credentials with placeholders; replaced XXX placeholders in the system overview. - Fixed 13 broken anchors and links; the build now reports no issues. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UKieNHmYbysTvGDK2SC1YM
1 parent 0b53ee0 commit a717da4

160 files changed

Lines changed: 7852 additions & 3915 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/about.md

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
---
2+
type: Reference
3+
title: "About this documentation"
4+
description: "What CyVerse is, who this documentation is for, and where each audience should start."
5+
tags: [about, orientation]
6+
status: stable
7+
generated: { by: process:okf-migration, at: 2026-07-29T00:00:00Z }
8+
sources:
9+
- id: okf-spec
10+
resource: https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md
11+
title: Open Knowledge Format v0.2 specification
12+
author: team:google-cloud-platform
13+
last_modified: 2026-07-29
14+
---
15+
16+
![](assets/cyverse_logo_2022.svg)
17+
18+
# What CyVerse is
19+
20+
CyVerse is a computational infrastructure for data-intensive science, and the
21+
people who operate it. It is fully open source and funded by the
22+
[United States National Science Foundation](https://www.nsf.gov/){target=_blank}.
23+
24+
It is both a Software as a Service platform and the Infrastructure as Code needed
25+
to run one: the same stack that serves the public US deployment can be deployed by
26+
another institution on its own hardware or in the cloud. That is what this
27+
documentation is for.
28+
29+
<figure markdown>
30+
![layercake](assets/layerCake.svg){width=800}
31+
<figcaption>Hardware at the bottom, services in the middle, products on top</figcaption>
32+
</figure>
33+
34+
# Who this is for
35+
36+
## Deploying CyVerse
37+
38+
Start with [prerequisites](deployment/planning/prerequisites.md), then read
39+
[deploying from scratch](deployment/from-scratch.md) end to end before running
40+
anything. Work the phases in [deployment](deployment/) in order, and check each one
41+
against [verification](deployment/07-post-install/verification.md).
42+
43+
Before provisioning: [component inventory and
44+
sizing](architecture/component-inventory.md) and [network
45+
requirements](architecture/network-requirements.md).
46+
47+
## Operating a deployment
48+
49+
[operations/](operations/) covers day-to-day administration: users and VICE access
50+
in [DE administration](operations/discovery-environment.md), data and curation in
51+
[Data Store administration](operations/data-store.md), accounts in [User Portal
52+
administration](operations/user-portal.md), and the recurring questions in the
53+
[FAQ](operations/faq.md).
54+
55+
## Integrating with the APIs
56+
57+
[Terrain](api/terrain.md) is the API behind every CyVerse product. The
58+
[endpoint index](api/endpoint-index.md) lists everything documented here, and the
59+
live [Swagger reference](https://de.cyverse.org/terrain/docs/){target=_blank} is
60+
the most current source. Authentication is [OAuth 2.0 through
61+
Keycloak](platform/authentication.md).
62+
63+
## Contributing code
64+
65+
[development/](development/) covers the development environment and contribution
66+
workflow. Source lives in the
67+
[CyVerse](https://github.com/cyverse){target=_blank} and
68+
[CyVerse DE](https://github.com/cyverse-de){target=_blank} GitHub organizations.
69+
70+
# What CyVerse offers its users
71+
72+
| Product | What it does |
73+
|---------|--------------|
74+
| [Discovery Environment](platform/discovery-environment.md) | Web-based data science workbench with hundreds of integrated tools |
75+
| [Data Store](platform/data-store.md) | Multi-petabyte iRODS storage with HTTPS, WebDAV, SFTP, and API access |
76+
| [Data Commons](platform/data-commons.md) | Publishing curated and community-released datasets, with DataCite DOIs |
77+
| VICE | Interactive computing — JupyterLab, RStudio, Shiny — inside the DE |
78+
| [Cloud services (CACAO)](platform/cloud.md) | Infrastructure as code for multi-cloud deployments |
79+
| [BisQue](platform/bisque.md) | Bio-image semantic query and analysis |
80+
| [DNA Subway](platform/dna-subway.md) | Educational genomics workflows |
81+
82+
# How this documentation is organized
83+
84+
This bundle follows the [Open Knowledge Format](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)
85+
v0.2.[^okf-spec] In practice that means three things you can rely on:
86+
87+
* **Every document declares itself.** Frontmatter carries its `type`, a one-line
88+
`description`, `tags`, and a `status` of `draft`, `stable`, or `deprecated`. A
89+
`draft` document is incomplete and says so rather than pretending otherwise.
90+
* **Every directory has an index.** `index.md` lists what is in a directory with a
91+
line of description each, so you can see what exists before opening anything.
92+
* **Derived documents cite their sources.** Where a document was written from
93+
something else, `sources` in its frontmatter says what, including material
94+
mirrored under [references/](references/).
95+
96+
Changes to the bundle are recorded in [the log](log.md).
97+
98+
# Links
99+
100+
* :material-web: [CyVerse website](https://cyverse.org){target=_blank}
101+
* :material-frequently-asked-questions: [FAQ](operations/faq.md)
102+
* :simple-github: [GitHub organization](https://github.com/cyverse-de){target=_blank}
103+
* :material-api: [Live Terrain API](https://de.cyverse.org/terrain/docs/){target=_blank}
104+
* :simple-docker: [Harbor registry](https://harbor.cyverse.org/){target=_blank}
105+
* :material-school: [User-facing learning materials](https://learning.cyverse.org/){target=_blank}
106+
107+
# Funding
108+
109+
[![nsf](assets/NSF.svg){width=100}](https://www.nsf.gov/){target=_blank}
110+
111+
CyVerse has been funded by the National Science Foundation from 2008 to the
112+
present.
113+
114+
[![NSF-0735191](https://img.shields.io/badge/NSF-0735191-blue.svg)](https://www.nsf.gov/awardsearch/showAward?AWD_ID=0735191) [![NSF-1265383](https://img.shields.io/badge/NSF-1265383-blue.svg)](https://www.nsf.gov/awardsearch/showAward?AWD_ID=1265383) [![NSF-1743442](https://img.shields.io/badge/NSF-1743442-blue.svg)](https://www.nsf.gov/awardsearch/showAward?AWD_ID=1743442)
115+
116+
!!! Info ":fontawesome-brands-creative-commons-by: SOFTWARE LICENSE"
117+
118+
Copyright (c) 2010-2026, The Arizona Board of Regents on behalf of The University of Arizona
119+
120+
All rights reserved.
121+
122+
Developed by: CyVerse as a collaboration between participants at BIO5 at The University of Arizona (the primary hosting institution), Cold Spring Harbor Laboratory, The University of Texas at Austin, and individual contributors. Find out more at http://www.cyverse.org/.
123+
124+
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
125+
126+
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
127+
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
128+
* Neither the name of CyVerse, BIO5, The University of Arizona, Cold Spring Harbor Laboratory, The University of Texas at Austin, nor the names of other contributors may be used to endorse or promote products derived from this software without specific prior written permission.
129+
130+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
131+
132+
[^okf-spec]: Open Knowledge Format v0.2 specification

0 commit comments

Comments
 (0)