# Storage Plan

Use a self-hosted-first distribution model.

## 1. data.luxnode.org

Domain: `data.luxnode.org`

Use this as the canonical public download surface. It should expose:

- `https://data.luxnode.org/registry.json`
- `https://data.luxnode.org/sources.json`
- `https://data.luxnode.org/datasets.json`
- `https://data.luxnode.org/download/<dataset>/<version>/<file>`

The first production implementation can run the zero-dependency
`lux-data-server` process against a local data root. When files grow, keep the
same URL interface and move the bytes behind it to Cloudflare R2, S3, or a
Railway volume.

## 2. Local warehouse on this computer

Use a real local warehouse path for raw and normalized files:

```text
/Users/luismantilla/LuxDataWarehouse/
  raw/
  normalized/
  public/
  checksums/
```

The repo should be able to rebuild `public/` from `raw/` plus import scripts.

## 3. GitHub repository

Repository: `su3-labs/lux-data`

Use GitHub for:

- import/build code
- source manifests
- schemas
- license review notes
- tiny CC0 seed data
- release tags

Do not commit large measured spectra directly to Git history.

## 4. Hugging Face Datasets

Dataset: `su3-labs/lux-data`

Use Hugging Face as an optional mirror for:

- normalized JSONL/Parquet artifacts
- sharded train/reference/test splits
- `datasets`-compatible loading
- dataset cards and discoverability
- `hf://datasets/su3-labs/lux-data/...` local downloads

Recommended layout:

```text
lux-computed-minerals/v0.1.0/spectra.jsonl
usgs-splib07/v0.1.0/spectra.parquet
usgs-splib07-luxnode-c12880ma/v0.1.0/spectra.parquet
indexes/v0.1.0/mineral_sam_index.npz
```

## 5. Zenodo

Create a Zenodo concept DOI for each public release. Zenodo should archive the
exact Hugging Face artifact manifest, checksums, schema, and source citations.

## Optional operational mirror

Use Cloudflare R2 or S3 for Lux Cloud if low-latency worker access becomes
important. Treat it as a mirror, not the canonical open-data registry.
