Schemas

Language-independent JSON Schema definitions for rtemis. The same config drives rtemis (R), rtemis-py and rtemislive — these documents are the contract between them.

213 schemasschema.rtemis.org index.json

Machine learning

175

Start here — configs you author

ClassificationMetricsResclassificationmetricsres

Classification metrics aggregated across resamples: each resample's metrics, the aggregate confusion matrix in long form, and the mean and standard deviation of the overall metrics.

ClusterConfigcluster

Language-independent config for an rtemis clustering pipeline: a data reference, a `ClusteringConfig`, and an output directory.

ConformalConfigconformal

Language-independent config for an rtemis conformal prediction region. Mirrors the `ConformalConfig` object: a construction plus its construction-specific settings. The construction is tagged rather than inferred, since which of them a run may use depends on the model it is applied to, not on the document. `type` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

ConformalConfigconformal

Language-independent config for an rtemis conformal prediction region. Mirrors the `ConformalConfig` object: a construction plus its construction-specific settings. The construction is tagged rather than inferred, since which of them a run may use depends on the model it is applied to, not on the document. `type` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `type` is that variant with every default.

DecomposeConfigdecompose

Language-independent config for an rtemis decomposition pipeline: a data reference, a `DecompositionConfig`, and an output directory.

Diagnosticsdiagnostics

The findings for one rtemis config, in the order they were made. An empty array means the config is clean: there is no separate validity flag, an empty list of problems being the same statement.

ExplanationConfigexplanation

Language-independent config for a per-case rtemis explanation. Mirrors the `ExplanationConfig` object: a kind of explanation plus its kind-specific settings. The kind is tagged rather than inferred, so a second kind of explanation can be added without changing how the first is read. `type` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

ExplanationConfigexplanation

Language-independent config for a per-case rtemis explanation. Mirrors the `ExplanationConfig` object: a kind of explanation plus its kind-specific settings. The kind is tagged rather than inferred, so a second kind of explanation can be added without changing how the first is read. `type` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `type` is that variant with every default.

IngestConfigingest

Language-independent config for reading a data file and normalizing it to Parquet. A delimited file or a spreadsheet carries no usable type information and Parquet does, so this is the one step that decides types -- everything after it reads a declaration rather than inferring one. `format` is what the file is, not a preference, so it has no default and a config that disagrees with its file is an error. `format` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

IngestConfigingest

Language-independent config for reading a data file and normalizing it to Parquet. A delimited file or a spreadsheet carries no usable type information and Parquet does, so this is the one step that decides types -- everything after it reads a declaration rather than inferring one. `format` is what the file is, not a preference, so it has no default and a config that disagrees with its file is an error. `format` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `format` is that variant with every default.

PartitionConfigpartition

Language-independent config for splitting a dataset into a training set and a held-out test set. A first-class, auditable operation -- the same reason `ingest` is one -- rather than a field on `SuperConfig`: how a held-out set was produced is a decision the record must be able to report. `method` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

PartitionConfigpartition

Language-independent config for splitting a dataset into a training set and a held-out test set. A first-class, auditable operation -- the same reason `ingest` is one -- rather than a field on `SuperConfig`: how a held-out set was produced is a decision the record must be able to report. `method` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `method` is that variant with every default.

PreprocessorConfigpreprocessor

Language-independent config for rtemis preprocessing. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive to identical output.

DataProfileprofile

What one dataset is, in the facts a validator needs: dimensions, columns with their types, distinct and missing counts, level counts for low-cardinality categorical columns, and complete-case and duplicate counts. Bounded by the number of columns rather than the number of rows, so it travels where the data cannot.

RegressionMetricsResregressionmetricsres

Regression metrics aggregated across resamples: each resample's metrics, plus their mean and standard deviation.

SuperConfigsupervised

Language-independent config for an rtemis supervised-learning run. Mirrors the `SuperConfig` object: data references, optional preprocessing / decomposition, an algorithm with hyperparameters, optional tuning and outer resampling, and execution settings. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive.

Components — composed into the configs above

ClassificationMetricsclassificationmetrics

Classification metrics for one sample: the confusion matrix in long form (one row per cell), overall metrics, and per-class metrics. Which overall columns are present depends on the task, so only the invariant ones are required.

ClusterConfigcluster

Language-independent config for an rtemis clustering pipeline: a data reference, a `ClusteringConfig`, and an output directory.

ClusteringConfigclustering

Language-independent config for an rtemis clustering run. Mirrors the `ClusteringConfig` object: an algorithm name and its settings. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive to identical output. `algorithm` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

selected by algorithm

cmeansdbscangmmhardclhopachkmeansneuralgaspampamkspectral

ClusteringConfigclustering

Language-independent config for an rtemis clustering run. Mirrors the `ClusteringConfig` object: an algorithm name and its settings. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive to identical output. `algorithm` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `algorithm` is that variant with every default.

selected by algorithm

cmeansdbscangmmhardclhopachkmeansneuralgaspampamkspectral

DataFingerprintdatafingerprint

Identity of one dataset: a content hash plus the structural facts that make a mismatch diagnosable rather than merely detectable.

DataRefdataref

A reference from a record to a file written beside it: where it is, how it is encoded, how big it is, and the digest that ties it to the record naming it.

DecomposeConfigdecompose

Language-independent config for an rtemis decomposition pipeline: a data reference, a `DecompositionConfig`, and an output directory.

DecompositionConfigdecomposition

Language-independent config for an rtemis decomposition (dimensionality reduction). Mirrors the `DecompositionConfig` object: an algorithm name, its settings, and an optional feature subset. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive to identical output. `algorithm` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

selected by algorithm

icaisomapnmfpcatsneumap

DecompositionConfigdecomposition

Language-independent config for an rtemis decomposition (dimensionality reduction). Mirrors the `DecompositionConfig` object: an algorithm name, its settings, and an optional feature subset. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive to identical output. `algorithm` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `algorithm` is that variant with every default.

selected by algorithm

icaisomapnmfpcatsneumap

DecompositionMetricsdecompositionmetrics

Decomposition metrics: a single-row table of reconstruction quality, component redundancy and effective dimensionality. Which cells are populated follows from the algorithm's traits -- a metric its algorithm cannot support is null -- so every column is declared for every algorithm and every cell is nullable. Unprefixed columns describe the data the decomposition was fitted on; `oos_` columns describe data the fit never saw.

Diagnosticdiagnostic

One finding from validating an rtemis config: a stable code, how much it matters, the technical and plain-language accounts of it, the numbers behind it, and -- where a deterministic one exists -- an RFC 6902 JSON Patch that fixes it.

ExecutionConfigexecution

Language-independent config for rtemis execution: sequential, parallel, or distributed. `backend` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

selected by backend

futuremirainone

ExecutionConfigexecution

Language-independent config for rtemis execution: sequential, parallel, or distributed. `backend` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `backend` is that variant with every default.

selected by backend

futuremirainone

PreprocessorConfigpreprocessor

Language-independent config for rtemis preprocessing. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive to identical output.

Provenanceprovenance

What produced a run record: package and language versions, platform, timing, how the run ended, and a fingerprint of each dataset used. Referenced by every `<family>/v1/record.json`.

RegressionMetricsregressionmetrics

Regression metrics for one sample: a single-row table of mean absolute error, mean squared error, root mean squared error and R-squared. Every cell is nullable, a metric being genuinely undefined for some samples.

ResamplerConfigresampler

Language-independent config for an rtemis resampler. Mirrors the `ResamplerConfig` object: a resampler type plus its type-specific settings. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive to identical resamples. `type` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

selected by type

bootstrapcustomkfoldloocvstratbootstratsub

ResamplerConfigresampler

Language-independent config for an rtemis resampler. Mirrors the `ResamplerConfig` object: a resampler type plus its type-specific settings. The same config drives rtemis (R), rtemis CLI/shell, and rtemislive to identical resamples. `type` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `type` is that variant with every default.

selected by type

bootstrapcustomkfoldloocvstratbootstratsub

SuperConfigsupervised

SupervisedPreprocessorConfigsupervisedpreprocessor

Language-independent config for the preprocessing a supervised run can fit: the preprocessing config without the operations a fitted preprocessor cannot replay at predict time (`complete_cases`, `remove_duplicates`, `remove_cases_thres`) or would learn differently in every resample (`remove_features_thres`). Those belong to `preprocessor`, applied to a dataset before training.

SupervisedPreprocessorConfigsupervisedpreprocessor

TunerConfigtuner

Language-independent config for rtemis hyperparameter tuning. Mirrors the `TunerConfig` object: a tuner type and its settings. `type` selects the variant; its resolved settings are siblings of it, validated against the variant's own record schema below.

selected by type

gridsearch

TunerConfigtuner

Language-independent config for rtemis hyperparameter tuning. Mirrors the `TunerConfig` object: a tuner type and its settings. `type` selects the variant and its settings are siblings of it, validated against the variant's own schema below. Anything not set takes rtemis's default, so an object holding only `type` is that variant with every default.

selected by type

gridsearch

Composition

How the schemas above assemble — every reference is a link

IngestConfigingest/v1/record.json

formatstring

columnsobject | null

character2factorboolean

clean_colnamesboolean

remove_duplicatesboolean

IngestConfigingest/v1

formatstring

columnsobject | null

character2factorboolean

clean_colnamesboolean

remove_duplicatesboolean

SuperConfigsupervised/v1

outcomestring | null

featuresarray | null

weightsstring | null

positive_classstring | null

preprocessor_config

SupervisedPreprocessorConfig

decomposition_config

DecompositionConfig

algorithm:PCA·ICA·NMF·UMAP·tSNE·Isomap

hyperparametersany

tuner_config

TunerConfig

type:GridSearch

outer_resampling_config

ResamplerConfig

type:KFold·StratSub·StratBoot·Bootstrap·LOOCV·Custom

execution_config

ExecutionConfig

backend:none·future·mirai

questionstring | null

outdirstring | null

verbosityinteger

dat_training_pathstring | null

dat_validation_pathstring | null

dat_test_pathstring | null

character2factorboolean

PartitionConfigpartition/v1/record.json

methodstring

PartitionConfigpartition/v1

methodstring

ConformalConfigconformal/v1/record.json

typestring

ConformalConfigconformal/v1

typestring

ClassificationMetricsResclassificationmetricsres/v1

samplestring | null

confusion_longarray | null

res_metricsClassificationMetrics[]

mean_metricsarray | null

sd_metricsarray | null

ClusterConfigcluster/v1

dat_pathstring | null

algorithmstring | null

clustering_config

ClusteringConfig

algorithm:KMeans·HardCL·NeuralGas·CMeans·DBSCAN·GMM·HOPACH·PAM·PAMK·Spectral

outdirstring

verbosityinteger

DecomposeConfigdecompose/v1

dat_pathstring | null

algorithmstring | null

decomposition_config

DecompositionConfig

algorithm:PCA·ICA·NMF·UMAP·tSNE·Isomap

outdirstring

verbosityinteger

Diagnosticsdiagnostics/v1

diagnosticsDiagnostic[]

ExplanationConfigexplanation/v1/record.json

typestring

ExplanationConfigexplanation/v1

typestring

DataProfileprofile/v1

n_rowsinteger

columnsarray | null

level_countsarray | null

n_complete_casesinteger

n_duplicatesinteger | null

fingerprint

DataFingerprint

RegressionMetricsResregressionmetricsres/v1

samplestring | null

res_metricsRegressionMetrics[]

mean_metricsarray | null

sd_metricsarray | null

PreprocessorConfigpreprocessor/v1

complete_casesboolean

remove_features_thresnumber | null

remove_cases_thresnumber | null

missingnessboolean

imputeboolean

impute_typestring

impute_missRanger_paramsobject

impute_discretestring

impute_continuousstring

integer2factorboolean

integer2numericboolean

logical2factorboolean

logical2numericboolean

numeric2factorboolean

numeric2factor_levelsarray | null

numeric_cut_ninteger

numeric_cut_labelsboolean

numeric_quant_ninteger

numeric_quant_NAonlyboolean

unique_len2factorinteger

character2factorboolean

factorNA2missingboolean

factorNA2missing_levelstring

factor2integerboolean

factor2integer_startat0boolean

factor2integer_levelsobject | null

scaleboolean

centerboolean

scale_centersobject | null

scale_coefficientsobject | null

remove_constantsboolean

remove_constants_skip_missingboolean

remove_duplicatesboolean

remove_featuresarray | null

one_hotboolean

one_hot_levelsobject | null

add_date_featuresboolean

date_featuresstring[]

add_holidaysboolean

holidaysstring[]

excludearray | null

Harness

Formats

harness artifactartifact

One immutable version of any rtemis harness artifact, routed to its shape by `type`. `name` and `version` say which revision of which artifact; `id` is a BLAKE3 digest over the RFC 8785 canonical form of `content`, so it says whether two versions are the same document.

Domain formats

Formats

A3a3

Amino Acid Annotation (A3) format — structured annotation of amino acid sequences.

chartchart

Language-independent config for an rtemis chart. A chart type plus its type-specific data binding, semantics and appearance. The same config drives rtemis.draw (R) and rtemislive (TypeScript) to the same chart.

chartchart

ClusteringMetricsclusteringmetrics

Clustering metrics: a single-row table of measure values beside a row of statuses saying why each holds what it does. Every measure is declared for every algorithm, so one an algorithm cannot support is null with a status of "unsupported" rather than an absent column -- null alone cannot distinguish a measure that was computed and came out undefined from one nobody asked for.