Specification v1.0

The ResourceRank Evaluation Framework

A deterministic, evidence-first methodology for assessing software, datasets, platforms, and digital resources without vendor pay-to-play bias.

1. Weighted Scoring Model

Every ResourceRank composite score ($S \in [0, 100]$) is computed as a weighted sum across four orthogonal evaluation dimensions. Each dimension must be backed by verifiable evidence artifacts.

25% Weight ID: provenance

Provenance & Lineage

Verifiable origin, authorship, licensing clarity, version history, and supply-chain integrity.

Required Evidence Indicators

  • Cryptographically signed releases or verifiable checksums
  • Explicit SPDX license identifiers and contributor attribution
  • Transparent changelog and historical versioning
30% Weight ID: evidence

Empirical Evidence & Reproducibility

Quantitative benchmarks, reproducible methodologies, independent audits, and peer-reviewed validation.

Required Evidence Indicators

  • Reproducible test suites or benchmark harnesses
  • Independent third-party security or quality audits
  • Documented methodology with raw data availability
25% Weight ID: reliability

Operational Reliability & Maintenance

Active stewardship, issue resolution velocity, architectural resilience, and backward compatibility.

Required Evidence Indicators

  • Sustained commit cadence and multi-maintainer governance
  • Automated CI/CD verification and regression coverage
  • Documented deprecation policies and SLA transparency
20% Weight ID: interoperability

Open Standards & Interoperability

Adherence to open protocols, portable data formats, non-proprietary APIs, and low exit friction.

Required Evidence Indicators

  • Support for open schemas (JSON-LD, Parquet, OpenAPI, etc.)
  • Complete data export without proprietary lock-in
  • Composable integration primitives

Classification Thresholds

92 /100 Exemplary

90–100: Meets all provenance, reproducibility, and open standard criteria with independent verification.

80 /100 Verified

75–89: Production-ready resource with documented lineage and minor governance or portability caveats.

65 /100 Provisional

60–74: Usable with explicit mitigations; incomplete audit trails or partial lock-in risks identified.

45 /100 Needs Review

0–59: Insufficient evidence, unclear licensing, or unverified provenance; not recommended for critical adoption.

Interactive Tool

Composite Score Simulator

Test how dimension scores combine under the ResourceRank v1.0 weighting specification.

90
85
88
92
Simulated Composite Score
88
Verified

(90×0.25) + (85×0.30) + (88×0.25) + (92×0.20) = 88.4

2. Provenance & Evidence Artifact Rules

Evidence Verifiability

Claims without verifiable artifacts are scored at zero for the corresponding indicator. Acceptable artifacts include:

  • Cryptographic hashes (SHA-256 / SHA-512) and Sigstore attestations
  • Persistent DOIs for benchmark papers and reproducibility packages
  • Publicly archived governance votes and SPDX license manifests
  • Reproducible containerized evaluation harnesses

Decision Record Linkage

Every evaluation should link to at least one ResourceRank Decision Record (RDR) documenting why a resource was selected, conditionally approved, or rejected for a concrete architectural or research context.

Inspect Decision Records