Methodology

How signals are discovered

An hourly pipeline collects new items from public sources through their official APIs or published feeds. It does not scrape pages against their terms. Each source is an adapter that can be added or removed without changing the rest of the system.

A cluster of events becomes a candidate signal when it appears in at least 2 distinct sources, or a measured series reaches 2× its baseline with at least 20 events in the latest window, or a repository gains 100+ stars in a window, or discussion reaches 150+ points. Everything else is stored as raw events and re-evaluated on later runs.

The pipeline

The frontend never polls sources. It reads stored results, which are cached and invalidated when a pipeline run writes new data. Each run is recorded with per-source counts and errors; one failing source never blocks the others.

How duplicates are removed

Every normalised event carries a deduplication key (source plus the source’s own identifier, plus the day for snapshot sources). Keys already stored are skipped. Within a batch, events that share a canonical URL are collapsed: tracking parameters, fragments, www. and arXiv version suffixes are stripped first.

Entity resolution then joins events that refer to the same thing. A Hacker News thread linking to a GitHub repository, an npm package whose repository field points to it, and an arXiv abstract that mentions its code URL resolve to one entity through shared references (union-find over github:, npm:, hf: and arxiv: identifiers).

How momentum is calculated

Momentum compares a signal with its own recent past, not with other signals. Activity is grouped into rolling 7-day windows (up to 8). Cumulative counts such as total stars are differenced into gains per window; a window with no earlier observation is left out rather than estimated.

momentum ratio = latest window ÷ mean(previous 4 windows)

points = min(1, log₂(max(ratio, 1)) ÷ log₂(8)) × 10
       × min(1, latest window ÷ 20)      ← low-volume dampening

With fewer than three windows, or a baseline of zero, momentum is reported as Insufficient evidence and scores zero.

How the Breakout Score works

Nine factors sum to 100, then the hype penalty is subtracted and the result is clamped to 0–100. Bands: breakout candidate ≥ 80, strong ≥ 65, worth watching ≥ 50.

Analyst rubric levels: 0 None · 1 Low · 2 Moderate · 3 High · 4 Exceptional. Automatically detected signals receive conservative heuristic levels that are labelled as such, and technical significance is left unassessed until a person reviews it — so unreviewed signals rank lower by design.

The score is not asked of a language model. It is a deterministic function of stored inputs, so the same inputs always give the same number, and every signal page shows the full breakdown.

How evidence is weighted

Each source is classified by tier and by whether its publisher is independent of the project.

primary

1.00

secondary

0.70

community

0.35

Primary sources (release notes, repositories, papers) say what changed. Secondary sources (independent reporting, third-party benchmarks, integrations by other projects) confirm it. Community sources show attention, not correctness, so they count half towards cross-source confirmation. Placeholder URLs exist only in the labelled demo dataset, and are never emitted as citations in structured data.

How hype is penalised

• Marketing language: −2 per superlative in source headlines, up to −6. Terms: “revolutionary”, “revolutionize”, “breakthrough”, “game-changer”, “game-changing”, “game changer”, “paradigm shift”, “unprecedented”, “insane”, “mind-blowing”, “killer”, “disrupt”, “10x”, “100x”, “agi”.
• Self-reported benchmarks only: −4 when benchmarks exist but none is independent.
• Attention outpacing evidence: −5 when community sources are at least 2× the substantive ones and evidence strength is below 5/10.
• Analyst flags: −1 to −3 each, always with a written reason shown in the breakdown.

The total penalty is capped at −15. A hype penalty also lowers confidence.

The AI contextual layer

When an AI key is configured, the highest-ranked new signals are read by a language model (by default openai/gpt-oss-20b on Groq; any OpenAI-compatible endpoint works). The model sees only text from the signal’s own sources — the repository README, model card, package readme or paper abstract — and returns structured JSON: whether the item is relevant to developers, a plain title and summary, claims, limitations, maturity and rubric suggestions.

• Every claim must carry a quote. Code checks that the quote appears verbatim in the cited source; claims that fail are discarded, not softened.
• Kept claims are shown as facts with their quote and an AI marker; judgements (who is affected, what to do) are shown as analysis.
• Rubric suggestions apply only with a verified quote, are capped at “high”, and never override a human reviewer.
• Items the model classifies as spam, off-topic or not useful to developers are removed from the feed.
• The model is never asked for the Breakout Score. The score remains a deterministic function of the stored inputs.
• Results are cached per signal for a week, and a per-run call budget keeps usage within free-tier limits.

Human review and moderation

Approved reviewers can hide a signal from public view, and restore it, with a written reason. Reasons and reviewer names are published in the moderation log. Reviewer access is granted by application: a structured case, including a sample review of a live signal, read and approved by the site owner. About the reviewer programme.

How uncertainty is represented

Every statement on a signal page carries one of three labels:

Fact

Directly supported by a cited source.

Analysis

Astrava’s interpretation of the evidence.

Estimate

A forward-looking judgement with uncertainty.

A statement cannot be stored as a fact without a source; the data layer rejects it. Where information is missing — no benchmark, no maintainer count, no architecture — the page says Insufficient evidence instead of filling the gap.

confidence = 0.3 × share of measured factors with data
           + 0.3 × min(independent non-community publishers, 3) ÷ 3
           + 0.2 × primary source present
           + 0.2 × evidence strength ÷ 10
           − 0.2 × |hype penalty| ÷ 15

high ≥ 0.75 · medium ≥ 0.50 · otherwise low
capped at low when no independent, non-community source exists

Implementation difficulty and time to impact are estimates for a typical application team. They are labelled as estimates wherever they appear.

Known limitations

• Coverage is limited to sources with public APIs or feeds; work discussed only in private channels is invisible.
• Stars, downloads and points can be gamed. Momentum dampening and the adoption factor’s reliance on integrations reduce, but do not remove, this risk.
• Three factors are human judgements. They are labelled, and their rationale is shown, but they are still judgements.
• A new project has no history, so its momentum is “insufficient evidence” for its first weeks.
• Rankings compare signals with each other; they say nothing about absolute quality.

Questions about a specific score? Every signal page links each point to its inputs. See also About Astrava.