Skip to main content

Interface: IStorageBackend

Defined in: src/types/interfaces/storage.interface.ts:379

Storage backend interface for Chive's local index.

Remarks​

This interface stores indexes, not source data. All data can be rebuilt from the AT Protocol firehose.

Implementation uses PostgreSQL for relational queries, JSONB columns for flexible schema, partitioning for scalability, and indexes on uri, author, and createdAt.

Methods​

countEprintsByAuthor()​

countEprintsByAuthor(author): Promise<number>

Defined in: src/types/interfaces/storage.interface.ts:526

Counts total eprints by author.

Parameters​

author​

DID

Author DID

Returns​

Promise<number>

Total count of eprints by this author

Remarks​

Returns the total count without fetching full eprint data. Used for displaying metrics in author profiles.

Example​

const count = await storage.countEprintsByAuthor(toDID('did:plc:abc')!);
console.log(`Author has ${count} eprints`);

countEprintsByAuthors()​

countEprintsByAuthors(authors): Promise<Map<string, number>>

Defined in: src/types/interfaces/storage.interface.ts:544

Counts eprints for several authors in one query.

Parameters​

authors​

readonly DID[]

Author DIDs

Returns​

Promise<Map<string, number>>

A map from DID to eprint count, omitting authors with none

Remarks​

For paths that need a count per author across a page of authors, where a call per author would be a round trip per author.

Example​

const counts = await storage.countEprintsByAuthors([didA, didB]);
console.log(counts.get(didA) ?? 0);

deleteByUri()​

deleteByUri(table, uri): Promise<void>

Defined in: src/types/interfaces/storage.interface.ts:925

Deletes a record from an index table by AT-URI.

Parameters​

table​

string

uri​

AtUri

Returns​

Promise<void>

deleteChangelog()​

deleteChangelog(uri): Promise<Result<void, Error>>

Defined in: src/types/interfaces/storage.interface.ts:846

Deletes a changelog from the index.

Parameters​

uri​

AtUri

AT URI of the changelog to delete

Returns​

Promise<Result<void, Error>>

Result indicating success or failure

Remarks​

Called when firehose receives a deletion event for a changelog.

deleteCitation()​

deleteCitation(userRecordUri): Promise<void>

Defined in: src/types/interfaces/storage.interface.ts:979

Deletes a user-provided citation by its record URI.

Parameters​

userRecordUri​

AtUri

AT URI of the user citation record

Returns​

Promise<void>

void

deleteEprint()​

deleteEprint(uri): Promise<Result<void, Error>>

Defined in: src/types/interfaces/storage.interface.ts:748

Deletes an eprint from the index.

Parameters​

uri​

AtUri

AT URI of the eprint to delete

Returns​

Promise<Result<void, Error>>

Result indicating success or failure

Remarks​

Removes the eprint from the local index. Does not delete from PDS. ATProto compliance: Chive never writes to user PDSes.

Called when firehose receives a deletion event for an eprint.

Example​

const result = await storage.deleteEprint(
  toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!
);

if (!result.ok) {
  console.error('Failed to delete:', result.error);
}

deleteRelatedWork()​

deleteRelatedWork(uri): Promise<void>

Defined in: src/types/interfaces/storage.interface.ts:1033

Deletes a related work record by its URI.

Parameters​

uri​

AtUri

AT URI of the related work record

Returns​

Promise<void>

void

findByExternalIds()​

findByExternalIds(externalIds): Promise<null | StoredEprint>

Defined in: src/types/interfaces/storage.interface.ts:626

Finds an eprint by external identifiers.

Parameters​

externalIds​

External service identifiers to search

arxivId​

string

dblpId​

string

doi​

string

openAlexId​

string

openReviewId​

string

pmid​

string

semanticScholarId​

string

ssrnId​

string

Returns​

Promise<null | StoredEprint>

First matching eprint or null

Remarks​

Searches the eprints index by DOI and external IDs stored in the external_ids JSONB column. Returns the first match found.

Search priority:

  1. DOI (most authoritative)
  2. arXiv ID
  3. Semantic Scholar ID
  4. OpenAlex ID
  5. DBLP ID
  6. OpenReview ID
  7. PubMed ID
  8. SSRN ID

Example​

const eprint = await storage.findByExternalIds({
  doi: '10.1234/example',
  arxivId: '2301.12345',
});

if (eprint) {
  console.log('Found duplicate:', eprint.uri);
}

getChangelog()​

getChangelog(uri): Promise<null | StoredChangelog>

Defined in: src/types/interfaces/storage.interface.ts:773

Retrieves a single changelog entry by URI.

Parameters​

uri​

AtUri

AT URI of the changelog record

Returns​

Promise<null | StoredChangelog>

Changelog view or null if not found

Remarks​

Changelogs are indexed from the firehose and describe changes between eprint versions.

Example​

const changelog = await storage.getChangelog(
  toAtUri('at://did:plc:abc/pub.chive.eprint.changelog/xyz')!
);

if (changelog) {
  console.log('Version:', changelog.version);
}

getCitationsForEprint()​

getCitationsForEprint(eprintUri, options?): Promise<CitationListResult>

Defined in: src/types/interfaces/storage.interface.ts:952

Retrieves citations for an eprint.

Parameters​

eprintUri​

AtUri

AT URI of the eprint

options?​

EprintCitationQueryOptions

Query options (limit, offset, source filter)

Returns​

Promise<CitationListResult>

Paginated list of citations

Remarks​

Returns both auto-extracted and user-provided citations. Filter by source to get only one type.

Example​

const result = await storage.getCitationsForEprint(
  toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!,
  { limit: 50, source: 'all' }
);

for (const citation of result.citations) {
  console.log(citation.title, citation.source);
}

getEprint()​

getEprint(uri): Promise<null | StoredEprint>

Defined in: src/types/interfaces/storage.interface.ts:445

Retrieves an eprint index record by URI.

Parameters​

uri​

AtUri

AT URI of the eprint

Returns​

Promise<null | StoredEprint>

Eprint if indexed, null otherwise

Remarks​

Returns null if the eprint has not been indexed by Chive. The eprint may still exist in the user's PDS.

Example​

const eprint = await storage.getEprint(
  toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!
);

if (eprint) {
  console.log('Title:', eprint.title);
}

getEprints()​

getEprints(uris): Promise<Map<AtUri, StoredEprint>>

Defined in: src/types/interfaces/storage.interface.ts:457

Fetches several eprints in a single query.

Parameters​

uris​

readonly AtUri[]

Eprint URIs to fetch

Returns​

Promise<Map<AtUri, StoredEprint>>

Map of URI to eprint, omitting URIs with no row

Remarks​

Exists so callers handling a page of results do not issue one round trip per item; a missing URI is absent from the map rather than an error.

getEprintsByAuthor()​

getEprintsByAuthor(author, options?): Promise<StoredEprint[]>

Defined in: src/types/interfaces/storage.interface.ts:506

Queries eprints by author.

Parameters​

author​

DID

Author DID

options?​

EprintQueryOptions

Query options (limit, offset, sort)

Returns​

Promise<StoredEprint[]>

Array of eprints by this author

Remarks​

Returns eprints in order specified by options.sortBy. For full-text search across all fields, use ISearchEngine instead.

Example​

const eprints = await storage.getEprintsByAuthor(
  toDID('did:plc:abc')!,
  { limit: 10, sortBy: 'createdAt', sortOrder: 'desc' }
);

eprints.forEach(p => console.log(p.title));

getEprintUrisForTerm()​

getEprintUrisForTerm(normalizedTerm, limit, offset): Promise<{ total: number; uris: AtUri[]; }>

Defined in: src/types/interfaces/storage.interface.ts:884

Retrieves distinct eprint URIs that match a normalized term as either a community tag (user_tags_index) or an author keyword (eprints_index.keywords).

Parameters​

normalizedTerm​

string

Normalized tag/keyword string (hyphen-separated)

limit​

number

Maximum results (0 for count only)

offset​

number

Pagination offset

Returns​

Promise<{ total: number; uris: AtUri[]; }>

Deduplicated eprint URIs and total count

getRelatedWorksForEprint()​

getRelatedWorksForEprint(eprintUri, options?): Promise<RelatedWorkListResult>

Defined in: src/types/interfaces/storage.interface.ts:1006

Retrieves related works for an eprint.

Parameters​

eprintUri​

AtUri

AT URI of the eprint

options?​

RelatedWorkQueryOptions

Query options (limit, offset)

Returns​

Promise<RelatedWorkListResult>

Paginated list of related works

Remarks​

Returns user-curated related work links. Each link includes the target eprint URI and the relationship type.

Example​

const result = await storage.getRelatedWorksForEprint(
  toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!,
  { limit: 50 }
);

for (const rw of result.relatedWorks) {
  console.log(rw.targetEprintUri, rw.relationshipType);
}

getTagsForEprint()​

getTagsForEprint(eprintUri): Promise<readonly IndexedUserTag[]>

Defined in: src/types/interfaces/storage.interface.ts:871

Retrieves user tags for an eprint.

Parameters​

eprintUri​

AtUri

AT URI of the eprint

Returns​

Promise<readonly IndexedUserTag[]>

Array of indexed user tags

Remarks​

Returns individual user tag records indexed from the firehose. Each tag includes the tagger's DID, original tag text, and creation time.

Example​

const tags = await storage.getTagsForEprint(
  toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!
);

for (const tag of tags) {
  console.log(`${tag.taggerDid} tagged: ${tag.tag}`);
}

indexCitation()​

indexCitation(citation): Promise<void>

Defined in: src/types/interfaces/storage.interface.ts:969

Indexes a user-provided citation from the firehose.

Parameters​

citation​

IndexCitationInput

Citation data to index

Returns​

Promise<void>

void

Remarks​

Upserts into the extracted_citations table using user_record_uri as the deduplication key. Sets source to 'user-provided'.

indexRelatedWork()​

indexRelatedWork(relatedWork): Promise<void>

Defined in: src/types/interfaces/storage.interface.ts:1023

Indexes a related work record from the firehose.

Parameters​

relatedWork​

IndexRelatedWorkInput

Related work data to index

Returns​

Promise<void>

void

Remarks​

Upserts into the user_related_works_index table using uri as the primary key.

indexTag()​

indexTag(tag): Promise<void>

Defined in: src/types/interfaces/storage.interface.ts:912

Indexes a user tag record into the user_tags_index table.

Parameters​

tag​

Tag data to index

cid​

CID

createdAt​

Date

eprintUri​

AtUri

pdsUrl​

string

tag​

string

taggerDid​

DID

uri​

AtUri

Returns​

Promise<void>

void

isStale()​

isStale(uri): Promise<boolean>

Defined in: src/types/interfaces/storage.interface.ts:721

Checks if an indexed record is stale (PDS has newer version).

Parameters​

uri​

AtUri

Record URI

Returns​

Promise<boolean>

True if index is stale, false otherwise

Remarks​

Staleness is detected by comparing:

  • Indexed CID vs current PDS CID
  • Last sync time vs PDS update time

When stale, the indexing pipeline should re-fetch and re-index.

Example​

const isStale = await storage.isStale(
  toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!
);

if (isStale) {
  console.log('Re-indexing required');
}

listChangelogs()​

listChangelogs(eprintUri, options?): Promise<ChangelogListResult>

Defined in: src/types/interfaces/storage.interface.ts:800

Lists changelogs for a specific eprint with pagination.

Parameters​

eprintUri​

AtUri

AT URI of the eprint

options?​

ChangelogQueryOptions

Query options (limit, offset)

Returns​

Promise<ChangelogListResult>

Paginated list of changelogs, newest first

Remarks​

Returns changelogs ordered by creation date descending. Each changelog describes changes introduced in a specific version.

Example​

const result = await storage.listChangelogs(
  toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!,
  { limit: 10 }
);

for (const changelog of result.changelogs) {
  console.log(`Version ${changelog.version.major}: ${changelog.summary}`);
}

listDeletedEprintUris()​

listDeletedEprintUris(limit?): Promise<readonly AtUri[]>

Defined in: src/types/interfaces/storage.interface.ts:481

Lists eprints marked deleted, for reconciling derived indexes.

Parameters​

limit?​

number

Maximum rows to return

Returns​

Promise<readonly AtUri[]>

URIs of soft-deleted eprints

listEprintUris()​

listEprintUris(options?): Promise<readonly string[]>

Defined in: src/types/interfaces/storage.interface.ts:563

Lists all eprint URIs with pagination.

Parameters​

options?​

Query options including limit

cursor​

string

limit​

number

Returns​

Promise<readonly string[]>

Array of eprint URIs

Remarks​

Used for browsing all eprints without facet filtering. Returns URIs only for efficiency; full metadata can be fetched separately.

Example​

const uris = await storage.listEprintUris({ limit: 100 });

listEprintUrisByFieldUri()​

listEprintUrisByFieldUri(fieldUris, options?): Promise<readonly string[]>

Defined in: src/types/interfaces/storage.interface.ts:587

Lists eprint URIs filtered by field URIs.

Parameters​

fieldUris​

readonly string[]

Field URIs to filter by (OR semantics)

options?​

Query options including limit

limit​

number

Returns​

Promise<readonly string[]>

Array of eprint URIs that have any of the specified fields

Remarks​

Used for discipline/field-based filtering in browse pages. Returns eprints where the fields array contains objects matching any of the provided field URIs.

Example​

const uris = await storage.listEprintUrisByFieldUri(
  ['at://did:plc:graph-pds/pub.chive.graph.node/5a6b7c8d-9e0f-1a2b-3c4d-5e6f7a8b9c0d'],
  { limit: 100 }
);

searchKeywords()​

searchKeywords(searchText, limit): Promise<object[]>

Defined in: src/types/interfaces/storage.interface.ts:899

Searches distinct keywords from eprints_index matching a search term.

Parameters​

searchText​

string

Normalized search text

limit​

number

Maximum results

Returns​

Promise<object[]>

Keywords with usage counts

softDeleteEprint()​

softDeleteEprint(uri, source): Promise<Result<void, Error>>

Defined in: src/types/interfaces/storage.interface.ts:470

Marks an eprint deleted without removing its row.

Parameters​

uri​

AtUri

Eprint URI

source​

How the deletion was detected

"admin" | "pds_404" | "firehose_tombstone"

Returns​

Promise<Result<void, Error>>

Ok on success, Err when the row does not exist

Remarks​

Removing the row outright leaves nothing to reconcile against if a derived index (Elasticsearch, Neo4j) fails to drop the record.

storeChangelog()​

storeChangelog(changelog): Promise<Result<void, Error>>

Defined in: src/types/interfaces/storage.interface.ts:833

Stores or updates a changelog index record.

Parameters​

changelog​

StoredChangelog

Changelog metadata to index

Returns​

Promise<Result<void, Error>>

Result indicating success or failure

Remarks​

Upserts the changelog (insert or update based on URI). Called when firehose receives a changelog creation or update event.

ATProto Compliance: Stores metadata only, not source data.

Example​

const result = await storage.storeChangelog({
  uri: toAtUri('at://did:plc:abc/pub.chive.eprint.changelog/xyz')!,
  cid: toCID('bafyreib...')!,
  eprintUri: toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!,
  version: { major: 1, minor: 2, patch: 0 },
  summary: 'Updated methodology section',
  sections: [],
  createdAt: '2024-01-15T10:30:00Z',
});

if (!result.ok) {
  console.error('Failed to store:', result.error);
}

storeEprint()​

storeEprint(eprint): Promise<Result<void, Error>>

Defined in: src/types/interfaces/storage.interface.ts:420

Stores or updates an eprint index record.

Parameters​

eprint​

StoredEprint

Eprint metadata to index

Returns​

Promise<Result<void, Error>>

Result indicating success or failure

Remarks​

Upserts the eprint (insert or update based on URI). Updates indexedAt timestamp on every call.

ATProto Compliance: Stores metadata only, not source data.

Example​

const result = await storage.storeEprint({
  uri: toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!,
  cid: toCID('bafyreib...')!,
  author: toDID('did:plc:abc')!,
  title: 'Neural Networks in Biology',
  abstract: 'This paper explores...',
  documentBlobRef: {
    $type: 'blob',
    ref: toCID('bafyreib...')!,
    mimeType: 'application/pdf',
    size: 2048576
  },
  documentFormat: 'pdf',
  publicationStatus: 'eprint',
  pdsUrl: 'https://pds.example.com',
  indexedAt: new Date(),
  createdAt: new Date()
});

if (!result.ok) {
  console.error('Failed to store:', result.error);
}

storeEprintWithPDSTracking()​

storeEprintWithPDSTracking(eprint, pdsUrl, lastSynced): Promise<Result<void, Error>>

Defined in: src/types/interfaces/storage.interface.ts:689

Stores an eprint and tracks PDS source in a single transaction.

Parameters​

eprint​

StoredEprint

Eprint metadata to index

pdsUrl​

string

URL of the user's PDS

lastSynced​

Date

Last successful sync timestamp

Returns​

Promise<Result<void, Error>>

Result indicating success or failure

Remarks​

Wraps both store and PDS tracking in a transaction for atomicity. If either operation fails, both are rolled back.

ATProto Compliance: Ensures consistent PDS source tracking.

Example​

const result = await storage.storeEprintWithPDSTracking(
  eprintData,
  'https://pds.example.com',
  new Date()
);

trackPDSSource()​

trackPDSSource(uri, pdsUrl, lastSynced): Promise<Result<void, Error>>

Defined in: src/types/interfaces/storage.interface.ts:662

Tracks PDS source for staleness detection.

Parameters​

uri​

AtUri

Record URI

pdsUrl​

string

URL of the user's PDS

lastSynced​

Date

Last successful sync timestamp

Returns​

Promise<Result<void, Error>>

Result indicating success or failure

Remarks​

ATProto Compliance: Essential for detecting when index is stale (PDS has newer data) and triggering re-indexing.

This enables rebuilding the index from scratch if needed.

Example​

await storage.trackPDSSource(
  toAtUri('at://did:plc:abc/pub.chive.eprint.submission/xyz')!,
  'https://pds.example.com',
  new Date()
);