File formats

Reference for every file Arcane reads and writes.

Inputs

File	Used by	Notes
Genome FASTA	filter	.fa/.fa.gz. Should be the species reference, e.g. GRCh38 for human.
GTF annotation	filter	.gtf/.gtf.gz. Should contain the gene annotations for the provided reference. If the annotation was created for a different reference, coordinate mismatches can occur. These lead to incorrect gene mappings.
R1 FASTQ	correct, map, express	Barcode + UMI reads. .gz/.xz supported.
R2 FASTQ	map, express	cDNA sequence reads. .gz/.xz supported. Must be in the same order as R1.
Inclusion list	correct, express	Gzipped barcode list. Supplied by --chemistry, or your own via --inclusion-list.

arcane filter outputs

Written into --prefix, with <NAME> from --name (a trailing _ is appended if you did not supply one, so --name human gives arcane_human_...).

arcane_<NAME>ref.fa.gz

The reference for arcane index. Each FASTA header is a single integer encoding the gene and the sequence's role:

(gene_number << 3) | exon(1) | junction(2) | coding(4)

It contains, per gene:

• the full gene including introns (flagged coding only),
• every exon of at least the mask width w (flagged exon),
• every exon-exon junction window of ±(w-1) bases (flagged junction).

Sequences on the - strand are reverse-complemented.

arcane_<NAME>genes.txt

Gene ids, one per line, in gene_number order. This file defines the gene numbering used everywhere else:

• its line count is what you pass as --ngenes to arcane count,
• it is embedded into the index as the column names,
• its line numbers are the gene_row values in the count matrix.

arcane_<NAME>new_id_to_original.csv

CSV, no header row:

gene_number,gene_id,chrom,start,end

Maps Arcane's internal gene numbers back to annotation gene ids and coordinates.

arcane_<NAME>mapping_gene_ids.pickle

Python pickle of a dict mapping gene_number → gene_id, plus a sentinel entry for the "undecided" gene id.

arcane_<NAME>extracted_genes.fa.gz

Only with --extract_genes. Each gene from start to end, introns included.

arcane index outputs

File	Contents
<index>.hash	The hash table itself (raw).
<index>.info	Pickled metadata: hash info, value info, optional info, and application info (rcmode, mask, k, gene column names).

Both are needed. Pass the shared prefix as --index; Arcane appends the suffixes. (The .info path is also accepted.)

arcane express outputs

arcane express produces the user-facing subset of the above, all sharing the --out prefix:

<out>_counts.mex.gz     # count matrix, see arcane map
<out>_genes.tsv.gz      # matrix rows, see arcane map
<out>_barcodes.tsv.gz   # matrix columns, see arcane correct

Intermediates go to a random subdirectory of --tmpdir and are deleted unless you pass --keep-tmp.

arcane correct outputs

File	Contents
<output>_barcodes.tsv.gz	One corrected cell barcode sequence per line. Row order = the columns of the count matrix.
<tmpdir>/arcane_barcode_counts_<sample>.data	Reads per valid barcode. Input to arcane count --counts.
<tmpdir>/arcane_barcode_mapping_<sample>.data	Barcode mapping.
<tmpdir>/arcane_barcode_bitmap_<sample>.data	Observed-barcode bitmap.
<tmpdir>/arcane_<file>_buq.data, _bu.data	Per-input-file encoded barcode/UMI streams.

<sample> is derived from the input filename.

arcane map outputs

File	Contents
<tmpdir>/arcane_ug_<sample>.data	Memory-mapped uint64 UMI-gene array, one entry per read, cell-major. Input to arcane count --umi-gene-array.
<outprefix>genes.tsv.gz	Gene ids, one per matrix row.

In color-only mode there is one line per gene. In splice-aware mode there are four lines per gene, suffixed:

Suffix	Meaning
<gene>-E	exon only
<gene>-U	unspliced (intron)
<gene>-S	spliced (exon-exon junction)
<gene>-I	ambiguous mix

arcane count outputs

Count matrix

<out>_counts.mex.gz — a gzipped, MatrixMarket-like sparse triplet file:

%% Arcane count matrix
%
36601 4823 9127438
15 1 3
15 7 1
...

• Lines starting with % are comments.
• The first non-comment line is <ngenes> <ncells> <non-zero entries>.
• Every following line is <gene_row> <cell_col> <count>.
• Both indices are 1-based: gene_row indexes into genes.tsv.gz, cell_col into barcodes.tsv.gz.

Reading it with SciPy:

import gzip
import numpy as np
from scipy.sparse import coo_matrix

with gzip.open("results/sample_counts.mex.gz", "rt") as f:
    lines = (ln for ln in f if not ln.startswith("%"))
    ngenes, ncells, nnz = map(int, next(lines).split())
    data = np.loadtxt(lines, dtype=np.int64).reshape(-1, 3)

matrix = coo_matrix(
    (data[:, 2], (data[:, 0] - 1, data[:, 1] - 1)),
    shape=(ngenes, ncells),
)

With --plot: <out>_combined_umi_hist.pdf, the UMI histogram before and after resolution.

arcane bulk outputs

Gene quantification

<out>_gene_quantification.csv

ENSG00000186092,298
ENSG00000284733,0
ENSG00000284662,0
ENSG00000187634,999
...

Every line contains an ENSEMBL gene and its count.