LoRA Adapters

Status: LoRA fine-tuning is defined in the protocol specification. On-chain runtime execution is coming in a future release. The AdapterFactory contract exists as a protocol stub.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that allows large AI models to be customized without retraining all of their weights. Citrate makes LoRA adapters a first-class protocol primitive, enabling permissionless model customization, on-chain adapter management, and network-wide sharing through mentorship signals.

What is LoRA?

Traditional fine-tuning modifies all parameters of a model, which is expensive and produces a full-sized copy for every variant. LoRA takes a different approach: it freezes the base model's weights and injects small, trainable low-rank matrices into each layer.

For a weight matrix W of dimensions d x d, LoRA decomposes the update as:

W' = W + BA

Where:
  W  = original frozen weights     (d x d)
  B  = low-rank down-projection    (d x r)
  A  = low-rank up-projection      (r x d)
  r  = rank (typically 4-64, much smaller than d)

Because r is much smaller than d, the adapter (B and A matrices) contains only a fraction of the parameters of the full model. A rank-16 adapter for a 70B parameter model might be only 50MB -- small enough to store on-chain and distribute across the network.

The LoRAFactory Contract

Citrate's AdapterFactory precompile (at address 0x0103) manages the full adapter lifecycle:

Creating an Adapter

// Deploy a new LoRA adapter for an existing base model
bytes32 adapterId = IAdapterFactory(0x0103).createAdapter(
    baseModelId,       // The model this adapter modifies
    "my-domain-expert", // Human-readable name
    adapterWeights,     // Serialized LoRA weight matrices
    16                  // Rank (r=16)
);

The adapter is stored on-chain (weight data via content-addressed storage, metadata in contract state) and becomes immediately available to any node running the base model.

Composing Adapters

One of LoRA's most powerful properties is composability. Multiple adapters can be blended together to combine capabilities:

// Combine a "medical-knowledge" adapter with a "formal-writing" adapter
bytes32 composedId = IAdapterFactory(0x0103).composeAdapters(
    medicalAdapterId,
    formalWritingAdapterId,
    70,   // 70% weight to medical knowledge
    30    // 30% weight to formal writing
);

This produces a new adapter that inherits traits from both parents. Composed adapters can themselves be composed further, enabling a rich tree of specializations.

Adapters as Mentorship Signals

In Citrate's Paraconsensus model, LoRA adapters serve as the primary vehicle for mentorship between nodes. When a node fine-tunes an adapter on its local data and tasks, the resulting adapter encodes what that node has "learned." By sharing adapter updates as mentorship signals, nodes teach each other without exposing raw training data.

The mentorship flow works as follows:

Local Training -- A node fine-tunes a LoRA adapter on its local inference workload.
Signal Broadcast -- The node publishes a compressed gradient summary (the adapter delta) as a mentorship signal.
Checkpoint Aggregation -- At each finality checkpoint, the network aggregates all adapter deltas using federated averaging.
Global Update -- The aggregated adapter is committed to chain state, improving the shared model for all participants.

This creates a virtuous cycle: nodes that contribute useful adapter updates (measured by downstream accuracy improvements) earn higher reputation scores, which in turn gives their future contributions more weight in the aggregation process. We designed this loop intentionally -- the network should reward nodes that make it smarter.

Adapter Versioning

Every adapter maintains a version history on-chain:

Field	Description
`version`	Sequential version number
`parentId`	Previous version or base adapter
`weightsHash`	Content hash of the weight matrices
`accuracy`	Network-measured accuracy delta vs. base model
`creator`	Address that submitted this version
`checkpoint`	The finality checkpoint that included this update

Applications can pin to a specific adapter version for reproducibility, or track the latest version for continuous improvement.

Gas and Storage

LoRA adapters are designed to be lightweight, but on-chain storage is still expensive. Citrate uses a hybrid storage model:

Metadata (name, version, rank, accuracy) is stored in LVM contract state.
Weight data is stored in content-addressed off-chain storage (IPFS/Arweave), with only the content hash committed on-chain.
Adapter deltas (mentorship signals) are compressed using quantization and stored ephemerally -- they are aggregated at each checkpoint and can be pruned afterward.