P2P AI inference: how it works and its limits
Peer-to-peer AI inference connects someone who needs a model response with a seller who can produce it. The buyer chooses an offer, sends a request and receives the result from that seller. The aim is to make intelligence more accessible by letting people produce it, sell it or give it away.
Gyblos is in development. The public marketplace and apps are not released. This guide explains the design and the questions to ask before choosing a service; the public paper records implementation and testing status.
By Gyblos · Reviewed
What is being exchanged?
Inference means running a trained model to produce an output, such as an answer or a summary. In a P2P inference marketplace, the buyer requests that work from a seller. The seller operates a model engine or uses an upstream service under its own responsibility.
The offer should identify the model, price units, availability, limits and data policy. A model name or an engine URL alone does not establish which weights ran, how usage was counted or what happened to the data.
P2P, distributed inference or GPU rental?
These describe different choices about who runs the model and what the buyer receives. A service can combine more than one approach; check the actual offer.
A P2P marketplace does not require splitting one model across buyers’ and sellers’ machines. In the Gyblos design, one selected seller serves a request. Combining devices behind a seller is a separate implementation choice, not an established Gyblos capability.
| Approach | What you obtain | What to check |
|---|---|---|
| P2P inference marketplace | Responses from an offer served by a seller. | Model, usage measurement, spending limits, data policy and availability. |
| Distributed inference | Model execution divided across devices. | How work is split, network requirements and who can access each part. |
| GPU rental | Access to computing capacity that you configure or manage under the rental terms. | Hardware, billed time, storage, setup and responsibility for operating the model. |
| Local inference | A model running on your own equipment. | Model licence, memory, power, software and the data paths you configure. |
A request, from offer to result
The intended Gyblos buyer-to-seller route follows these steps. They explain the protocol design, not a released setup tutorial.
- Discover and compare offers. Check the declared model, input/output prices, service limits and data handling.
- Choose a seller and a budget. For paid use, the design uses USDC payment channels on Solana; model execution happens off-chain.
- Send the request from the buyer node to the seller node. The seller runs its configured engine and returns the output. A direct connection or encrypted relay can carry the node-to-node traffic.
- Check usage and the result. Settlement and return of unused funds are part of the payment design; a connection or response alone does not prove payment recovery.
Compare cost per useful result
A low token price is one input to a decision. Include input and output usage, unsuccessful work, retries where chargeable, network or protocol costs and the number of useful tasks completed. Check what the quoted price includes before adding fees separately.
Set a ceiling for spending and for the number of calls. If a payment outcome is uncertain, stop new paid calls and reconcile it before proceeding. A technical retry must not silently become another payment.
Free offers depend on the seller, availability and limits. They still consume equipment, electricity or someone else’s budget. Neither P2P connectivity nor an open market establishes a lowest-price or unlimited-free-access claim.
Who can read the request?
The seller can read the prompts and responses it processes. If it uses an upstream provider, that provider is another data-handling boundary. The application-to-buyer connection is also separate from the buyer-to-seller connection.
Iroh encrypts traffic between its endpoints. A relay forwards that encrypted traffic when a direct path is unavailable and cannot read its content. This protects transport between the buyer and seller nodes; it does not hide content from the seller or establish anonymity.
Review retention, logging, upstream services and deletion policies before sending sensitive information. A declared policy is not proof that it is enforced. A future managed gateway introduces its own request and funds-handling responsibilities.
Identity, usage and model checks answer different questions
An identity record or a signed offer can help establish who made a commitment. A signed receipt can record what its signers attest to. Neither proves which model produced a response, that every token count is truthful or that the answer is correct.
The Gyblos design distinguishes a verified model, measurable usage and declared usage. The planned verified-model seal applies only to eligible open-weight models with the information needed for statistical audits against reference weights. A public tokenizer can help recount usage without proving model identity; declared usage relies on the seller’s report.
These descriptions apply to the model and usage, never to a “verified seller”. Statistical model audits and their production handling remain separate from identity and transport checks. Read current implementation status before relying on a planned guarantee.
Can a home computer participate?
Participation depends on the model, memory, context, concurrent requests and the host software. Test the workload you intend to offer. Hardware ownership alone does not establish usable capacity, compatible software or demand.
The node design can connect to a model engine, including a local engine or an upstream API under the seller’s responsibility. Check licences and resale rights. The longer-term vision includes more kinds of devices; complete buyer and seller support on every device is not established.
Home networks can require relay fallback. Measure the full request path, availability and operating costs rather than relying only on a local generation-speed result.
Five checks before choosing an offer
Use these questions to compare the actual service you can access today. Gyblos documentation and catalog examples do not establish a currently available seller.
- Can I access this offer, and does it support the model, context and workflow I need?
- What is measured, what is only declared, and what evidence is available?
- What can I spend, what happens on failure, and how is an uncertain payment reconciled?
- Who receives my content, and what are their data-handling policies?
- What happens if the seller or connection becomes unavailable?