Community LangChain integration for pixelrag-serve

[navneet-singh2907]

Hi, thanks for PixelRAG, the paper and the hosted demo endpoint are both excellent.

I built a small unofficial LangChain adapter for pixelrag-serve: a PixelRAGRetriever and a PixelRAGSearchTool that drop into LangChain/LangGraph pipelines and talk to either a self-hosted instance or (as an explicit opt-in) api.pixelrag.ai.

Repo: https://github.com/navneet-singh2907/pixelrag-langchain, Apache-2.0, tested against the live endpoint, with the paper and this repo credited as the source of everything it wraps.

Three questions:

1. Response schema — I mapped fields from live responses (results[].hits[], vector_id, article_id, path, and url carrying the article title). Is that shape stable, or is there a spec I should track before pinning a release?

2. Hosted endpoint usage — is pointing a pip-installable package at api.pixelrag.ai (opt-in only, default is localhost) okay with you, and would you like me to document any rate limits?

3. Retrieving tile images — search returns image_base64: null and relative paths. Is there a way to fetch the actual tile images via the API, so downstream VLMs can read them?

Happy to change anything about naming or framing if you'd prefer, and to hand the repo over or archive it if you'd rather ship an official integration. Cheers.

[andylizf]

Thanks for building this — the explicit hosted opt-in and configurable self-hosted base URL are exactly the right shape. I checked current pixelrag-serve, the live endpoint, its OpenAPI document, and your client before answering the three questions.

1. Response schema: the current contract is results[].hits[]. The authoritative machine-readable spec is served at https://api.pixelrag.ai/openapi.json; it currently reports API version 0.1.0. We have not declared a pre-1.0 backward-compatibility guarantee yet, so please track that spec and keep parsing additive fields leniently. Three important details from the live response:

   • vector_id may be an integer or string depending on the backend.
   • url can currently be a Wikipedia title/slug, not necessarily a fetchable URL.
   • path is relative to the server's tile store and is not a public download URL.

   For tile access, treat article_id, tile_index, and chunk_index as the useful coordinates rather than constructing anything from path.

2. Using api.pixelrag.ai: yes — an explicit opt-in like PixelRAGConfig.hosted() is fine, and you’re welcome to point users at it. This is a production service, not a throwaway demo: GET /health and GET /status are public, and /status reports the live index (currently 26,315,959 vectors at dimension 2048, built 2026-05-07), so your client can verify the backend programmatically.

   What we don’t publish is a benchmark: no throughput numbers, no latency percentiles, no documented rate limit. So don’t promise your users a specific speed. Anyone who needs guaranteed capacity should self-host — the pipeline is the same one behind this endpoint.

   And if you hit a limit or need something the public endpoint can’t give you, open an issue and talk to us. We’d rather hear about it than have you work around it.

3. Fetching tile images: use the coordinates from each hit:

   GET https://api.pixelrag.ai/tile/{article_id}/{tile_index}/{chunk_index}
   

   I verified a live search hit at /tile/4466356/0/3: it returned HTTP 200, image/png, 123,920 bytes. The alternative is to send "include_images": true in /search; the same live query returned a non-null base64 image (165,228 characters). The separate GET is usually preferable because it keeps search responses small.

Your client already correctly avoids treating url as an image URL and recognizes that path is server-relative. The missing piece is to construct the coordinate-based /tile/... URL (or request inline images) so downstream VLMs receive actual pixels.

Happy to keep this community-owned rather than requiring a handoff. Once tile retrieval is wired in, this looks like a useful integration to link from PixelRAG's docs.

[navneet-singh2907]

Thank you. This answers all three questions and gives me a clear implementation path. I’ll add tile_index and chunk_index to the result model, construct /tile/{article_id}/{tile_index}/{chunk_index} from the configured base URL, and expose separate tile fetching as the default path, with include_images as an opt-in. I’ll also track the OpenAPI contract, add /status support, and document the hosted service’s capacity caveat. I’ll follow up once the tile retrieval path and tests are complete.