Introducing Outserv: Blockchain Search with GraphQL
Decentralization — the dispersion or distribution of functions and powers. — Merriam Webster
Today, I’m excited to announce the launch of Outserv — a GraphQL-powered Blockchain Search Engine. Outserv enables you to run production-grade GraphQL Search APIs over any blockchain data 10x faster than existing mechanisms.
Outserv is a unique mix of GraphQL, Search, and Cache systems — all put together into one single, easy-to-run executable binary. Anyone wishing to search over any portion of the blockchain data can now set up and run search using Outserv within minutes where it used to take hours or within hours where it used to take days.
On my desktop, all the transactions in the entire (15M blocks of) Ethereum blockchain were indexed by Outserv within 12 hours from scratch, ready to be served — a 10x improvement over conventional methods.
While there’s plenty of decentralization in blockchain technology itself, blockchain search infrastructure is still largely centralized. My work on Outserv is motivated to enable a level of decentralization not yet seen in the blockchain search space.
Outserv is 10x Better
Here are six reasons why Outserv is a 10x jump over conventional methods.
1. Outserv Provides Fast, Production Grade GraphQL APIs
Outserv starts with a GraphQL schema and provides GraphQL APIs out of the box. The schema can be hot-reloaded without any downtime. Any indices can be built or rebuilt in real-time.
Powered by advanced indexing techniques involving integers and roaring bitmaps, most of the queries run within milliseconds. Even the JSON output from the execution of GraphQL query uses advanced integer encoding techniques to create the output exceptionally efficiently.
2. Outserv is Easy and Fast to Set Up
Outserv is the only system that combines the power of a search engine, a cache engine, and a GraphQL layer, thus replacing three different systems in production.
It is a single binary, which is easy to run — all with an outserv graphql command. Anything can be indexed within minutes or hours, allowing the user to set up and run a search service within the day.
With Geth and Outserv bootloader running on a single machine with Ryzen 5950x chip, I indexed all 15 million Ethereum blocks’ transactions within 12 hours, producing 750GB of compressed output. Conventional mechanisms would take days to index the same amount of data.
3. Importers: Write Once, Benefit Forever
Importers are what connect Outserv to the blockchain. Outserv comes with an Ethereum importer that imports Ethereum transactions. It is easy to fork or extend this importer to add new importers.
You can write importers in any language — Rust, Go, Javascript, etc., and use them to import anything — L1/L2 blockchains, smart contracts, bridges, NFTs, etc.
I plan to add more importers to make it easy for Outserv users to make their data searchable. I encourage Outserv users to submit pull requests to add new importers or extend existing ones, thus benefitting everyone in the web3 world.
4. Outserv runs Custom Business Logic
Outserv spins up multiple NodeJS processes and uses interprocess communication to execute high-performance custom Javascript lambda functions. You can use these functions to extend GraphQL APIs to run complex, custom logic keeping the client code simple.
5. Outserv can Shard, Replicate and Scale
Outserv uses the Raft consensus algorithm to ensure synchronous replication. That means any write, once confirmed, is guaranteed to be read by all replicas. Adding replicas is a quick way to serve more query traffic.
It uses advanced column-level sharding mechanisms to divide up data evenly across machines. So, there’s no limit on how much data you can index and serve 1.
6. Run Outserv on the Edge
Gone are the days when a single server running in US-East was sufficient to serve the entire world. Run a replicated Outserv instance in every geography where your users are located, so they can get results in the fastest possible time.
Why Build Outserv?
Decentralization plays a significant role in web3. The decentralization factor of blockchains is measured by how many instances of the chain are running in the wild. Bitcoin and Ethereum have thousands of active instances2.
From a storage perspective, a blockchain is a relatively simple ledger of transactions with minimal search functionality. To better understand the blockchain and use its data, we need to enable advanced search. And that’s where the limitations of web3 infrastructure start to show.
Web3 infrastructure beyond blockchain is largely centralized. There are (centralized) search services available. But, no system exists which enables the user to set up a production-ready search locally.
The ability to download and run software (like the Ethereum node) is the ultimate power of decentralization. In fact, for something to be truly decentralized, it should have hundreds, if not thousands of instances running. That’s what Outserv enables.
Outserv is my vision for true decentralization — transferring the power of search to the users. I envision thousands of instances of Outserv running in the world — enabling every developer to search their world in their way.
Outserv Has Many Use Cases
- Enable web3 developers, companies, and foundations to set up search services for internal or external usage.
- Enable dapp developers to set up GraphQL APIs to power their apps.
- Enable dapp developers to run their search service closer to the edge.
- Enable web3 researchers to search block data for analysis.
Compared to The Graph
The Graph (thegraph.com) also provides GraphQL search APIs and aims to be decentralized, so how is that different from Outserv? Taking an analogy, The Graph is a community-powered solar park. In contrast, Outserv enables you to install solar panels on your roof.
The Graph is a search service; Outserv allows you to setup search locally. You don’t have to wait for The Graph indexers to service the data you want to search over. You can spin up an Outserv instance to make any portion of information you want searchable, run it anywhere you want, add custom notes, custom functions, etc. With Outserv, you’re fully in control.
Outserv is Liberally Licensed
Outserv is licensed under Sustainable License — a very liberal commercial license. Sustainable License is a creator-friendly alternative to open source licenses, providing all the user freedoms expected from open source.
You can read more about the reasons behind creating Sustainable License here.
Outserv Roadmap
See the product roadmap here.
The first big thing is to write importers for various blockchains, smart contracts, NFTs, and so on. For those who are interested, importers are also where there’s a great potential to contribute to the project.
Then, I’d like to extend the GraphQL APIs to enable various use cases. There are infinite ways to develop these, so I’d rely upon the Outserv user base to provide feedback about how they are using Outserv and where it falls short.
Join the Community
Do try out Outserv and join the Discord community to ask for help and to let us know how it goes.