We are trying to build an indexer/aggregator in the style of TheGraph. Using TheGraph directly is not feasible, since we need to merge blockchain data with offchain data in our datastores.
The core idea is to listen to blockchain events using a golang service, and aggregate them into our database, using something like https://goethereumbook.org/en/event-subscribe/.
However, my concern is around fault tolerance. If some events are missed (eg the service goes down), there need to be measures to query the missed blocks and look for missed events.
How can I know if some events have been missed? AFAIK there are no nonces associated with events. Should I listen to individual blocks instead of listening to events? This helps since I can now identify missed blocks, but seems much more inefficient.
This looks like a solved problem in the industry and multiple organisations run their own indexers/aggregation services, but there isn't much documentation available around this. Would appreciate some pointers to resources or design-patterns.
[link] [comments]
You can get bonuses upto $100 FREE BONUS when you:
π° Install these recommended apps:
π² SocialGood - 100% Crypto Back on Everyday Shopping
π² xPortal - The DeFi For The Next Billion
π² CryptoTab Browser - Lightweight, fast, and ready to mine!
π° Register on these recommended exchanges:
π‘ Binanceπ‘ Bitfinexπ‘ Bitmartπ‘ Bittrexπ‘ Bitget
π‘ CoinExπ‘ Crypto.comπ‘ Gate.ioπ‘ Huobiπ‘ Kucoin.
Comments