Indexing data on ICE/SNOW with Subsquid

Using Subsquid with ICE

Prerequisites

To run a squid project, you need to install:

Creating the Project

Subsquid has a template repository squid-templateon GitHub which can be used to start the project. You can fork this repository to your GitHub account and then start building on it.

  • Installing Dependencies
npm ci
  • Installing Additional Dependencies
npm install @ethersproject/abi ethers @subsquid/substrate-evm-processor @subsquid/evm-typegen

Define Entity Schema

To index ERC-721 token transfers and approvals, we will need to track:

  • Token transfers
  • Ownership of tokens
  • Contracts and their minted tokens
  • Approval of tokens
  • @entity - indicates that this type will be translated into a database-persisted ORM model
  • @derivedFrom - indicates the field will not be persisted in the database, rather it will be derived
  • type references (i.e. from: Owner) - establishes a relation between two entities
npx sqd codegen
TypeScript Entity Class generated

ABI Definition and Wrapper

Subsquid supports the automated creation of TypeScript type-safe interfaces for Substrate data sources (events, extrinsics, storage items).
Changes are recognized automatically throughout the runtime.

  • Create a folder named abi inside the src folder, and there create a JSON file named ERC721.json
mkdir src/abi
touch src/abi/ERC721.json
npx squid-evm-typegen --abi src/abi/ERC721.json --output src/abi/erc721.ts

Define and Bind Event Handler

The Subsquid SDK includes a processor class called SubstrateProcessor, or in this example, SubstrateBatchProcessor. To obtain chain data, the processor connects to the Subsquid archive. It will index from the specified starting block to the specified end block, or until new data is added to the chain.

Managing the EVM Contract

It is also necessary to define some constants and some helper functions to manage the EVM contract. You can create an additional file for these items:

touch src/contract.ts
  1. Define the chain node endpoint
  2. Create a contract interface to store information such as the address and the ABI
  3. Define functions to fetch a contract entity from the database or create one
src/contract.ts file
  • ArcticToken with symbol ARTK deployed at 0x822f31039f5809fa9dd9877c4f91a46de71cde63
  • MyToken with symbol MTK deployed at 0x581522ca7b73935e4ad8c165d5635f5e15a7658d

Configure Processor and attach handlers

The src/processor.ts file is where the template project instantiates the SubstrateBatchProcessor class, configures it for execution, and attaches the handler functions.
It defines the Squid processor which retrieves on-chain data from the endpoint exposed by Squid Archive, applies arbitrary modifications, and stores the result in the target database schema (that we defined in the schema.graphql file).
It also defines data handlers (handleTransfers & handleApprovals) to subscribe to log entries of interest. The data handlers specify the data to be retrieved as well as how the data is processed and saved to the destination database. The handlers rigorously process the execution log items in the sequence prescribed by the data contained in the historical chain blocks.

src/processor.ts file
  • setBlockRange(Range): Limits the range of blocks to be processed
  • setBatchSize(number): Set the maximal number of blocks fetched from the data source in a single request
  • setDataSource(DataSource): Set the data source to fetch the data from:
  • addEvmLog : used to subscribe to the EVM log data (event) emitted by a specific EVM contract

Launch Database

When executing the project locally, like in this article, the docker-compose.yml file in the root directory is included with the template which is used to launch a PostgreSQL container.
Run the following command in your terminal to accomplish this.

docker-compose up -d
  • Build the code
npm run build
  • Remove previous migrations
rm -rf db/migrations/*.js
  • Generate and apply the migrations so that tables are created on database
npx squid-typeorm-migration generate
npx squid-typeorm-migration apply
Applying Migrations to DB
docker-compose -f archive/docker-compose.yml up
docker-compose -f archive/docker-compose.yml down -v

Launch Project

To launch the processor (this will block the current terminal), you can run the following command in new terminal:

node -r dotenv/config lib/processor.js
Output after running the processor
npx squid-graphql-server

Query Example

Let’s query owners of token with their balance and id(address), approvals events with address owner and approved along with their balances, and tokens with their name and symbol

query MyQuery {
owners(limit: 10) {
balance
id
}
approvals(limit: 10) {
approved {
balance
id
}
owner {
balance
id
}
}
tokens(limit: 10) {
contract {
name
symbol
}
}
}
GraphQL Query Console

Summary

You can view the finalized and complete project on this GitHub repo.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
ICONOsphere

ICONOsphere

With a deep history of commitment to building projects that foster community growth & personal empowerment we work to create a nation built on trust with icon.