How to Utilize the Mempool
Hey DEFI TIMES community,
the ETH mempool is an exciting place. Most people don’t even know about it, let alone all the things you can do with it.
Today’s piece is all about the mempool, specifically how to utilize it!
It was written by the QuikNode team. QuikNode is a Web3 infrastructure provider and, most importantly, a great alternative for Infura.
Don’t forget to follow them on Twitter!
Subscribe to our newsletter to level up your DeFi game!
Intro
On Ethereum, when a transaction is sent, before being added to a block, it resides in what is called a Mempool. To receive information about this transaction, the Mempool must be queried. This guide will demonstrate how to query a node’s mempool using QuikNode Ethereum nodes.
What is the ETH Mempool
In blockchain terminology, a mempool is a waiting area for the transactions that haven't been added to a block and are still unconfirmed. This is how a Blockchain node deals with transactions that have not yet been included in a block.
When an Ethereum node receives a transaction, it will propagate the transaction to peer nodes until a miner approves the transaction and adds it to a new block. Before it’s added to the next block, the transaction remains in a staging/waiting area called mempool or txpool. It is also known as “pending transactions” to some people.
This waiting area or buffer zone functionality is needed because transactions aren't immediately added to the Blockchain. Nodes also run a series of validity checks on these transactions. These checks include ensuring that the funds are still available, the output is not exceeding the input; the signature is valid, etc. The transaction is rejected if it fails any of these tests. Note that the Mempool cannot be considered as a master reference shared universally by all nodes. Each node configures its own rules for the node’s mempool. Additionally, a node may be the first to receive a transaction but may not have propagated the transaction to the rest of the network.
For example, the default settings for slots and memory on Geth and OpenEthereum dictate their behavior with mempool transactions. When a new pending transaction is allowed into the mempool, but all the node spaces are full -- the transaction with the lowest gas fee will be dropped from the pool. The rules for accepting and dropping transactions vary for different nodes and depend on their settings.
Even the term itself differs depending on clients:
OpenEthereum f.k.a Parity client calls it TX-QUEUE.
However, “mempool” is a widely used and accepted term.
Ways to utilize an ETH Mempool
As stated above, transactions need to be stored before they are confirmed and picked up by a miner. Usually, the transactions with lower gas prices spend more time in a mempool. Such transactions may be overwritten with replacement transactions (commonly known as 'cancel and speed up transaction'). One of the most prominent uses of the Mempool is to get pending transactions. Here’s why you may want to do that:
In Trading, to obtain trading prices on DEX like Uniswap, Balancer, etc.
To analyze the gas prices and adjust the transaction price to avoid getting dropped.
To analyze and simulate pending transactions, which can help reduce rejection rates.
Let’s get started!
Booting an Ethereum node
For the purpose of this guide, any Ethereum client can be used, such as Geth or OpenEthereum (fka Parity). To boot a new Ethereum node - first, select a client and configure it. Note that syncing and maintaining an Ethereum node is a challenging task and could take days.
To make things easy, we'll grab a node from QuikNode, which is a lot easier to set up. After a QuikNode Ethereum node is up, copy your HTTP Provider endpoint:
This guide uses an OpenEthereum (Parity) dedicated node with an Archive add-on. A Geth dedicated node can also be used to illustrate the difference between their Mempool approach.
Copy and save your HTTP provider endpoint.
How to access and query the Ethereum Mempool
First, query the OpenEthereum node's Mempool (make sure to have an Archive add-on). OpenEthereum calls its mempool a Transaction Queue or TX-QUEUE. An OpenEthereum client listens to the network for transactions to include in the blocks (if mining/validating) or for transaction broadcasting. These transactions (emitted locally or externally) are verified and ordered into a transaction queue.
Make a cURL request to the transaction queue/mempool of your node:
curl --data '{"method":"parity_pendingTransactions","params":[],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST ADD_YOUR_ETHEREUM_NODE_URL
Replace `ADD_YOUR_ETHEREUM_NODE_URL` with the HTTP provider you saved earlier.
The above command is making a cURL JSON-RPC request to your node's transaction queue/mempool.
The `parity_pendingTransactions` method returns the pending transactions. In the params section, the number of transactions to return is set with the `limit` parameter. Transactions can also be filtered based on the `to` and `from` address, gas, gasPrice, value, and nonce.
A successful output should look like this (a single transaction is highlighted):
The output contains the stream of transactions arranged by priority. Each transaction includes the hash, nonce, block hash, block number, value, gas price, gas, raw transaction data, `to` and `from` address, public key, chain id, etc.
The other txqueue methods are `parity_localTransactions,` which returns 25 local transactions that may be mined, pending, or dropped, and `parity_allTransactions` to return the transactions to be processed in the future as well as dropped transactions.
To query a Geth node's Mempool or TX-POOL or Transaction Pool:
Make a cURL request to the transaction pool/mempool of your node:
curl --data '{"method":"txpool_content","id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST ADD_YOUR_ETHEREUM_NODE_URL
Replace `ADD_YOUR_ETHEREUM_NODE_URL` with the HTTP provider you saved earlier.
This will make a cURL JSON-RPC request to your Geth node's transaction pool/mempool by using the `txpool_content` method, which will return the pending transactions and those queued for future execution. The output stream will have two fields: pending and queued.
A successful output should look like this (a single transaction is highlighted):
This output features the same elements as in the previous example.
The other txpool methods are `txpool_inspect,` which returns a listed summary of pending transactions in text form for a quick view, and `txpool_status,` which returns the current number of pending and queued transactions.
Querying with Web3
Let’s examine how to subscribe to pending transactions from mempool using Web3.js.
Prerequisites
NodeJS installed on your system
A text editor
CLI
Check our guide on [How to connect to Ethereum network with Web3.js](https://www.quiknode.io/guides/web3-sdks/how-to-connect-to-ethereum-network-with-web3-js/) to learn how to install and work with the Web3.js library.
Step 1.
Install web3.js by executing the following command:
$ npm install web3
Now, create a text script file and call it index.js
var Web3 = require('web3');
var web3 = new Web3(Web3.givenProvider || "ADD_YOUR_ETHEREUM_NODE_WSS_URL");
var subscription = web3.eth.subscribe('pendingTransactions', function(error, result){
if (!error)
console.log(result);
});
When we run this file, it gives out a stream of hashes of pending transactions currently in the mempool.
$ node index
This can be useful for the purpose of mempool tracking, tracking specific transactions, for trading bots and much more.
Conclusion
This short guide showed you how to easily query a mempool of an Ethereum node using QuikNode as an example. Understanding pending (in-flight) transactions provides valuable insights into your (& others’) transactions, so you can make valuable adjustments.
Subscribe to our newsletter for more articles and guide on Ethereum. If you have any feedback, please feel free to reach out to us via Twitter, and you can always chat with us if you have a question via our community server on Discord. It has some of the coolest devs out there, thanks :)
All information presented above is meant for informational purposes only and should not be treated as financial, legal, or tax advice. This article's content solely reflects the opinion of the writer, who is not a financial advisor.
Do your own research before you purchase cryptocurrencies. Any cryptocurrency can go down in value. Holding cryptocurrencies is risky.