This is a proposal to introduce a block packaging system for Nano to improve network security by solving the ledger bloat problem and significantly reducing node operational costs.
Nano nodes have three costs: bandwidth, storage and compute. The bandwidth and compute required to confirm a transaction is a one-time cost. The cost to store a transaction however, is a recurring cost for all nodes in the network to pay indefinitely.
At current network parameters a spammer could add 50GB of spam to Nanos ledger in one month. This would start increasing node storage costs at a rate of US$5 per month (based on AWS costs). After one year, all node operators would be paying US$60 per month just to store these useless spam transactions.
Nano's beta network has recently seen ~2200 transaction confirmations per second. If the live network were to scale up to this capacity, a spammer could add up to 1TB of useless spam to the ledger in a single month that all nodes would need to pay to store every month forever.
Rising costs harm decentralization as nodes will start dropping off if costs become too high. Preventing centralization is critical for Nano. If one entity were to gain just 34% of the online voting weight they could censor transactions or stall the network.
Nano currently has two defense mechanisms to keep ledger bloat under control: bandwidth throttling and proof-of-work.
Proof-of-work is the first line of defence against ledger bloat. It adds a cost to spam which is currently sufficient to hold back spammers.
In reality this cost is very small for a spammer with incentive and the right hardware and skills. You can buy an ASIC on eBay for $200 that could produce the proof-of-work for a ledger bloat attack while consuming just 200W.
This cost, however, is very significant for real users using Nano as intended. The following video shows the true speed of Nano.
Nano claims to provide free, subsecond transactions. This is only true when using some centralized and subsidized GPU work service such as BoomPOW.
It is orders of magnitude easier for a spammer using dedicated hardware to produce proof-of-work than it is for real users. For this reason, using proof-of-work for this purpose is fundamentally broken and a weak defense.
By throttling bandwidth the network can set a hard cap on the rate of ledger growth. During a spam attack, prioritization techniques should allow for “legitimate” transactions to still be confirmed while keeping the throughput low.
Currently the bandwidth limits are high enough to allow for devastating ledger bloat. In the event of a spam attack they would need to be lowered. This comes at a cost of reducing the transaction capacity, and more worryingly, makes it easier for a more sophisticated attack to overwhelm the prioritization system and start delaying transactions.
Furthermore, by relying on bandwidth throttling Nano can't show it's true potential and demonstrate its ability to scale to the world.
In theory a node could instead just store the hash of each accounts frontier block. Then when a new block is published it would have to be accompanied by the account's frontier block, which nodes can verify using the stored hash.
Although, currently this isn't possible because nodes need to store all blocks in order to allow other nodes to bootstrap by going through and verifying that all the blocks were valid.
If we had a way to ask the network to vote on the entire ledger at once then this validation during bootstrapping wouldn't be needed. In this case you could know you had the correct ledger with certainty without seeing the blocks. Hold that thought.
We can go even further than only storing the frontier hashes, and reduce the additional storage required per block to just two bits! We can do this using the Simplified Payment Verification (SPV) concept as described in section 8 of the Bitcoin whitepaper.
In Bitcoin, SPV actually doesn't have much use because SPV nodes cannot be sure that their chain is valid without trusting whomever gave it to them. In Nano however, a node can determine if a block is confirmed just by asking the network.
The problem with this idea is that Nano doesn't have a global blockchain which can be used to group transactions to build the Merkle trees needed for SPV.
We can solve all our problems by overlaying a pseudo global blockchain which I will call the "package chain". It sounds dramatic but really it's just an extra number associated with each block.
This addition has no impact on block confirmation, it's just a way of grouping blocks.
If all blocks belong to packages and the packages from a package chain, then you only need to ask the network to vote on the latest package to be certain that the entire ledger is confirmed.
This simplifies the bootstrapping process enormously.
I propose that instead of nodes storing the entire ledger, they instead package blocks, ship them off, and store only a package label.
The package label consists of the root of the package's Merkle tree and the previous package's hash.
Blocks are packaged at a fixed time interval T such that each package will be packaged at a particular known time.
Packaging has no effect on block confirmation. Confirmed but unpackaged blocks are still confirmed.
When a node publishes a new block it includes a proposed package number which is the index/height of the package that this block will be included in when it is confirmed.
A block will not be voted on by a node if the proposed package has already been packaged. So a block publisher must be careful to choose a package proposal far enough in the future so that the block will have enough time to confirm.
Sometimes a node will receive the final votes necessary to confirm a block once its proposed package has already been packaged. In this case it can simply be repackaged. A package is only finalized and ready to ship once the final block has been confirmed.
When the package is ready to ship, the package label can be voted on and confirmed, and the blocks that were in the package can be safely deleted. A websockets shipment notification containing the Merkle tree is made so that users can store a Merkle path to their frontier block.
When a user sends a new block they must accompany this block with their frontier block and its Merkle path so that nodes can verify that the frontier has been confirmed.
To validate this new block nodes must be sure that the frontier block is in fact truly the current frontier of the account. To track which blocks are frontiers, nodes store a bit array associated with each package. Each index in the bit array corresponds to the index of a block in the Merkle leaves. The value of the bit indicates whether the block is a frontier block. These bit arrays are updated as necessary, as new blocks get confirmed.
To validate a receive block the node must also know if the corresponding send block has already been received. Another bit array is used to track this. Each index of this bit array corresponds to the index of the send block in the Merkle leaves when all non-send blocks are filtered out.
The rate of ledger growth in bits becomes (512 + B + S)/T where T is the time interval between packages, B is the number of blocks in a package, and S is the number of send blocks in a package.
Each block adds at most 2 bits to the ledger. This means the effect that an increased transaction rate has on the ledger size is negligible. Spam no longer causes permanent harm to the security of the network.
The cost of spam is only the bandwidth and compute used by the nodes to confirm the block. Unlike ledger bloat, this is a one time cost to the network and can be completely reflected in a proof-of-work, i.e. the proof-of-work for a block can be made greater than the total cost to the network that block will ever cause.
This solves the ledger bloat problem and allows the network to focus on scaling up the CPS capacity as high as possible, showing the world that Nano is truly scalable. The much greater CPS capacity combined with existing prioritization methods will mean that even if spammers can still saturate the network it will have negligible effect and will cost them more. There is no longer any incentive to spam.
Users/wallets will need to ensure that their account chain and frontier Merkle path is stored somewhere. This is public data so can be stored in some cloud storage provider. Many wallets already provide a free proof-of-work service and it is likely they could automatically store their users blocks seamlessly, free of charge and without sign up required. It's also likely that there will be some archival nodes which do store the entire ledger on hard drives or tape storage, providing a backup.