What is the typical network transaction amplification rate?

Question

With IOTA, every neighbor, either directly or by request, is required to share the transactions it receives with its neighbors in order to ensure that the entire network sees all transactions eventually.

1) Given that every IRI node has 8 neighbours, for a 1000 TPS network transaction rate, what is the actual number of transactions that will need to be transmitted within the network of all nodes (suppose a number of nodes, perhaps anywhere from 1,000 nodes up to 10,000 nodes worldwide)?

2) Given that a transaction is approximately 1.6K bytes, how many bytes will be transmitted by the entire network (usage of worldwide internet quota) for every actual transaction created on the network?

Example

If a network of 4 nodes received one transaction and shared it with each neighbor using a simple 'broadcast to all neighbors' protocol and the origin of the transaction is not broadcast to again from any recipient, the amplification might look like this:

Nodes = [n0,n1,n2,n3]
A single transaction is created and sent to node n0.

n0 -> sends to n1 and n2 and n3
n1 -> sends to n2 and n3
n2 -> sends to n1 and n3
n3 -> sends to n1 and n2

In this example above: 
 - the network amplification is 9 times
 - 14.4K bytes of data is shared

Important details

With the IRI, nodes share with some neighbors but not all due to the p_propogate variable. They also request missing transactions from the network. They might also send some back to the origin (original sender of received transaction). There is also code in the IRI designed to randomly drop received transactions and/or to randomly thwart the data from being processed. You will need to check the IRI to see what actually happens. So those probabilities wired into the IRI need to be taken into consideration.

With the IRI, the quantity (TPS) and payload (bytes sent/received) amplifications would need to include that of all transactions broadcast (gossiped) AND that of all transactions requested.

I will award a 50 point bounty if you also include the quantity and payload of all the missing data requests being sent.

Recap

We are looking to find out the total number of transactions per second that the entire network must handle and also the bytes for all those transactions, given that the sharing is exactly as specified in the current IRI reference software and that network load is 1000 TPS and given that each node has exactly 8 neighbors and that each transaction uses exactly 1.6K bytes of internet bandwidth.

Answer 1

I just did a simple software simulation with 10,000 nodes and 8 neighbors each with plain send-to-all-peers gossip and no sending to the origin of the message.

The node neighboring was randomized to ensure that all nodes had a set of randomly mutually-tethered peers. The specification was for 8 neighbors with an even distribution from a min of 5 to a max of 11.

The effects of 1 new transaction in the network produces:

• Total transactions propagated between all nodes: 89,949 (min: 89,145, max: 90,485)
• Average number of nodes that missed being sent the transaction: 0
• Number of gossip generations to permeate the network: 7 on average. Rarely 6.
• Total bytes transmitted: 143.9 MB
• Nodes received 9 messages on average and sent 9 messages on average.

(All numbers are the average after 100 repeated simulations.)

Notes

0) Each node had to handle an IO Bandwidth load of 18 messages per new network spend. That means that each node used 18 times 1.6KB (transaction raw payload) for each new network transaction. That means there was 28.8 KB of internet traffic per node per new transaction.

1) In reality, since the IRI suppresses some gossips and also uses a "request-from-peers" for missing transactions, the actual rates would increase quite a bit more.

2) If this changes to 1000 TPS, then the number of actual transactions that the entire network must handle is 89,949,000 per second with the transmitting of 143.918 GB of data per second.

3) I haven't added in all the P_Probabilities in the IRI yet. I will eventually.

4) The amplification at the 'node level' for 8 neighbors was 18 times. Since it was 28.8 KB for a single new transaction, then it becomes 28.8 MB per second if the network does 1,000 TPS. Every node must perform at that level.

I look forward to investigating what happens if some nodes fall behind and can not match that requirement.

5) As Come-From-Beyond commented on my original question, there is also the issue of TCP or UDP (see chat). The actual TCP/IP stack is not being investigated in this. Nevertheless, either of those protocols add additional internet load from packetization and delivery techniques. This answer just looks at raw simple 'per new transaction' data as the actual size if each raw packet.

6) The mutual-tethering was specified at 8 with a min of 5 and max of 11. I am interested to see the effects if the distribution is changed from a normal to a long-tail with most modes having a recommended setup and yet allowing for a small set of nodes to have many more peers, say 25 to 50.

7) What is the atmospheric CO2 impact of a single transaction? Using a "datacentre" estimate of 1GB -> 1kg CO2 from The Carbon Footprint of Your Online Habit, the CO2 generated per second at 1,000 TPS is 143 KG. At the level of VISA's capacity, say 50,000 TPS, with only 10,000 nodes, there would be 7,150 KG of CO2 generated per second and with 1,000,000 nodes, there would be 700,150 KG of CO2 generated per second.

Waxing philosophical for a second and a bit of a tangent

But you say, "Moores Law" will reduce the CO2 per transaction". TRUE. But also Moores Law will increase throughput and the number of IoT devices. The result of a mass scaling of IoT on the IOTA network would mean, say 1M nodes and 1M TPS (conservative?). Then the CO2 would go to the order of 700 million KG of CO2 per second. This is a guesstimate of course. However, if MAM payloads accompany the transactions, then it could be even more than that guestimate.

That all assumes that 1GB -> 1kg of CO2 is reasonable. It seems much too high to me. So what is a reasonable figure that everyone agrees with?

Nano-payments are the core feature of IOTA and if as said there will be "billions" of devices, then what prevents the network from growing to 10 million nodes with billions of devices and, say, 10 million TPS constantly? With an accompanying CO2 load of, say, 70 billion KG of CO2 per second? It is something to think about, no?

8) What plays a bigger role, the number of nodes, the peering configurations, or the TPS? So far, tests show that the increase in number of peers is linear. So for a change from 10,000 peers to 20,000 peers, all the numbers double.

Stay tuned!

---

Follow-up:

• A few better-peered nodes can make a big difference in network dynamics. If 3 neighbors is chosen and a long-tail distribution is chosen whereby 1% of the peers are assigned a random up to 105 neighbors (upper limit), and where 4% of peers are assigned a random up to 55 neighbors (upper limit), then the avg sent is 40K and average rounds is 9.

• PEERS: 10,000, NEIGHBORS SPEC: 3 (min 2, max 4) Average sent: 25,020 Lowest: 24,755, highest: 25,251 Average rounds to permeate entire network: 13 Average skipped: 0 Avg tx received: 2.5, Avg tx sent: 2.5

• PEERS: 10,000, NEIGHBORS SPEC: 5 (min 3, max 7) Average sent: 50,007 Lowest: 49,713, highest: 50,477 Average rounds to permeate entire network: 8 Average skipped: 0 Avg tx received: 5, Avg tx sent: 5