-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add EIP: Hardware and Bandwidth Recommendations for Validators and Full Nodes #9270
base: master
Are you sure you want to change the base?
Add EIP: Hardware and Bandwidth Recommendations for Validators and Full Nodes #9270
Conversation
File
|
- Statista states that as of January 2024: | ||
- The global average download for broadband is 92 Mbps and the global average upload is 43 Mpbs. | ||
- The global average download for mobile is 50 Mbps and 11 Mbps | ||
- GSMA report showing the state of mobile internet connectivity in 2024 shows that: | ||
- The mobile upload speeds in Higher Income Countries (HIC) is about 18 Mbps | ||
- The global average mobile download is 48 Mbps. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Page 39 for the upload speeds of HIC and page 5 for the 48 Mbps figure.
Statista link is: https://www.statista.com/statistics/896779/average-mobile-fixed-broadband-download-upload-speeds/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me know how you would want these to be referred to in the document
|
||
## Backwards Compatibility | ||
|
||
This EIP is informational and requires no protocol changes. We recommend that future EIPs include an assessment of their impact on these hardware requirements. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would the recommendation here require another EIP if this gets finalized?
In any case, I don't think it belongs here since this section is explicitly about whether the EIP breaks backwards compatibility
|
||
RAM/memory is dominated by state cache. As of January 2025, it is possible to run a full node with 16GB of RAM, however this has been known to not work with all combinations of EL and CL clients in the past. | ||
|
||
On 32GB vs 64GB; 32GB works right now, however we recommend 64GB as [preliminary benchmarks](https://hackmd.io/@han/bench-hash-in-snark) have shown that zk-STARKS can consume a significant amount of memory and the difference in cost relative to the entire hardware setup for a validator is insignificant. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw that EIPs 2926, 2938, 3298, 3416 and 3607 also used hackmd links however the contents of them can easily be changed, so I wonder if there is a rule about using them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given EIPs are immutable once they become final, does it make sense to place these requirements in an EIP? I expect requirements will need to be updated in a couple of years. In that case, we will need to create a different EIP amending this one. Then, operators will need to check 2 different EIPs to know what the current requirements are. Seems impractical to me.
good idea! |
In the call someone mentioned that, the status of this could be changed to "live" and we modify it in-place instead of creating a new EIP each time. I would defer to the EIP maintainers regarding that and whats possible there. I agree that creating a new EIP whenever we change specs is undesirable. |
@yorickdowne @potuz Thanks for the comments on hardware -- any comments on bandwidth? |
The commit fda25ca (as a parent of e2c2d40) contains errors. |
ethereum/EIPs#9270 spelling eigen cleanup
Made adjustments to make it a bit easier to read. Included information about how even if you use mev-boost you may need the same bandwidth as a local block builder under certain conditions. Included some additional details for choosing a NVMe drive (such as avoiding QLC and DRAMless drives). Added some information about what the PassMark CPU ratings translate into Added a CPU section Removed Statista as I don't believe it serves much of a purpose in the EIP, and I'm not sure how accurate these numbers really are when looking at the median. Added Quick Reference for those who just want the details
Update eip-7870.md
Co-authored-by: Justin Traglia <[email protected]>
Co-authored-by: Justin Traglia <[email protected]>
Co-authored-by: Justin Traglia <[email protected]>
General question to this one: Should we better treat that as "Recommended setup" or "Minimal setup"? Minimal would be then something we can refer to and adjust to it - having any users on our support lines which will be struggling with weaker setups could be then advised to think of any improvement and referred to this EIP. Also having "Minimal supported setup" in place we can adjust any benchmarking tools to it and then make a proper assumptions. |
|
||
Node operators typically run both an **Execution Layer (EL)** client and a **Consensus Layer (CL)** client on the same machine. The specifications below assume the combined resource usage of both. | ||
|
||
| Node Type | Storage | Memory | CPU Cores / Threads | **PassMark CPU Rating** | Bandwidth Download / Upload | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are all those numbers applied based on some research made? Or based on experience from discussions with stakers?
From Nethermind perspective those are super comfortable.
Also shall we distinguish EL and CL requirements to be treated separately? I saw a lot of setups where actually both are running on different machines.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the average user would likely run these on the same machine?
The rationale for them is in the linked hackmd in the comment box, we could not include all of those links so the EIP has been somewhat watered down
|
||
This EIP is informational and requires no protocol changes. We recommend that future EIPs include an assessment of their impact on these hardware recommendations. | ||
|
||
## Security Considerations |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One thing I'd suggest doing prior to applying that is to do any research among community either by pool on X or anything else which can have bigger outreach.
If it will appear that 51% of users have weaker than specified setup (which is super unlikely but still) we could potentially open a route for some unexpected attacks because we will test on too good hardware.
At least worth to know in case of noticed issue which affect weaker setups how much network (in terms of nodes) is put in danger.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could alternatively, test against the old NUCs for a year too though the current idea was that instead of us polling N validators, we would give them a tool to allow them to assess themselves and decide when they would want to update
This would be treated as "recommended" -- you can use weaker hardware but they would not be tested/benchmarked against is the thinking. The minimum is somewhat client specific, so I'd be hesitant to add that maintenance burden and it also would likely change more often per hardfork vs recommended due to the headroom. |
|
||
| Node Type | Storage | Memory | CPU Cores / Threads | **PassMark CPU Rating** | Bandwidth Download / Upload | | ||
| ----------------------- | --------- | ------ | ------------------- | ----------------------- | --------------------------- | | ||
| **Full Node** | 4 TB NVMe | 32 GB | 4c / 8t | ~1000 ST / 3000 MT | 50 Mbps / 15 Mbps | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Considering history expiry, 4TB sounds like a lot to recommend just for a full node. I see it's elaborated on below but I would either provide an estimation how long it will sustain with 2 or 4tb.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would also make it clear that these are recommended, not minimum viable specs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same goes for 32GB, I believe 16 is enough for different client pairs I have been running. Maybe 24 could be more realistic than 32
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO nice to work on 16 or even 8gb but this EIP is some commitment of supported hardware and some guidelines for users on what to use - I'd prefer users to use better hardware especially cost difference (especially for RAM) is like 50-100 bucks to upgrade from 16 to 32 and it gives a lot and makes us a bit more secure from some heavy blocks especially if we consider an increase of their size.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Considering history expiry, 4TB sounds like a lot to recommend just for a full node. I see it's elaborated on below but I would either provide an estimation how long it will sustain with 2 or 4tb.
Yeah, we recognize that 2TB will be do-able with history expiry, but it's uncertain when that will actually ship. I would prefer the EIP's recommendations be on the safe side, 4TB.
I would also make it clear that these are recommended, not minimum viable specs
The word "recommendation" is used several times in this doc. It's even in the title:
Line 3 in 69d6453
title: Hardware and Bandwidth Recommendations for Validators and Full Nodes |
Same goes for 32GB, I believe 16 is enough for different client pairs I have been running. Maybe 24 could be more realistic than 32
Yes, this 16GB is enough for today (and what my validator has) but might not be enough 3 years from now. Once again, this EIP does not state that you cannot run a full-node with less, just that 32GB is recommended.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Setting minimum supported hardware for me is super beneficial and already mentioned that few comments above.
Will make it clear under which setups we can't put our signature and which ones we don't believe are good enough.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, minimum hardware requirements would be useful too, but it's a much more difficult problem. I wouldn't expect the minimum for all clients to be the same. In my opinion, minimum requirements are best left to client teams. For example, the big advantage of Nimbus is that it works very well with low-resource hardware. I believe the current recommendations will work for all clients for at least the next three years.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hardware will be much better in next 3 years and currently recommended will probably be much cheaper, so it will be even cheaper than some super cheap ones on which any client barely works now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand it's much harder to determine what are minimum specs and also harder to sustain so I don't believe it needs to part of the table. However settling now on this much room in recommended also makes me think devs might get too comfortable with the extra performance and it might become the new minimum. I believe we are building also for people with less hardware options.
@kamilchodola I wouldn't be so sure about significant change in the hardware getting cheaper. That assumes people need to upgrade to latest hardware, their existing nodes are not going to get stronger, only rustier. And I am not very bullish on hardware getting cheaper in general, last few years are proving otherwise and although inflation is slowing down, the progress in hardware is much smaller than it used to be, Moore's law is dead and we cannot count on getting twice as performance in couple of years. Current hardware generations offer rather incremental improvements for high prices
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we planned to prune pre-merge data in the first half of the year, which is roughly half a terrabyte.
I got this number by computing the total size of all of the files here: https://era1.ethportal.net/
Total size: 458,497,023 kB
= 447,751.00 MB
= 437.26 GB
If we use the history growth numbers from paradigm as an approximation, this buys us roughly a year: https://www.paradigm.xyz/2024/05/how-to-raise-the-gas-limit-2
For full history expiry, the storage occupied will be constant, though I think its still uncertain as to when that officially lands in most clients. If it lands in a few months, I think we can change this number for full nodes and validators -- most validators AFAIK have already updated to 4TB, ethstakers for example recommend this for new validators: https://ethstaker.org/staking-hardware
|
||
- **Why 64 GB for Validators (Attester / Local Block Builder)?** | ||
- Running an Ethereum validator (both EL & CL clients) can be memory-intensive, with state cache dominating RAM usage. | ||
- Certain memory intensive client combinations have historically failed with 16 GB. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's worth adding that although some pairs might have trouble with 16gb by default, it's possible to tune cache size in the client to make it work
hey @kevaundray , if you could get the linter/bots to pass, we can merge this PR and then you can continually update this basis discussion and feedback |
@@ -0,0 +1,145 @@ | |||
--- | |||
eip: 7870 | |||
title: Hardware and Bandwidth Recommendations for Validators and Full Nodes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the EIP title, I think we should remove "for validators and full nodes" for the following reasons:
- The current name is too long. The max is 44 characters, this is 68. With that part it is 38 characters.
- It's sort of implied that we're talking about validators & full nodes. But I suppose we could add recommendations for light clients too. A somewhat ambiguous title would allow us to do that.
title: Hardware and Bandwidth Recommendations for Validators and Full Nodes | |
title: Hardware and Bandwidth Recommendations |
Original documents are here (for validators):