As suggested by ethernaut in https://twitter.com/the_ethernaut/status/1361489841519796225 the compiler should allow contracts to be split up more easily to avoid having to implement a proxy pattern manually. In my opinion, this is a good idea in principle, but requires a lot of details to be worked out.
Calls between two “fragments” of a contract will be expensive and create an external call, which is also problematic with regards to call protection. Maybe it would be better to explicitly group parts of a contract into fragments / modules that then have to use external calls (maybe delegatecalls) between each other.
We also have to be careful that this still works for how people currently upgrade their contracts.
I’ll start by stating my opinions on the topic. I believe that the 24kb restriction should be removed from EVM rather than it being a useless rushed restriction that thousands of end users have to work around. I was pushing to get rid of it for a while but then lost motivation. If anyone else wants to lead the charge, I’ll be happy to help. There’s some useful and relevant content on
youtube dot com/watch?v=5W33u2OS92Q
Coming back to the language design, one way I think we can achieve this is by introducing contract level scopes or sub contracts and using delegatecalls to call into each other. The main contract should act as a static router that dispatches call to different sub contracts. All storages defined in sub contracts must be aggregated and assigned slots in the main router contract. A sample contract can then look like (psuedo code):
contract Main {
uint256 A;
function B() {
Foo.J(); // Delegate call
Bar.T(); // Delegate call
C() // Internal call
A = Foo.storage.I; // Direct read, write
}
function C() {}
contract Foo {
uint256 I;
function J() {
Main.C() // Delegate call
Bar.T() // Delegate call
K() // Internal call
I = Bar.storage.S; // Direct read, write
}
function K() {}
}
contract Bar {
uint256 S
function T() { }
}
}
The syntax can be made less/more explicit but I think this conveys the basic idea. Basically, it puts the developer in charge of defining scopes rather than compiler making a guess. Keep in mind that the router will still eventually hit the 24 kb limit but that’ll take a long while.
So yes, there are multiple areas of slight awkwardness if all this is not handled at a lower level:
Storage: Namespaces are nice, but it means that “natural” contract slots are not used at all
Module calls: As stated above, expensive and with varying degrees of weird syntax
Router: Having selectors ranged between a set of numbers for each contract would be really helpful to completely avoid the need of a binary search in the router
This looks rather logical and straightforward. I assume that all externally visible function of the sub-contract are visible in the main contract (and proxied through). I think for the sake of readability, there should be something like contract MainContract { import SubContract; }.
Let me brainstorm a bit more: What could also be interesting is to make sub-contracts and especially their construction more explicit:
The imported functions can then be called directly like A() while unimported functions will need to be called explicitly like Foo.C().
If you do not use immutable , then the target pointer can be updated.
Just to be clear, if immutable is not used, the address should be hardcoded in the bytecode like it’s done for libraries so that no storage read/writes are required.
One problem with dynamic proxies is that we do not know which functions it has, so it is difficult to do this with more than one sub-contract.
Why do we care about that? The router should use the subcontract provided at compile time to create the routing table and then It will just throw an error in runtime if the user tries to use a sub contract with a different interface. Similar to how contract objects work right now.
FWIW I think this needs a lot more thought but I believe we are on the right track.
“to be visible in MainContract” - I actually meant “externally visible in MainContract”, i.e. for callers of MainContract’s instances.
And to clarify the immutable part:
My point was that if you use export immutable SubContract subContract, then you can actually get the address of the sub contract. You can call functions using subContract.f() and you can even have multiple instances of the same sub-contract - not sure if that is useful, though. The immutable keyword means that the address is stored in code and cannot be changed after deployment. Furthermore, this notation allows you to provide constructor arguments for the sub-contract, because its creation is explicit.
As an extension, this allows the address to be stored in storage if the immutable keyword is not present. This has the benefit that the address can be changed if you like, for example for a code upgrade. This also means that the actual type of the contract is unknown at compile-time, and it can even changed during the life-time of the contract, as long as the interface is still compatible.
“to be visible in MainContract” - I actually meant “externally visible in MainContract”, i.e. for callers of MainContract’s instances.
IMO public/external functions should always be visible. I don’t see any use of export keyword here just like I don’t see any use of payable addresses or calldata keyword in external functions.
The immutable keyword means that the address is stored in code and cannot be changed after deployment.
ah right, I read that as mutable for some reason. Sorry for the confusion. It makes sense now lol.
It seems to me that we’re discussing the potential syntax of a better router contract, i.e. a contract that can be used to associate a set of modules in the same context. I’d love this of course, anything that improves what we currently have to do to achieve the same effect is awesome!
I’m just wondering if we should broaden the scope of the conversation. Imagine that we had an ideal solution to whats been discussed above. Developers will still need to code around a certain pattern (i.e. inter modular communications, storage in namespaces, etc), and coding around the pattern would deviate from the natural code style of the language.
So, I ask the question? Shouldn’t the language output artifacts that abstract this routing all together? I.e. if it finds that a particular artifact will exceed the contract size limitation, then it will split it in two and take care of the routing under the hood. That way, users of the language can maintain whatever code style they choose, and still be able to overcome the size limitation, or other limitations that may be imposed by the EVM.
Of course, this is just food for thought, I have no idea how complex this would be for the compiler, or if it would be possible at all.
It’s very hard to abstract the routing completely. It’s easy In basic cases where the contract can be split in shards that do not interact with each other. However, when the functions start calling each other, it’s very hard for the compiler to optimize the sharding strategy. Also, the internal functions can’t be called directly across shards so there will always remain a difference in coding a single contract and a sharded contract.
The earlier proposal does force people to adapt to a new pattern but it doesn’t seem out of place to me. The syntax we were discussing looks quite natural (apart from exports ). In any case, here’s a simpler example that can achieve the same goal while keeping bloat at minimum.
contract Main {
uint256 A;
function B() {
J(); // Delegate call
T(); // Delegate call
C() // Internal call
A = I; // Direct read, write
}
function C() {}
{
uint256 I;
function J() {
C() // Delegate call
T() // Delegate call
K() // Internal call
I = S; // Direct read, write
}
function K() {}
}
{
uint256 S;
function T() { }
}
}
Basically, this rips out namespaces and says everything must roll up to the main contract. The scopes only define what functions end up in which shard. I think this achieves 90% of what we want while still being relatively simple to understand, implement and use. tbh I’d prefer the earlier featureful solution but I can’t deny that having a simpler approach has its own benefits. If anyone who is reading this has any other ideas, feel free to share .
It’s easy In basic cases where the contract can be split in shards that do not interact with each other.
This already seems quite useful in a lot of cases & as you say doesn’t sound too difficult to implement. I bet there are a lot of folks who have contracts which could effectively be auto-sharded but that still struggle with contract code size.
Also really appreciate this discussion! This would be an awesome feature.
@maxsam4 One thing that strikes me in your initial example is that nested libraries match your concept much more than nested contracts:
Your nested contracts can call each other’s functions even though they don’t inherit from each other.
External calls work via DELEGATECALL.
They don’t have their own storage and only use storage held by another contract.
Though unlike libraries, they can access storage variables directly. I actually don’t think it would be possible without an external call.
Calls qualified with the contract name (e.g. Bar.T()) are external. This is already the case now when Bar is a library. If it’s a contract, it’s an internal call.
Just like in case of libraries you’d probably want a call guard. You don’t want someone to call it directly by mistake and use the storage that belongs to it rather than to the router.
Another thing is that the way storage would work needs some adjustment. I think that the common pattern with upgradeability is that you create a proxy which holds all the storage variables and in newer versions of your contract you only append new ones at the end. If we assume that with nested contracts the compiler would visits them in some deterministic order and add their storage variables to the router in that order then it would be too easy for users to break upgradeability by adding add a new variable in the middle of a block of existing variables. In fact, the only way not to break the contract on upgrade would be to only ever add new variables in the last nested contract (or in the router contract if we assume that variables placed after all nested contracts are placed last).
I see three ways to solve it (and there are probably more):
Require placing the variables directly in the router contract. This would be yet another restriction common with libraries.
Place variables from nested contracts in something that works like a mapping and mirrors the nested contract structure. One improvement over an actual mapping would be that the compiler, knowing all possible keys at all nesting levels, could make all accesses O(1).
Have user state the maximum number of slots each nested contract can use (only counting the static part that is normally placed at the beginning of storage). Then the compiler would pad it to that length or issue an error if it’s taking too much space. As long as the numbers are never changed once set, it would be safe to add new variables inside nested contracts.
Place variables from nested contracts in something that works like a mapping and mirrors the nested contract structure. One improvement over an actual mapping would be that the compiler, knowing all possible keys at all nesting levels, could make all accesses O(1).
I like this idea.
One thing that strikes me in your initial example is that nested libraries match your concept much more than nested contracts
I agree with that but contracts and libraries are technically the same thing. We can call them whatever we want :). Heck we can introduce a third keyword like “SubContract”/“Scope” etc. I am not good at naming things though.
I agree with that but contracts and libraries are technically the same thing. We can call them whatever we want :). Heck we can introduce a third keyword like “SubContract”/“Scope” etc. I am not good at naming things though.
I mean, it was not just a pedantic point about the naming What I’m getting at is that what you propose looks like a radical change on the surface but if we just made a relatively small change to libraries to allow to have an immutable state variable that represents a single contract they’re associated with, we would already have 90% of this feature. Libraries have almost the exact semantics you’re proposing. The only things missing would be the neat syntax for nesting them in the contract body and declaring some of the contract’s state variables inside them.
It would be nice if we could find a solution to make the fallback function obsolete and at the same time does not need too many changes, i.e. re-uses the library concept. So yeah, if you can come up with a complete proposal, then please go ahead! I also appreciate this coming from people who more often use proxy patterns than the compiler developers do
I haven’t seen discussed how a split contract would be deployed. Would it be a single transaction or multiple transactions? In either case what should the creation code look like? Maybe the library linking mechanism should be reused?
I don’t think it is necessary to implement splitting large contracts at the Solidity language level. The reason is because EIP-2535 Diamonds solves the problem in a sufficiently elegant way.
I think what could help at the Solidity language level is special support for state variable layout for proxies, because the default state variable layout in Solidity doesn’t work for proxies with multiple facets/implementation contracts. Diamond Storage and AppStorage are solutions that are easy to use, but it would be good to have the state variable layout work more at the language level.
EIP-2535 is a mechanism and smart contract pattern that splits a large smart contract into multiple smaller ones, but they are accessible at a single Ethereum address.
Since there is already a standard that does this I think Solidity language features that makes it easier to implement EIP-2535 would be very helpful.
). In any case, here’s a simpler example that can achieve the same goal while keeping bloat at minimum.
.