Allowing immutables of non-value types is being considered for the future. A general implementation would probably store the data in the data area of a contract and use codecopy to make it useable in memory. This would work for structs and even for dynamically-sized arrays.
But some of the details are not entirely clear, yet.
While it would be possible to copy an entire immutable of reference-type to memory using codecopy, you’d probably usually only want to access a single struct member or array element at a time without needing a full memory copy.
While statically-sized immutable structs or arrays could just be syntactic sugar for a number of value-type immutable (i.e. any member- or index-access to them would result in a placeholder “PUSH” in the bytecode to be filled by the corresponding value at construction time just as for value types), this won’t work for dynamically-sized reference types anymore.
We’d need to refer to those using some kind of “reference to data at some offset in the bytecode” and then successively use codecopy to read e.g. the size of a dynamic array or the code offset of nested arrays.
The gas costs and benefits of this would probably need to be evaluated on real-world examples first.
There has also been the idea to introduce a new data location called code (besides calldata, memory and storage) for this instead of calling them immutable as well (this might for example look more consistent when passing a reference to data stored in code to functions), but there is no consensus on it yet.
We did introduce calldata array slices for precisely this reason - but, unfortunately, the ABI encoding does not allow slicing of nested calldata arrays (since arrays of arrays in calldata use offsets relative to the “base pointer”, i.e. the start of the containing array, these offsets would become invalid in slices, unless we drag along the base pointer, which creates too much overhead).
Having .pop in calldata arrays is an interesting idea, though. I think, since calldata is inherently read-only, this has not been considered so far, but given that the length of a dynamic calldata array is stored on stack, introducing .pop would still be possible and basically yield an alternative way of “one-sided” slicing (and since the base-pointer would stay the same, this would also work for nested calldata arrays). So thank you, we will consider that!
Furthermore, there have been several ideas and suggestions for conveniently packing and unpacking data in general, which could be applied to packing to and unpacking from calldata, but we have not yet arrived at consensus about a final design.
So yes, we are aware that there is room for improvement in this area, but are still in the process of trying to find suitable solutions.
In case you have any concrete suggestions for this or if you have specific limitations or missing features in mind that we may have missed, it would definitely help, if you open github issues for them!
Point 1. Calldata parameters in external functions (cheaper compared to memory arguments).
even in 0.6.8, this already works as expected. we can specify calldata parameters as arguments instead of memory and they will be cheaper. So I don’t quite get your statement about this.
Point 2.
Assertion failures, out-of-bounds errors, overflow/underflow errors, etc. don’t eat up all the remaining gas in your transaction
If one uses safeMath , gas doesn’t get eaten up when overflow/underflow happens. I guess, what you meant is using assert doesn’t eat all the gas and almost behaves like require ? If that’s so, there’s not so much difference between assert and require anymore other than the fact that assert is just to use where code is not supposed to get there. do you agree in all that I said ?
Point 3 Free functions and file-level constants.
could you somehow put a link of the docs where explanation about this is explained ?
I’d appreciate your insights on this … Thanks a lot again.
Sorry, my bad. The support for calldata parameters was being extended over the course of several versions so I might be mixing up what was added exactly when. I said external functions but I just checked and indeed they already worked in 0.6.8. The change in 0.6.9 allowed you to use calldata everywhere so also in internal or public functions and for return values. After that, in 0.7.x there were several changes that fixed/added cases like calldata structs or slicing of calldata arrays. There were also multiple bugfixes. You’ll find the details in the changelog. So I guess it depends whether the limited support for calldata in 0.6.8 is enough for you or if you need one of the cases that were not there.
Point 2.
Right. SafeMath uses require so it does not eat up all gas and before 0.8.x there was no built-in safe math so it would not eat your gas either. So it’s mostly about assertions, out-of-bounds errors, bad enum conversions, etc. I listed underflows/overflows because they are classified as the same kind of error now but yeah, it’s not much of a change compared to SafeMath in that regard.
If that’s so, there’s not so much difference between assert and require anymore other than the fact that assert is just to use where code is not supposed to get there. do you agree in all that I said ?
Well, there are still differences. You can’t supply a custom message for a Panic while you can for revert’s Error. In the new custom error feature that we’re currently implementing you can even define your own errors and give them arbitrary parameters. But yeah, both reverts and assertions are now returning stuff even if it’s encoded a bit differently and you can catch panics just like errors.
The semantic distinction is still useful though. You want require/revert for validation while assert is for stuff that’s not meant to happen if your contract is not buggy.
Point 3.
Sure. There’s a bit about them in Structure of a Contract > Functions. But their semantics are very similar to internal library functions so there’s not much to write about. But if you think something particular is missing, please open an issue on github and we’ll fill the gap
@cameel Thanks for the follow up. much Appreciated .
Point 1.
So it seems like that you’re saying I can specify calldata in the public functions. What if the public function gets called from the same contract ? does it mean that copying into memory is not necessary anymore at all and the both of the functions can just use the same data from the same calldata ?
Point 2:
I guess, the only difference between assert/require is a semantic distinction. One of the difference where assert was eating all gas is already removed in the new versions. Also, one can add custom messages to assert . So truly, I can’t see any other difference other than just semantic one which just is better for programmers to think and look at the code. If they see assert, then they immediatelly know that the program should never end up here.
That’s just my observation. do you think the same ? and it’s a good thing you made it so that assert doesn’t eat all the gas.
Exactly. You can pass calldata values down to your internal functions without forcing the extra decoding into memory.
Not really. assert() only accepts a condition. You cannot specify a message. In 0.8.x there’s an error code though. See Panic via assert and Error via require in the docs.
The semantic difference is actually pretty significant when it comes to formal verification:
Solidity implements a formal verification approach based on SMT solving. The SMTChecker module automatically tries to prove that the code satisfies the specification given by require/assert statements. That is, it considers require statements as assumptions and tries to prove that the conditions inside assert statements are always true. If an assertion failure is found, a counterexample is given to the user, showing how the assertion can be violated.
Lately assert() became a lot of closer to require() in how it works under the hood but syntactically they’re likely to diverge in the future. For one, we’ve long been considering a more formal support for invariants and they’d be another way to assert if enforced at runtime. At the same time we’re currently replacing revert() with a revert statement (with syntax similar to emit) to make it more apparent that custom errors are not function calls and we might also change require() syntax to match it (though to be honest we have no consensus about how it should look like and it might turn out that it actually stays as is in the end).
I’m curious about your impression of the current state of 3rd party tooling around Solidity and more generally smart contract development, e.g., editor support, debuggers, linters, etc. Are you mostly happy or do you think it is lacking? Do you have any favorite tools, do you miss anything in particular? Is there a lot of exchange between the Solidity team and third party teams or is everyone mostly working on their own?
I’m deeply grateful for the tooling teams putting up with our frequent breaking changes. The space is certainly still lacking a lot and we need better support from the Solidity compiler directly. We are in regular contact with all the tooling developers and we try to get feedback before making bigger changes that could break assumptions they might have about how the compiler behaves.
We are currently developing a language server that we hope will spark the development of new tools. It will be useful mostly for editors and auditing, instead of debugging because the protocol is focused around that area.
I think that the big amount of existing third-party tooling is a great strength of the Ethereum ecosystem in general. For everything you need you’re likely to find several different tools, which is very healthy and gives you choice. Many of these tools are open source and, personally, I’m pretty happy seeing what is already available.
It’s actually hard to give an example of something that has not already been implemented in one way or another. Maybe a linter for Yul? Chris Parpart did take a stab at writing one but it ended up being a bigger task than we anticipated and it was abandoned. Another, very general idea would be some tool to make debugging contracts easier. There are some good debuggers out there but this is still a common pain point for users and there is probably some room for improvement, both for the existing tools and for the compiler in how it supports them.
That being said, the ecosystem is still relatively young and not all the tools are already mature. Solidity itself is evolving rapidly which we know can be a challenge to keep up with. To help with that we provide nigthly builds of the compiler that anyone can use to see how the newly added features will impact their project. There’s also an ongoing initiative to integrate some of the devtools’ test suites into our CI to get early feedback about breakage. Every team works separately but definitely not in isolation and we’re trying to improve the flow of information in both directions.
Overall, I’d encourage any tool developers to reach out and share their needs regarding interfacing with the compiler. We’re in regular contact with people working on the most popular frameworks and libraries but we care about your feedback even if the project is small and just starting out. There’s never too much of such feedback. Even though we get a lot of it already, not everyone cares about every feature and it can sometimes be hard to get enough information on how some specific change will impact other projects.
There’s one question that I forgot to ask on 10th March and I will try to explain it here. Hope I can hear your opinions on this. It’s not related to solidity, but still, wanted to ask and appreciate your input on this very much.
My contract has 2 functions.
function1 and function2
Users first call function1, they pass a huge struct as an argument (one of the types in the struct is Action[]) array.
Users can only call function2 on the same struct if they called function1 on it previously. function2 unfortunatelly uses much more gas than the function1 which gives us the following problem.
User can call function1 with the array of Action[] and each of the data, he can specify huge calldata. By calculating a little bit, what he can do is he can call function1 on that struct, and then he will never be able to call function2 due to the fact that function2 uses much more gas and if the Action[] is huge and if each of the bytes data in the action is huge, the gas cost of function2 grows unproportionaly, hence exceeds the block gas limit.
Other than ipfs solution which means that I don’t pass Action[] to the struct and only pass the hash of the ipfs, I’ve been thinking of limiting the length of the Action[] (100) let’s say. But this is not enough, as in attacker can also pass huge bytes data for each Action. So we need to limit the length of each bytes data.
It seems like we limit the Action[] length and we loop through it and check if each of the bytes data length is below some number.
Question 1: What do you think about the above solution ?
Question 2: I don’t know what length limit to take for each bytes data. If I take 1000, it means each action’s bytes data will only be able to call a function that has 1000/32 = 31 elements which is too low, any function can have such arguments because their argument would be an array.
