Although it is pretty technical stuff, this is the document you are likely going to want to study:
http://espressif.com/sites/default/file ... ual_en.pdf
It is called the ESP32 Technical Reference Manual. Specifically, look at Figure 6 on page 28. From there we see that there are two "mapping" elements. A thing called the IO_MUX and a thing called the "GPIO matrix". From this I believe that there are "logically" 256 distinct peripheral control signals that can be mapped to 40 physical pads (pins). Of the possible 40 pads (numbered 0-39), five are not exposed (24, 28, 29, 30, 31) giving is 35 specific pads (pins).
Now if we look at table 16 on page 34, we see the 256 possible signals ... which for example include 16 in (U0RXD) and 16 out (U0TXD). Great ... that then seems to say that I can use the matrix to map signal 16 in to an arbitrary input pad and 16 out to an arbitrary output pad. But then things take a turn for the "ahh" ... when we look at table 17 on page 38. This seems to show that U0RXD is tied to pad 3 while U0TXD is tied to pad 1. So how can we explain the contradiction? Are the functions to pins fixed or not?
And unfortunately I don't have enough knowledge/skills to answer that ... but I'll stick my neck out and make a guess in the hope that better folks than me can correct.
I think that in the ESP32 there are two types of function/pad mapping. A matrix mapping using a GPIO matrix and a "faster" direct IO mapping known as "IO_MUX". My belief is that one CAN map any function to any pad using only matrix mapping. However, should one wish/need, one can bypass matrix mapping and expose functions on very specific pins to get "maximum" performance/throughput.
Again ... making it up as I go along.... you can map U0RXD and U0TXD to arbitrary pins using GPIO matrix ... however somehow "performance" or "quality" will be less than using "direct IO" through IO_MUX which forces particular functions to specific pins.
You will see I am being VERY woolly on the concept of "why" one would use matrix mapping vs direct IO.... and I have no idea. I don't know what the benefits may be in one fashion or another. Hopefully this post can be used as a basis to say "yeah/neigh" to some concepts and ideas ....