When the potentially incentivized CarrIOTA Field nodes become a reality, is it possible (or does it make sense) to run multiple nodes on 1 machine? For example, spinning up a large cloud VM and then running as many nodes as possible on it.
Yes, it is possible. But you need to expose other ports on the same host. But I think it's not really useful. (If you don't have different IP addresses) I would recommend to have them on separated hosts and running other services on the same node. It just makes you way more fault-tolerant if you use separated hosts and racks.
It seems like I/O is a issue on full-nodes. So, smaller nodes with their own SSD/HDD should give you a better overall performance. I'm currently working on IRI/Nelson on Kubernetes with IPv6 only and ARM processors.
I tried running both Shimmer and Shimmer Testnet on the same server and failed. Separately, each worked, but both did not. I had different ports on each Node, that was not the problem. Docker was reporting a duplicate image name when running the second Node. Later, I had little time to delve deeper into the problem.