Some of the chips have a bios that does not allow the flash mode 'QIO'. Changing it to 'QOUT' fixed it for all of them.

Introduction: ESP32 is already integrated antenna and RF balun, power amplifier, low-noise amplifiers, filters, and power management module. The entire solution takes up the least amount of printed circuit board area. This board is used with 2.4 GHz dual-mode Wi-Fi and Bluetooth chips by TSMC 40nm low power technology, power and RF properties best, which is safe, reliable, and scalable to a variety of applications. Features: High performance-price ratio Small volume, easily embeded to other products Strong function with support LWIP protocol, Freertos Supporting three modes: AP, STA, and AP+STA Supporting Lua program, easily to develop